melreams.com

Nerrrrd

What does being an experienced developer actually mean?

in career, development, hiring

Unrelated image from pexels.com to make this post look nicer in social media shares.

I was poking around in /r/javahelp, saw a question about something I haven’t personally done, and found what I strongly suspect was the answer right away.  I try to explain how I found the answer when I answer questions like that because “oh it’s _____” isn’t really that helpful in the long term. I mean, what is the question asker supposed to do next time if I’m busy or sick or on vacation?

But how do you explain “well obviously that’s a_____ problem”? I can explain that I googled a thing and didn’t get very helpful results so I googled another thing and found what I needed, but not how I knew what to search for in the first place.

The closest I can come to explaining how I recognize problems is a term I recently learned from Katrina Owen‘s talk Cultivating Instinct. That term is perceptual learning, which according to Wikipedia is:

learning better perception skills such as differentiating two musical tones from one another or categorizations of spatial and temporal patterns relevant to real-world expertise as in reading, seeing relations among chess pieces, knowing whether or not an X-ray image shows a tumor.

After you’ve worked with enough code and seen enough different kinds of problems, you just kind of know what the answer is (or at least a few things it could be) when you see a new problem. And that, finally, is what being an experienced programmer really means. It’s not about some magical level of skill or about instantly knowing the perfect solution for every problem or even finally feeling like you know what you’re doing, it’s just about having seen enough problems that you end up with a library of them in your head.

I can’t tell you how long that will take but on the upside I can tell you it will definitely happen if you just keep coding. Okay, there are people with “ten years of experience” who really just have one year ten times, but if you care enough about programming to bother reading my blog, you’re not one of them.

Another part of being an experienced programmer is that once you feel confident about getting your code to do what you wanted, you sort of move up a level and start worrying about whether it’s a good idea to do it that way. That’s where people with one year of experience ten times really fall down, they can make things work but can’t make them maintainable. Getting to the point where you worry about the right way to do things takes a little more deliberate effort, but the most important step is just to care about whether it’s possible to change your code.

Once you’re past struggling to get your for loops to work and once you’re past feeling totally overwhelmed by starting a new project, it’s really tempting to think that you’re done, you’re a “real” programmer now and there’s nothing else you need to learn. But if you stop there, not only will you never be a very good programmer, but more importantly you’ll miss out on one of my favourite parts of programming, which is figuring out how to design things so you won’t curse your name in six months when you need to change them. For me that’s way more fun than Strings and Arrays.

All this is to say that if you’re a beginner programmer and you see a job posting looking for developers with x years of experience, all they really want is someone who can reliably get things to work (not necessarily immediately) and who knows a bad idea when they see one. If you can do that, go ahead and apply. It’s not unusual for job postings to be more of a wishlist than a list of what the job actually requires, anyway :)

Be a better programmer while still having a life: part 8

in becoming a better programmer, development, opinion, productivity, unsolicited advice

The big tip for post #8 in the be a better programmer while still having a life series is to become a witch. A Terry Pratchett style witch, to be precise. Terry Pratchett’s witch characters are really great at two things: first sight and second thoughts. To quote him directly:

First Sight and Second Thoughts, that’s what a witch had to rely on: First Sight to see what’s really there, and Second Thoughts to watch the First Thoughts to check that they were thinking right.

And no, I’m by no means the first person to connect Pratchett-style witchery with programming or design. Hint: go read that blog post, it’s really good.

Back at my original point, first sight is seeing what’s actually there, not what you wish was there or what you thought was there or what you meant to put there. Does that remind anyone else of debugging?

Fortunately for programmers, we have tools like debuggers and IDEs to help us see what’s actually there. We also have techniques like simply getting up and taking a walk, or explaining our problem to a rubber duck (or maybe another programmer if it’s a really hard problem), or commenting out half of our code and then half of that half and so on until we find the problem line. Let’s just not think about how much programming must have sucked in the days before friendly IDEs that highlight mistakes for you :)

Unrelated image from pexels.com to make this post look nicer in social media shares.

Another part of first sight for programmers is also your attitude. If you don’t want to see the problem, you’re just not going to no matter how observant you are normally. I’m by no means perfect at it myself, but I’m convinced the most useful attitude you can bring to debugging is the simple acceptance that you got at least one thing wrong. The longer you spend insisting that your code should work, the longer it takes to figure out what’s actually wrong with it.

Moving on, second thoughts are thoughts about your thoughts. When you think you know the best way to build something, why do you think that? How do you know you’re right? Is that actually the best way or is it just the first way you thought of? How would you know either way? What constitutes the “best” way to do something? Is “best” the most performant, the easiest to read, the easiest to change, the quickest to write, the easiest to test? If “best” for your project meant quickest to write yesterday, does it still mean that today? How would you know when that changes?

Checking up on yourself like that is really hard to do and that’s why this post is more for me than for you – I’m trying to remind myself to question my assumptions.

One of the traps I fall into most often is looking for an example of what I want to do in our existing code and then assuming the first thing I find is the right way to do it. Shockingly enough, codebases change over time. Just because something worked well when it was written doesn’t mean nobody has thought of a better way since then or that the rest of the app hasn’t changed enough to make the old “right way” completely different from today’s “right way.” Just like you look for a couple of sources that agree with each other when you’re Googling what an error message means, look for a couple of examples in your codebase and if they’re different, check which one is newer.

Getting into the habit of thinking about how you think is not easy (at all!), but it’s useful and, like the other installments of this series, not something that you have to devote all of your free time to. It’s also useful in pretty much every area of your life. When you have any problem to solve, how do you know you’re right about how to solve it? For that matter, how do you know you’re right about what the problem is?

When I’m stressed out, every little thing drives me absolutely crazy. I can end up convinced that what’s bothering me is that this stupid freaking feature won’t work no matter what I do when the real problem is that I’m trying to hit a tight deadline and marketing keeps changing their minds about what’s important and half the QA team is sick so they need extra time to test everything and that means I need to deliver even sooner and everything is terrible!

Okay, so what do you do about that? For starters you really should read that blog post I linked earlier, Amy Hoy goes into a lot of detail about learning to notice yourself thinking. My big tip is just to get into the habit of asking yourself “Why? Why did I decide that? Why is that the best way? Why is that bothering me so much?” Sometimes the answer is going to be stupid simple: I decided to go to cafe at the front of my building for lunch because the weather was hideous and I didn’t want to go outside. Sometimes the answer will lead to more questions, like when you ask yourself “Why did I decide to put that config file in that directory?” In my case the answer was “Because that’s where the other config file lives” which leads to another question: “How do I know both config files should go in the same directory?” From there I learned all sorts of stuff about which files were supposed to go in the original directory and why, and where the other file that was related but not the same type of config ought to live.

This is the kind of thing that takes a lifetime to master, so don’t feel bad if you don’t get it right away. Asking yourself those questions is still worth it even if you only remember to do it sometimes.

The wall

in becoming a better programmer, development, learning, unsolicited advice

Unrelated image from pexels.com to make this post look nicer in social media shares.

I was talking with a friend who’s learning to code the other day, and the subject of the wall came up. Not the one that keeps the wildlings out, the one everybody slams into when they’re learning to code. Learning to code starts out great, there are so many tutorials that break things down really clearly, but once you’ve got a handle on the basics and want to move on to building your own projects, that’s where you hit the wall. It’s a huge leap from following tutorials where all the hard decisions are made for you to building your own projects with no one standing by to tell you where to start or what database to use or what your UI should look like.

The single thing I most want you all to know is that hitting the wall is normal. It happens to everyone. It absolutely does not mean that you’re dumb or not meant to be a programmer or that you’ll never get over the wall.

As a bit of an aside, I think the number of beginners who hit the wall and assume they’re just not smart enough to be programmers says more about how bad we collectively are at teaching programming than about the intelligence of anyone who hit the wall and walked away. I’m suspicious there’s a connection between how easy it is to write total beginner tutorials and how many of them there are, and how much harder it is to teach people to break down a problem and how few tutorials there are for that.

But anyway, I have some ideas for people who have hit the wall or who can see it in the distance and are getting worried.

One of the coolest things about programming is how many open source projects there are. Find one that you like and take it apart to see how it works. Search for text from the UI in the code and see if you can change it. Throw log messages all over the place to make the code show you what it’s doing. See if you can find some constants in there you can mess with. And don’t feel left out if you want to make games, those can be open source too.

Once you’ve found a project you like and have some idea how it works, see if you can change how it works. Let’s say you found a simple todo list app. Can you add due dates to your list items? Or subtasks? Could it play a sound and/or an animation to congratulate you when you check something off? Could you add a new feature like recurring tasks or email reminders? If you don’t have an open source app handy, just take a tutorial and mess with that.

Speaking of tutorials, if you do enough of them you’re going to start seeing similarities. Most apps have some sort of UI, some sort of data model, maybe a way to save that data for the next time you open the app (that’s pretty advanced, though, don’t worry about it right away), some logic about what users are allowed to do (like due dates can’t be in the past or players can’t have more than x hitpoints no matter how many health packs they use), maybe some communication with other APIs (but again, that’s advanced, don’t worry about it right away), and honestly, that’s pretty much it.

You can try mashing up different tutorials or open source projects too. Let’s say you have a tutorial for a driving game and one for a game where you run around and collect coins or stars or whatever. What if you could drive around and collect stars? What if you had tutorials for a todo list app and a weather app and mashed them up to make a little morning dashboard for yourself?

Don’t forget, you don’t have to do it all yourself. There are great communities like CodeNewbie, /r/learnprogramming, CodeRanch, (and lots more if you do a little Googling) full of people who will help you out.

Dev tool of the day

in dev tool of the day, development, learning, tools

Exercism is a tool that lets you download and solve practice problems in over 30 different languages. I mentioned it in passing before, but let’s talk more about why it’s great.

First of all, each problem in Exercism has a set of unit tests, so you don’t have to wonder if you’re doing it right, you can just run the tests and know for sure. The tests are also great for experimenting with your code and seeing if you can make it easier to read or easier to change without breaking it.

The problems are also carefully chosen to help you learn concepts that are important to each language. Just because you can solve a problem in a certain language on a coding challenge site like HackerRank doesn’t mean you’re learning anything interesting about that language in particular. I know that’s not really what challenge sites are meant for, but I’ve seen them recommended to a lot of people who are learning and think it’s important to be clear about what challenge sites are good for (general programming concepts) and what they aren’t necessarily good for (learning individual languages).

And finally, Exercism directly incorporates both giving and receiving feedback. Obviously getting feedback is helpful – to directly quote the Exercism site: “You can write FORTRAN in any language, as the saying famously goes, but with enough feedback, you’ll quickly find yourself writing the language the way it wants to be written.” – but giving feedback is seriously underrated. To tell someone what you think of their code, you have to read it carefully and then think seriously about what makes code good or bad. That’s enormously helpful when you’re new and don’t really understand what “good code” means yet, or when you’re new to a language and just don’t know the best way to do things in that particular language.

Give Exercism a try!

Be a better programmer while still having a life: part 7

in becoming a better programmer, career, development

Unrelated image from pexels.com to make this post look nicer in social media shares.

Back in part 1 I talked about how important it is to make sure you understand the problem you’re trying to solve. Today I want to expand on that because there’s much more to problem solving. Having a great understanding of the problem you’re trying to solve is great, but it’s not always enough. Sometimes you’re wrong about what the problem actually is. No matter how well you understand the problem you think you have, it’s not going to do you much good if you’re trying to solve the wrong problem.

Telling people to make sure they’re solving the right problem is all well and good, but an actual example always makes things a lot clearer. Conveniently enough, I saw a great example of this problem on workplace.stackexchange.com the other day. To summarize the question quickly in case it disappears someday, the questioner wants to know if there are any alternatives to doing code reviews because not everyone likes doing code reviews. To quote part of the question:

Are there any alternative processes that could replace the code review for the goal of improving the code quality? Would it be possible to have something else instead of this process? While review may be required where software bugs kill humans, could some weaker method be sufficient where the situation is far from that critical?

An edit clarified that the reason the question asker is looking for an alternative to code reviews is because people in their organization use them to play power games and prevent other team members from contributing to the project. At this point you may be developing a theory about why I think “what can we do instead of code reviews?” is the wrong question :)

This particular question did happen to contain a great clue – there’s really no substitute for reviewing your code if you want to improve it. That’s kind of like saying you want to be a better writer but you don’t want anyone to proofread your work. When your solution goes directly against your stated goal, there’s almost certainly a deeper problem. Sometimes that problem is fixable and sometimes it’s not, but there’s definitely something there you need to look into.

Given that the reason the question asker wants to find an alternative to code reviews is because team members are using them to jerk their colleagues around, I don’t think it’s too much of a leap to the conclusion that the real problem is that people are being jerks and playing power games when they’re supposed to be working as a team and that trying to avoid code reviews is just a workaround for a serious culture problem.

To be clear I don’t blame the question asker for trying to solve the wrong problem. I’m assuming they aren’t a manager and/or don’t have the authority to tell the power game players to knock that shit off and start acting like grownups, so finding some way to avoid code reviews without completely ignoring code quality is about all they can do to work around the real problem. But if you’re going to do that, and sometimes finding a workaround/bandaid solution is the only thing you actually can do, I still think it’s important to be honest with yourself that what you’re doing is putting a bandaid on the real problem. If you forget that, you’re going to get a nasty surprise later when it turns out the real problem has popped up again in a different form.

To keep harping on the code review example, just because you’ve removed one avenue for for jerks to play power games doesn’t mean everyone is going to start playing nice. If you do retrospectives or post-mortems of any sort, jerks are going to use those to throw their colleagues under the bus and/or to take credit for their work. Whatever system you use to assign work, jerks will try to abuse it to keep the interesting/fun/easy/politically valuable tasks for themselves and leave the dregs for someone else. And no matter what you try to do to control bad behaviour in your development process, you can’t prevent someone malicious from going to lunch with their dev manager buddy and complaining that that one feature sales keeps pushing for has to be postponed again because so-and-so just isn’t contributing anything (of course they’ll leave out the fact that the malicious dev won’t approve any of their pull requests), they’re such a drain on the team.

This particular problem is especially difficult to actually solve because the real solution is for management to do their jobs and enforce consequences for sabotaging team mates and otherwise refusing to act like a professional. Making anyone, especially someone who outranks you, do their job is never an easy task, so I completely sympathize with the urge to “fix” the symptom rather than the root cause. Some problems are simply above your pay grade, others may be so complicated or expensive to fix that it’s better for the business to keep working around them.

In other cases, fixing the root cause of the problem actually is doable and cheaper or more efficient than keeping a clumsy workaround. Even then, you can’t fix the root problem without knowing what it is, so keep asking why until you get down to a bedrock answer like “Because that’s how this company makes money.”

 

How does a hash function work anyway?

in algorithms, development, how does it work?

A while ago I wrote about how hash maps work, but something’s been bugging me. How does the hash function do its thing? I know hash functions make variable length data into fixed length data but how do they do that? To be clear I’m interested in the kind of hash you would use for a hash map, you would definitely want a more secure hash to keep your passwords safe.

Thanks to the magic of the internets, it’s really easy to find the function java uses to calculate a String’s hashcode.

/* Returns a hash code for this string. The hash code for a String 
object is computed as s[0]*31^(n-1) + s[1]*31^(n-2) + ... + s[n-1]
using int arithmetic, where s[i] is the ith character of the string, 
n is the length of the string, and ^ indicates exponentiation. 
(The hash value of the empty string is zero.)
Returns: a hash code value for this object. */

public int hashCode() {
  int h = hash;
  if (h == 0) {
    int off = offset;
    char val[] = value;
    int len = count;

    for (int i = 0; i < len; i++) {
      h = 31*h + val[off++];
    }
    hash = h;
  }
  return h;
}

Okay great, that’s totally clear, right? ;)

Yeah, I have no idea what it’s actually doing either. But I can find out!

First of all, where are the values of hash, offset, and count coming from? They must be instance variables because they weren’t passed in as parameters. I poked around in the String code a little more and it turns out hash is defaulted to 0 when it’s declared, offset is set to 0 in the constructor, and count is set to the size of the string when it’s created.

Unrelated image from pexels.com to make this post look nicer in social media shares.

The first thing hashCode actually does is checks if hash is 0. If it’s not, then we know we already computed the hash and we can just return it and go on with our day. Makes sense, why do the same calculation over and over again when we can just do it once and store the result? I think that’s the same reason count is stored separately instead of just calling value.length() when you need it. We know the length will never change because Strings are immutable, so why not save ourselves a lookup?

The next weird thing is how the method is adding a number to a char. Chars are characters, not numbers, aren’t they? Well, yes and no. According to the docs, a char is “a single 16-bit Unicode character. It has a minimum value of '\u0000' (or 0) and a maximum value of '\uffff' (or 65,535 inclusive).” That 0 to 65,535 part seems suspiciously like a number :) You can also test that out yourself in the Java REPL. It turns out Java will happily treat a char like an int if you ask it to.

The rest of it is pretty simple, we’re just looping through every character in the string and adding (31 * current hashcode) + current character to the existing hashcode.

Okay, but how does that map a string of any length to a hash code of fixed length? Shouldn’t a longer String always have a larger hash code? Not if your hashcode is an integer! Those just roll over into negative numbers if you add too large of a number to them. And because 2 and -821785444 are both integers they take up the same amount of memory, which means that no matter what size String you start with, the hashcode is always the same size.

Another interesting little detail of how hashmaps actually use those hashcodes is that they rehash your hashes. If everyone used random Strings for keys then they wouldn’t need to, but because keys are usually Strings with some kind of meaning, that means the hashes for those keys won’t be evenly distributed. That is, a hashcode doesn’t have an equal chance of being any number from -231 to 231-1, you’re going to get clumps of hashes around some numbers because you’re more likely to use some Strings than others.

Great, but why does that matter? Performance! The more collisions you have (different Strings that happened to work out to the same hashCode), the more elements you need to look at to find the one you wanted and the worse your performance is. To get around that, java does some bitwise operations on the hashcode to reduce the number of collisions.

Now we all have some idea what actually happens when you use a HashMap :)

Link of the day

in delightfully odd, development, link of the day

This post titled How I Ruined Office Productivity With a Face-Replacing Slack Bot (Without Really Knowing What I Was Doing) has been floating around the internets lately so you may have already seen it. Even if you have, it’s an interesting example of how a programmer breaks down a new problem and experiments with the pieces before putting it all together. I especially like how that post shows the tiny experiments Jason started with and how those built on each other.

Do you have to be good at math to be a programmer?

in career, development, learning

Slightly related image from pexels.com to make this post look nicer in social media shares.

One of the most common misconceptions I’ve heard about programming is that you have to be a math whiz to be a good programmer. I’ve mentioned this before but I want to attack that particular stereotype more directly.

It is true that there are similarities between math and programming – programming uses some math terms like function and variable, it’ll be a lot easier to get things onto the screen if you understand coordinate systems, and of course to do math or programming you spend a lot of time manipulating symbols and thinking abstractly, but you know, programming is also a lot like language. I mean, they call them programming languages for a reason. To quote Hal Abelson, Jerry Sussman and Julie Sussman’s book Structure and Interpretation of Computer Programs:

First, we want to establish the idea that a computer language is not just a way of getting a computer to perform operations but rather that it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute.

Programming is fundamentally about communication. Not only are you communicating with the machine, you’re communicating with your future self, the rest of your team, and potentially other teams if you end up working for a large enough company. And that’s just when you’re writing code! You’ve also got to make sure you’re building the right thing and let other people who are waiting for that thing know how it’s going. You know what’s great practice for communication? That’s right, English classes! Sarah Mei has a great blog post about the same point I’m trying to make in this one, here’s an especially interesting quote:

The students I saw – all adults – came from a wide range of backgrounds. People with a math background did fine, of course, but people with a heavy language background often did better.

Nobody seems to have an official study on this, but I think there’s something there. Language requires just as much abstract thought as math does, with the added benefit (to your communication skills) of failing your classes if you can’t make yourself understood to anyone else. As a bit of an aside, the biggest problems I’ve run into at work have never been problems of pure programming skill (although those ones do suck a lot), they’ve been communication problems with team members, other teams, management, other departments entirely and other offices.

And to stick another nail in the coffin of the idea that programming is math, I’ve been a developer for ten years now and the most complicated math I’ve ever had to do at work is stuff like “if I have a div that’s x pixels wide and y pixels tall and I want it centered inside another div that’s a pixels wide and b pixels tall, what should my margins be?” I used to believe that programming was math, and some programming is certainly math heavy, but it’s hard to argue with the lack of math in my day to day job over ten years and four companies.

So given that programming is not math, why do so many people think it is? I’m going to quote Sarah again because she already said it really well:

When programming was just getting started, early in the last century, we used it to solve highly mathematical problems like calculating missile trajectories and decrypting secret messages. At that point, you had to be good at math to even approach programming. Tools, such as programming languages, were designed specifically to solve mathematical problems, because those were the ones we thought it was worth spending money on. Computers were for doing math.

I think another issue is the way we teach programming (credit where it’s due, I stole that idea directly from a comment on Sarah’s post). Computers are great at doing boring and tedious calculations, plus it’s easy to tell them how to do it, so why not start teaching programming by having students make the computer calculate something for them? It makes it really easy for the students to check their work too – assuming they’re able to do the math problem themselves, they can easily check if the computer got the same answer.

That assumption can be a real problem, though. If you can’t do the math problem or don’t care about the math problem, you end up effectively shut out of programming, something you could very well be great at, because of something that has next to nothing to do with the actual job of software development.

If you can do basic arithmetic and deal with the concept of variables, congrats, you can be a programmer! That is literally all the math you actually need!

Be a better programmer while still having a life: part 5

in becoming a better programmer, best practices, career, development

Unrelated image from pexels.com to make this post look nicer in social media shares.

Today’s tip for becoming a better programmer while still having a life isn’t just good for you, it’s good for your whole team. That tip is documentation.

I know, nobody actually likes doing documentation. Fortunately, I’m not talking about dry design docs or endless specifications, I’m talking about a simple wiki (or whatever works for you), written by you and your team for you and your team. It doesn’t have to be polished, it doesn’t have to be formal, it doesn’t even have to be spelled perfectly. All it needs to be is correct and understandable. The idea is to explain things just like you would to another dev on your team, not to waste hours proofreading.

Great, but what should you document? In short, anything you’re going to want to know later. Especially useful things to write down are why architectural decisions were made, how parts of the system work at a high level, guides to testing certain features (great for complicated payment provider integrations, not that I’ve been fighting with those), and especially guides to setting up your development environment, whatever that thing is that you forget every time and have to look up again or ask someone about.

Just saving yourself time relearning parts of your codebase will help you get things done faster, which is basically being a better programmer, but explaining things (even if you’re writing to yourself) helps you understand them better. Explaining something forces you to think about every little detail, and putting something in your own words helps it stick in your brain – it’s the same principle that helps you remember things better when you take notes on them, whether or not you ever look at those notes. The better you understand your codebase, the better decisions you can make about how to add new features or fix bugs.

As a bonus, reflecting on why you made certain decisions and being able to go back later and see if the reason you did things the way you did holds up over time also makes you a better developer. You don’t even need to make a special effort to do this, if code gets used it gets changed. All you need to do is wait, eventually you will have to revisit the code you documented and by the time you do that, you will probably have forgotten the details and will need to go look it up in that handy wiki you created :)

Writing things down also helps you get that answer again without interrupting anyone, and the less you can interrupt teammates the better considering how expensive interruptions are for programmers. I couldn’t find a hard number, but I feel comfortable saying that it takes at least ten minutes to get back into a complex task like programming once you’ve been interrupted. It can be even worse if you get interrupted at just the wrong time. I have a terrible time getting into a task when I know I’ll have to stop soon for a meeting or appointment or whatever, which means an unfortunately timed interruption can kill a solid half hour of productivity even if it only took a couple of minutes to deal with the interruption itself.

All the benefits you get from making knowledge about your code available and searchable also apply to your teammates. If you just document how to set up your development environment, that can save hours every time someone has to set up their environment whether it’s a new employee starting or a long time employee getting a shiny new computer to replace an old one. It’s even better for everyone around you if the whole writing things down idea catches on – then they also get the benefits of explaining things and documenting the particular things they forget and need to look up as well.

And best of all, this is another work thing you can do at work that has no effect on your personal time.