melreams.com

Nerrrrd

Stubbornness > intelligence

Unrelated image from pexels.com to make this post look nicer in social media shares.
Unrelated image from pexels.com to make this post look nicer in social media shares.

As part of my ongoing effort to demystify development, let’s talk about what personality traits actually matter even a little bit for developers.

There are tons of stereotypes out there about how you have to be super smart or great at math or have started programming when you were six to be a really good programmer. That’s all total bullshit.

I believe there is a single personality trait that’s extremely important to becoming a programmer, but it has nothing to do with how smart you are.

Intelligence, in as much as you can even measure something so poorly defined, simply does not matter. In my experience being clever has nothing to do with whether your code is readable or maintainable. I would much rather work with someone who knows they aren’t a genius and codes to avoid common mistakes than someone who is in love with how smart they think they are.

Being great at math can be helpful, but it’s by no means necessary. Math is really good practice at thinking through a problem one step at a time, particularly stuff like proofs, but if you’re not working on games or graphics programming you may never use any math more complicated than basic arithmetic in your day to day job.

Having started programming when you were very young can also be helpful because that gives you more years of experience, but it’s not as if there aren’t plenty of great programmers who never touched a computer until high school or even college.

So if none of those things really matter, what does? Stubbornness. I firmly believe the one personality trait you really need to be a programmer is stubbornness.

Programming, particularly when you’re just starting out, can be immensely frustrating. Teeny tiny little details like where exactly you put that semi-colon will ruin your day. The sheer amount of stuff you need to learn will seem totally overwhelming. You will feel stupid and think about quitting over and over and over.

Learning to code these days with friendly modern languages like python is certainly easier than learning assembler back in the day, but you need to be pretty stubborn to learn enough programming to be able to build something interesting.

But don’t think it’s all smooth sailing once you learn to code. Just the other day I spent a couple of hours debugging what I thought was an issue in my server code, but was actually me passing in the wrong parameter names through a javascript api like a dope. The more time you spend programming, the more horrible bugs you will run into, the more tough design problems you will have to solve, the more times you’ll have to figure out how to refactor existing code to get something new working, the more times you will kick yourself when the thing that seemed like a great idea six months ago turns out to be a terrible idea after all, and so on and so on.

As much as I love programming, I still think that you have to be a little bit weird to put up with this level of frustration. If programmers had the sense to do something that was less of a pain in the ass, they wouldn’t be programmers :)

Podcast of the day

As a person who takes the bus to work and has trouble reading in moving vehicles (motion sickness sucks), I’m a big fan of podcasts. My favourite tech podcast is Hanselminutes. It’s run by Scott Hanselman, a programmer, teacher, and speaker, to directly rip off his about page.

Hanselminutes is my favourite tech podcast because of the variety of topics and guests and Scott’s skill at interviewing. He’s had guests come talk about everything from data visualization to funding open source to getting started making video games, and no matter what they’re talking about the conversation always flows smoothly. It could be that Scott is amazing at picking guests and editing out the awkward bits, but I think he deserves some credit for being able to guide a conversation like that too :)

If you like podcasts at all and don’t already listen to it, give Hanselminutes a try. You can find it on iTunes or just by searching for Hanselminutes (yay for unique podcast names!) in any Android podcast app.

How does big O notation work, anyway?

Unrelated image from pexels.com to make this post look nicer in social media shares.
Unrelated image from pexels.com to make this post look nicer in social media shares.

You didn’t think I was going to stop beating that programming interview concepts post into the ground, did you? Unlike some of the other concepts, big O notation is something you actually need to understand to write good code. Even better, you don’t need to know the term big O notation to have a conversation about the underlying concept.

Big O notation, also known as the order of a function, is a way of describing how much worse a function performs when you give it more input. If you’re sorting a small enough list, for example, it really doesn’t matter how efficient your algorithm is. If your list is a million items long, then the efficiency of your algorithm suddenly matters a whole lot.

The specifics aren’t necessarily that interesting, but let’s run through them quickly. Some functions, like figuring out whether a number is even or odd, are constant, or O(1). No matter how big a number you give that function, it’s going to take about the same amount of time to figure out whether it’s even. Functions that take longer in direct proportion to the size of the input, like finding an item in an unsorted list are called O(n) – the order is the same as the size of the input. Inefficient sort algorithms like insertion sort or bubble sort get much worse, O(n²). A bad sort has to compare each item in the list to every item in the list, which means that performance gets worse much faster than the input size increases. An extremely inefficient algorithm could even be O(n!), for example brute-forcing the travelling salesman problem.

In other words, nested loops are really, really bad :)

Understanding just how much worse an inefficient algorithm performs is a really important part of being able to make reasonable tradeoffs in your code. If your input is large enough, creating a binary tree out of it may be worth the extra memory usage for the improvement in time to find any given item. Assuming the computer will just magically handle it is not good enough when you may have to process hundreds of thousands of items or more.

As an interview question, I actually like big O notation – as long as you don’t insist on using those exact words to describe it. There’s no reason you need to use comp sci jargon to talk about which function of two examples would perform better with a massive amount of data to work on. It’s also something you can have an actual conversation about. Sometimes an inefficient algorithm is your only option, sometimes it can be avoided by using a different data structure, sometimes you can avoid the issue entirely by finding a different way to meet the user’s needs. That much more interesting to talk about than whether your candidate thought to look up how a breadth first search works :)

Authenticator tip of the day

Fun fact about authenticators like Google Authenticator and Authenticator Plus: they have their own internal clocks that can get out of sync with the server you’re logging into. Most of the time they don’t drift far enough to cause problems, but AWS is very, very particular about your clock being in sync with theirs.

If you find that AWS rejects your one time codes but the rest of your accounts work just fine, it’s probably a clock sync issue. In Google Authenticator the sync function is easy to find under ‘Time correction for codes’ in the menu. In Authenticator Plus it’s hidden behind ‘Others’ -> Sync with Google. Try that before you waste a morning trying to figure out how one and only one of your accounts got corrupted ;)

What should you ask in an interview: culture edition

Unrelated image from pexels.com to make this post look nicer in social media shares.
Unrelated image from pexels.com to make this post look nicer in social media shares.

Last time I talked about what technical questions you should ask in an interview, which is important but culture fit is at least as important. We’ve all heard of The No Asshole Rule, right? I firmly believe there is no such thing as an asshole so brilliant they’re worth working with, so for me culture fit trumps technical skill as long as all the candidates are halfway competent.

So what should you ask in interview to gauge culture fit?

First of all, the way many people talk about culture fit is completely and utterly wrong. Culture fit is not even slightly about figuring out if someone is cool enough to work with you. It is NOT about looking for a brogrammer who will play foosball and drink heavily with your team. It’s also not about finding someone who enjoys the same nerdy TV shows, books, games (video, card, or board), comics, etc as you do.

To quote Donna Choi’s Medium post about how they make hiring decisions at stackoverflow:

Be careful with “Cultural Fit”. This is often a catch-all for a vague sense of “would not fit in”, which can come to mean “is like me”. If you feel someone is a good or bad cultural fit, you must explain what you mean.
Valid “Cultural Fit” things: self-motivated, passionate, gets stuff done, cares about open source / giving back to the community, likes “default open”, hates office politics / meetings, pragmatic attitude towards tools / best practices, etc.
Invalid “Cultural Fit” things: obvious stuff like race, gender, sexual orientation, religion but also softer things like age, personality or hobbies (does not have to like Magic the Gathering to be a good dev). Assume that your bias is to hire people you “like” and be very careful of that.”

Another way to put it is how Johanna Rothman describes culture:

It’s about what you can and cannot say in the organization, how people treat each other, and what we reward.

Culture fit is about values, not about hobbies. A workplace run by grownups does not care even slightly about how “cool” you are or whether you’d be fun to hang out with. They care that you all have compatible views of the world. For example, some companies are very cautious and have very rigid procedures for deploying to production. Others do their best but ultimately feel that rolling back a bad release is not the end of the world. Commitment to work/life balance is another example of culture fit: is the company standard that nobody works overtime unless production is on fire, or are occasional spikes in hours to be expected? Priorities are also part of cultural fit: does the feature need to be perfect or just good enough? Are you planning to grow the company quickly and then sell it off, or are you planning to work here for the next ten years?

Rand Fishkin from Moz has a great definition of what culture fit actually is:

  1. Shared beliefs – the things that you collectively hold to be true about the world (e.g. good people tend to have traits like X, the right way to treat others is Y, what’s appropriate/inappropriate at work is Z).
  2. Shared priorities – what matters in terms of big, overarching things like work/life balance, short vs. long term commitment, how decision are made, etc.
  3. Stylistic cohesion – some people don’t work well together, others find themselves able and inspired to do more when surrounded by a certain type. Cohesion isn’t about finding lots of people who are the same, but about making sure there’s no one on the team that detracts from others and that many get more enjoyment and progress from the diverse perspectives their co-workers bring.

Now that we know what culture fit actually is, how do we figure out whether a candidate will fit?

First of all you need to know what your company’s culture actually is. Without some sort of solid definition, you’re stuck stumbling around asking largely random questions and hiring someone you “have a good feeling about,” which usually comes from them being like you. This is how you end up with a total lack of diversity and bad hires because you don’t actually know what you’re looking for.

Once you know what your culture actually is, it becomes a lot simpler to ask questions about it. If part of your culture is that everybody gets their say when a decision is made, ask your candidate how they think decisions should be made. If part of your culture is that when things go wrong you fix it first, try to keep it from happening again second, and look for someone to blame never, then ask your candidate how they think problems should be handled. If it’s important that your people get enough downtime, ask a team lead/manager candidate how they make sure their team gets enough downtime and how they handle it if upper management suddenly drops a new requirement on them. If you often have to meet rigid deadlines, ask how much flexibility they need.

Like I said in my post about technical interviewing, you are allowed to straight up ask about culture! Just make sure you’re actually asking about culture and not about whether you’d enjoy hanging out with the candidate.

For more example questions, I recommend looking at lists of questions for candidates to ask their interviewers. There’s some good stuff in this list and there are plenty more of them out there. Not all of those questions still make sense if you turn them around to ask a candidate, but you can steal a good chunk of them :)

Sadly, that’s the simple part of interviewing for culture fit. Another really important part is asshole detection, which can be a lot harder. Good culture fit questions will definitely detect at least some assholes, but people can be jerks in very subtle ways. I mean, nobody is going to tell their interviewer that they don’t listen to new ideas, but there is no shortage of people who will do things their way and only their way. People can also quietly wreck your company’s reputation by being enough of a jerk outside of work, in case you weren’t already paranoid about making a bad hire.

Normally I don’t recommend trying to trick your candidate because I think it’s a dick move, but I’ve heard it can be really useful to take a candidate out to lunch or for an after work drink and see how they behave outside of work. You would think people would stay on their best behaviour when they haven’t even started the job yet, but something about being outside the office seems to help people forget they’re still being interviewed. To be fair, I wouldn’t want to work somewhere I couldn’t be myself, so as tricks go I don’t think that one is that bad.

Readers, do you have any tips on catching jerks who are smart enough not to be obvious about it?

Perl tip of the day

If you have to use Perl, a) don’t feel bad about having trouble understanding it sometimes, everybody else does too, and b) there is an operator that’s extraordinarily difficult to google. If you happen to run into it, in Perl =~ returns true if the value on the left matches the regex on the right and false if it doesn’t.

That type of operator is called a binding operator and if you’re lucky and you already know that it becomes possible to google it. If you don’t, it’s kind of a terrible experience. Seriously, try googling “perl =~ operator” It takes some serious digging to find any useful results.

What should you ask in an interview?

In a few of my how does it work? posts I’ve mentioned that asking about a particular data structure or algorithm is a boring interview question that doesn’t tell you much of anything about whether your candidate can code.

First of all, the reason I think asking someone to implement a data structure or an algorithm is boring is because all you learn by doing that is whether or not your candidate looked it up beforehand. Not even that, necessarily. Interviews are pretty stressful and plenty of people’s minds go blank when they’re stressed. I once choked on a question about the servlet lifecycle and it’s not like I don’t know how servlets work. If a question might tell you one thing and might tell you nothing, I don’t think it’s a good use of the limited amount of time you have in an interview.

Unrelated image from pexels.com to make this post look nicer in social media shares.
Unrelated image from pexels.com to make this post look nicer in social media shares.

I hate ridiculous brainteasers for the same reason – they’re just so binary.  Either your candidate gets the right answer or they don’t, which may or may not tell you anything because a stressful situation where you’re really scared of looking stupid is basically the worst possible environment to solve puzzles in.

Given all that, it seems fair to ask “Well what on earth should I ask in an interview?”

My take on it is that we’re overcomplicating the question. Which is kind of an occupational hazard, so don’t feel too bad about it :) Basically, if you want to know if someone can program, ask them about programming! Just straight up ask them how they would solve a problem.

One of my many pet peeves are wildly improbable scenarios questions like how many piano tuners are there in New York? Google, that’s how many. If you want to know how someone approaches a problem, just give them a programming problem. I really think it’s just weird to dance around the subject you really want to know about. You are allowed to straight up ask people about programming! It’s not as if programming problems are thin on the ground either (hey, what do you do all day again?), although it does take some work to find a problem that doesn’t require twenty minutes of backstory to explain why the problem is a problem. And if you’re asking improbable scenario questions because you want to see how a person thinks about a problem without them jumping directly into code, I think you’re depriving yourself of useful information. If somebody jumps directly into coding without thinking the problem through, I want to know that!

The kind of questions I think are interesting are realistic-ish (obviously you need something relatively simple if you want to ask more than two questions before you run out of time) scenarios like “given a system like x, how would you add feature y?” and “how does your solution change when you discover that the system is actually more like q than like x?” or “what would you do if production went down or was painfully slow?” or “given this description of a weird bug, what would you do to track it down and solve it?”

While every interviewing method has flaws, I like the idea of giving someone a take home project that either takes under two hours or that you pay them for, then talking about the choices they made in the interview. If you want to know if someone can code, just ask them to write some code! But for the love of god don’t ask them to do it on a whiteboard, that’s ridiculous. Coding on a whiteboard in no way resembles what a programmer does all day, even if your candidate is good at it it’s simply irrelevant to the job you’re trying to fill.

The downsides to take home projects are that they’re very biased in favour of people who have the free time outside of work to do them and that like everything else, it’s very very easy to do them badly. I’m willing to spend an hour or two on a take home project, but any more than that and I expect to get paid for my time.  That’s coming from someone who has no children, no pets, no elderly or sick relatives to look after, and practically no chores to do, too. For people who do have responsibilities, asking them to spend n hours of their scant free time on you is a big deal and you need to keep that in mind when you design your take home project.

Which is the other potential problem with take home projects: they need to actually take the amount of time you say they will. Yes, that means you need to have someone at the level you’re trying to hire for solve it first. I would say solve it yourself, but if you’ve been programming for ten years and you’re trying to fill a junior position, you need an actual junior to make sure what you’re asking for is reasonable.

That said, even the worst take home project is much more realistic than the ridiculous idea of asking people to take a short contract with you to see how they actually work with everyone in the office. There is basically zero chance anyone with any responsibilities is going to think “Sure, I’ll definitely quit my stable full-time permanent job to take a two week contract with some yahoo who might hire me if they decide they like me. What could go wrong?” That said, if you’re looking for people you can exploit, by all means keep hiring that way.

On a more cheerful note, a really good idea that came from a discussion on slack is having the candidate read some code and explain what it does. Most professional programming involves way more reading code and making sure you understand it, so much so that it seems like a really good idea to test people on that directly.

In general, ask stuff that directly relates to the job you’re asking someone to do. If you really do want them doing algorithm design then fine, ask them about algorithms. If you’re doing web apps, ask them about the frameworks you use or about general web concepts like concurrency, multi-threading, sessions, basic security, etc. If you do consulting, lean a little harder on scenarios where the candidate needs to ask questions to figure out what the client really needs, which is usually not what they asked for.

What do you think, readers? Do you have any favourite interview questions?

Mongo tip of the day

If you ever use Morphia and Jongo in the same codebase, be warned they can interact in very confusing ways. Quick note for people who may not have any idea what I’m talking about: Morphia maps Java objects to Mongo documents and provides a really nice fluent interface for querying. Jongo is not quite as friendly but gives you all the same functionality as the Mongo shell. Most of the time we use Morphia at work, but once in a while for weird queries that Morphia doesn’t support (like regexes) we use Jongo.

Morphia has annotations that let you have short, space-saving field names in your documents and nice descriptive field names in your code. Jongo can convert its results into Java objects for you, but it uses Jackson to do that. Neither Jongo not Jackson know about Morphia’s annotations, which means if you used annotations to change the field names in your document, Jongo will try its best to build objects out of your results but they will be empty because it can’t match your document field names to your object field names. My coworker Eric figured that one out after I spent all afternoon swearing at it.

Fortunately, there’s a workaround! If you don’t want to dirty up your code with Morphia and Jackson annotations, you can get Jongo to give you back JsonNodes by using the JsonResultHandler like so:

Iterable<JsonNode> iterable = jongo.getCollection("collection")
    .find("{ query }").map(new JsonResultHandler());

Pulling fields out of a JsonNode isn’t as nice as having an object already filled in, but on the upside it’s a lot less hassle (and ugliness) than doubling up on annotations. If Jongo misbehaves for you in weird ways, try getting simple JsonNode results back instead of messing around with object mapping.

How does quicksort work, anyway?

Unrelated image from pexels.com to make this post look nicer in social media shares.
Unrelated image from pexels.com to make this post look nicer in social media shares.

Why yes, I am going to keep mining that article about stuff you should know for programming interviews for blog post ideas :) While I don’t think that a lot of the common interview concepts from that article are actually worthwhile to ask about in an interview, I do think they’re interesting bits of nerd trivia and going in depth into how stuff works shows that nothing the computer does is magic.

Sort algorithms in particular are a weird interview question because you should basically never implement one at work. There are always edge cases, but in general if you actually write a sort function you have done something bad and you should feel bad. The correct way to implement a sort function is to import a library and go on with your day.

That said, sort algorithms are interesting in their own right. They’re one of those things that seem incredibly simple and boring until you start thinking about how you would tell a computer how to sort things. There are also way more sort algorithms than you might think, all with their own pros and cons.

Quick sort uses a divide and conquer strategy – instead of sorting the entire array you give it, it picks a pivot point (different implementations do this in different ways, one of the simplest methods is just to take the middle element of your array), rearrange the elements of your array so that everything less than the pivot is on the left and everything greater is on the right. Then you break the array into halves and recursively search each one until everything in the array is in order. There’s a really helpful gif at the top of the wikipedia article about quicksort that explains it better.

Because quick sort rearranges the array elements by swapping them, it requires very little memory, which was a big deal when it was invented by Tony Hoare in 1959. To this day it’s one of the fastest sorting algorithms, provided you do a good job of picking your pivot point. If you do a bad job of that things go off the rails, particularly if your array is mostly sorted already. In that case quick sort can (if you don’t check for a sorted or mostly sorted array) effectively unsort and resort your array which is pretty slow, surprisingly enough.

Another efficient (in this case it’s a technical term for sort algorithms that are efficient enough to actually use) sort algorithm is merge sort. Merge sort is even older than quick sort, it was invented in 1945 by John von Neumann. Like quick sort, it uses a divide and conquer strategy, the difference is that merge sort divides the array into the smallest pieces it can, then merges those pieces into two element arrays, then merges those into four element arrays and so on until it produces a completely sorted array. As usual, wikipedia has a gif that explains it visually.

Merge sort requires much more memory than quick sort does because of the way it creates new arrays while it’s sorting. This can be an issue if you’re sorting especially large arrays, although I’m sure more advanced algorithms based on merge sort can do some sort of trickery to mitigate that :) On the upside, it’s a stable sort – if you have two objects in the array with the same sort order, they’ll stay in that order – unlike quick sort. It’s also good at handling slow sequential media like tape drives and handling linked lists, which quick sort is slow at and heap sort can’t handle at all.

Heap sort, the last sort algorithm I want to talk about today, is an interesting one. Unlike quick sort and merge sort, heap sort puts all the elements of the array into a heap first, then uses that to sort the array.

Quick digression from sorting: a heap is a partially ordered tree structure. In a heap, the child nodes are always less than (in a min heap, or always larger in a max heap) the parent node, but siblings aren’t in any particular order relative to each other. The root node is always the largest or smallest element in the heap, and if you remove it the heap rebalances itself so the next largest or smallest element becomes the new root.

Back to heap sort: once you have a heap it’s very simple, you just take the root, add it to your array, let the heap rebalance itself, take the new root, and so on until your heap is empty.

In comparison with other sorts, heap sort is a little slower than quick sort on average but has better worst case performance. Merge sort has similar time bounds (average, best case, and worst case time it takes to sort an array), but takes up more memory because a heap sort can be done in place. On the other hand, merge sort is stabl, parallelizes well, and works on datasets too large to fit into memory at once, which neither quick sort not heap sort can do.

One last piece of trivia: the Timsort algorithm, implemented in 2002 by Tim Peters, is based on merge sort and insertion sort (a very simple sort algorithm) and is the standard sort function in Python and Java.

There’s a huge amount of detail I skipped over, I recommend poking around wikipedia if you’re interested in more detail about the many, many, many ways you can sort a list. Just don’t ask about them in interviews, because all you’ll learn by doing that is whether your interviewee looked them up beforehand :)

Atom plugin of the day

Lately I’ve been updating some docs that we generate from YAML files and while I do like YAML as a format (beats the hell out of XML, but that’s the kind of bar you trip over), it can be really painful to figure out where exactly you messed up your indenting.

Fortunately, there’s an atom plugin for that! linter-js-yaml gives you incredibly fine-grained and helpful error messages. Isn’t that screenshot pretty? It’s got a nice obvious red dot and an underline and an error message that actually tells me what I got wrong! If you ever edit YAML, try linter-js-yaml, it’s awesome.

YAML linter screenshot showing a helpful error message
YAML linter screenshot showing a helpful error message