melreams.com

Nerrrrd

“Show me what you build, and I will tell you who you are.”

in development, opinion

I think this talk by Eric Meyer is really interesting and worth watching. Be warned, he does discuss his daughter Rebecca’s death of brain cancer (not in detail), so maybe you don’t want to watch this one at work.

Aside from the crushingly sad part, he has some really interesting stuff to say about how the way we build the tools we do (social media like twitter, facebook, linkedin, for example) reflects our values and how that will shape the next generation’s view of what’s normal on the internet. Like he says in the talk, “Show me what you build, and I will tell you who you are.”

When is it done?

in career, development, opinion, project management

“Done” is a surprisingly ambiguous word in software development. Back when I was in college I thought an assignment was “done” if it compiled and produced more or less the result I was expecting. Then I got a job in the industry :)

It turns out “it compiles” doesn’t mean much when you need to ship software that handles edge cases correctly and works in more than one browser and doesn’t throw exceptions when it’s used just slightly differently from the way I assumed it would be when I built it.

Another complication is that people use the word “done” to mean a lot of different things. I’ve heard people say “it’s done, I just need to test it” and “it’s done, I just need to integrate it” and “it’s done, it just needs to go through QA” and “it’s done, I just need to fix a couple of bugs.” It’s certainly useful to know what stage of development a feature is in, but none of those are done.

To me the only meaningful definition of “done” is “ready for production.” That is, is when the developer has tested the feature, QA has passed it, it’s been merged onto the production branch and any regression testing that needed to happen is done. When it’s ready to be deployed it’s done and not before.

That may seem overly picky but imagine how far off the rails things go when one person thinks done means “ready for QA” and someone else on the team thinks done means “ready for production.” If your feature (or project or report) isn’t actually done, your team lead needs to know that whatever other task they give you might get bumped if your original task comes back from QA with bugs. Obviously you should aim for bug free features but in reality things don’t always go according to plan. That’s why we have a QA department in the first place.

And to go on a bit of a rant, that example above of “it’s done, I just need to integrate it” is not even slightly done and is almost always a terrible idea. If you’ve developed something outside of the system it’s going to be a part of, you’ve exposed yourself to an enormous risk of terrible surprises. Sure, you’ll be fine if you just happen to have a perfect understanding of every little quirk of the larger system, but let’s be honest; you do not. Unless it takes a truly intolerable amount of time to compile the main system, you will actually save time in the long run by building your feature directly where it’s going to be used and just getting up for a drink of water or something while your project compiles. You’re almost guaranteed to have to compile repeatedly while you get your feature integrated anyway, so you’re really not saving any time by developing in isolation. Now, the plural of anecdote is not data and other people have probably had different experiences, but I have personally never seen that turn out well and strongly advise against it. Okay, rant over.

We like to pretend software development is perfectly logical and unambiguous, but you’ll actually run into issues like different definitions of done all the time. Even if you feel dumb doing it, it’s often really helpful to ask teammates exactly what they mean. A surprising number of the problems you’ll run into developing software are just miscommunications that could have been cleared up with a five minute conversation at the beginning of the project. Interfaces between client and server code in particular can be tricky. I’ve personally had to throw out work because I misunderstood what the client team actually needed and built something that was pretty close but not quite right.

Just because you think you know what your teammates mean when they say something as simple as “done” doesn’t mean you actually do. I’d love it if we could all agree what done actually means, but until then you can save yourself a lot of trouble by asking questions.

Why your QA department is smarter than you

in development, opinion

Something that’s baffled me for a long time is the animosity some programmers have for the QA department. It’s incredibly frustrating to try to get a feature out and have QA find bugs over and over, but that’s not the QA department being jerks, that’s them doing their jobs properly. QA people actually do a lot of awesome things for us developers. They protect us from releasing a broken product and looking like dumb jerks who don’t care about doing good work. They give us bug reports that actually make sense and can be reproduced. They protect us from the wrath of the thousands of angry people who just wanted to be able to use the thing we released.

QA just wants the project to work when it comes out, which is the exact same thing I want. I don’t want end users to have to wonder whether I’m terrible at my job or whether I just don’t care. Personally I’m a terrible tester, but if a product comes out broken there’s no way for end users to tell a dev who tries really hard but isn’t good at testing from one who counted the minutes until they could go home. I don’t want to look like a fuckup. QA protects me from that, which is basically the nicest thing they could possibly do for me.

Another great thing the QA department does is produce usable bug reports. If you don’t appreciate good bug reports, you haven’t worked in the industry for very long. I have had to try to work with terrible bug reports that had none of the details I needed, were not reproducible, were duplicates, and even reports of totally predictable problems caused by continuing to work with a broken account. A good bug report is a thing of beauty and you should appreciate it! Having a clear set of steps that reliably reproduce a bug saves you so much work. So much! This is extra true of game development where the lack of a single linear path through the product produces some truly weird bugs.

Aside from protecting me from looking like I’m bad at my job, QA also protects the whole team from the many angry people who will flood your customer service and social media with rants about how your product is junk and you should be ashamed of yourselves and they want a refund yesterday. Nobody wants to disappoint their customers, and no one wants to deal with that kind of PR nightmare. You might think that a few angry people are no big deal, but they can really hurt your business and they can hurt your whole industry’s reputation. Toward the end of 2014 so many big-budget AAA games shipped broken that it shook customer’s confidence in the entire industry. Obviously no-one’s releasing numbers about how doing a bad job hurt their sales, but it only makes sense that there is now a huge pool of people who simply will not buy a game until they hear from other players that it actually works.

Given all the great stuff QA does, why don’t they get more respect as a group? Well, in general the tech industry has a problem with fixating on developers to the exclusion of basically every other job title. It’s also not unusual for project managers and team leads to see QA as an obstacle to release the features that upper management is pressuring them for, which leads to nobody else on the team giving QA the respect they deserve.

QA is not just important, they want the exact same thing you do: to put out a good product. But if the rest of this post didn’t convince you, read How to lose $172,222 a second for 45 minutes and then tell me QA isn’t important :)

So what is the play framework anyway?

in development, play framework

The Play framework is a super scale lightweight java (and Scala) web app framework. It’s built on top of Akka, a toolkit and runtime, which, to quote their website: helps you build highly concurrent, distributed, and resilient message-driven applications on the JVM.

So why is Play special and what makes it so different from something like Spring?

Play gets all kinds of nifty async features from Akka. Normally java is all synchronous – if you need to get something from the database or from another system, your app blocks until it gets a response. Play doesn’t block, it just picks the request back up when the response comes in (or times out), allowing it to serve other requests instead of just standing around. This means you can serve more requests without paying for more servers.

 

Another cool thing about Play is that it’s RESTful by default. Play’s endpoint configuration lends itself really easily to clean REST interfaces.

Here’s an quick little example of how Play endpoint config works:

# Display a client
GET    /clients/{id}             Clients.show

Isn’t that pretty? It’s so clean and shiny :)

GET is the HTTP method accepted for this endpoint, surprisingly enough. /clients/{id} is the URL (relative to your project). {id} is a placeholder for the client id that gets passed in as part of the URL, and Clients.show is the controller and method that handle this request. Compared to old-school Spring xml config this is freaking amazing. I even like it better than Spring 3 annotations because all my routes are clearly defined in one file. If I need to look up the exact parameters a route needs so I can test it or document it or whatever, there’s one and only one place I need to look.

Play also has an awesome php-like “just refresh” workflow. Yes, if you can get Eclipse to talk to your local app server you can make a very limited amount of changes without doing a full rebuild and redeploy, but Play does it so much better. Once in a while I have needed to do a full clean and rebuild but only if the code has changed a lot, like when I switch from a feature branch with a major refactor back to the staging branch. If you haven’t done that, then Play will load your changes as soon as you make another request.

Another thing I really like about Play is its non-maven build tools! Play uses sbt (scala build tool), which unlike maven actually works. When you add dependencies, sbt pulls them in! every time! In the entire 8ish months I’ve been working with Play, I have never (not even once!) needed to delete my entire local repo and re-download everything like I used to have to do with maven all the time. I’d honestly rather manage dependencies manually by checking them all into my repo than use maven ever again, so you may want to take the maven hate with a grain of salt, but sbt really does work well.

Play’s config file syntax is in HOCON (Human-Optimized Config Object Notation, looks a lot like JSON), which is so much easier to read and edit than xml. It takes some work to write an invalid config, and even if you manage it it’s much easier to figure out what you did wrong and how to fix it than it is with verbose xml tag names.

As for the actual code, Play hasn’t been that hard to adjust to. I’ve been using Spring and Struts for years and the basic controller/action setup isn’t that different. The async stuff like Promises is pretty weird to get used to, but if you’ve done much of any javascript then it won’t be totally new. It’s not so very different from making an AJAX call – you know what you’re supposed to get back, just not exactly when you’re going to get it.

One thing to keep in mind though is scope, particularly if you’re also using a dependency injection framework like Guice. Every time you hand off computation to another thread you lose the previous scope. Things can get ugly when you thought you had a user account id in scope but it turns out you don’t – that might sound like a stupid mistake but it’s pretty easy to do if you’re trying to put together a complicated result with data from different subsystems. Thanks to the magic of dependency injection everything will compile happily and you won’t know you lost your id until you start getting dependency injection errors all over your logs.

All in all I’m really enjoying working with the Play framework and recommend trying it out.

Debugging

in debugging, development, Uncategorized

There’s no way I can possibly say everything about debugging in just one blog post, but I can certainly share a few useful tips.

First, let’s talk about what debugging fundamentally is. It’s the art of seeing what actually is, not what you meant or what you thought. It’s going to be uncomfortable, and if you get too tied up in your own ego you won’t be able to do it at all. Years ago one of my teachers at Camosun told us (I’m paraphrasing heavily here because I don’t remember the exact words) that there’s no point insisting you didn’t change anything. If it used to work and now it doesn’t, you obviously changed something. Just accept that you broke it and start trying to fix the problem.

One of the first things you need to do when you’re debugging is to make sure you can reproduce the problem reliably. If you can’t do that, then you don’t really know what the problem is (or you’ve got some sort of unholy race condition bug and you’re beyond my help :) ). If you’re not the one who found the bug, ask the person who did if they can show you, or ask for more information if it came through a helpdesk and you don’t have direct access to the user who found the bug.

Once you can reproduce the problem, you’ll be able to track down what’s going wrong and figure out whether a fix actually… fixes the problem. The first step I recommend after reproducing the issue is double checking all of your inputs, even the most stupid simple stuff you’re sure you couldn’t possibly have gotten wrong. Last week I thought I had broken staging when I actually just hadn’t chosen the right value in a dropdown box. Garbage in, garbage out, as they say.

After that, you’re going to be very tempted to just read through your code and hope you can spot the problem. Resist this temptation! I’m always pretty sure I know where the problem is or that it’ll jump out at me right away, but it almost never does unless it’s an extremely simple bug. Read over the code once if you really want to, but then move on to narrowing down exactly where the bug is. Believe me, it’s faster than staring blankly at your code and feeling dumb.

If you have a particularly chatty log, you may be able to start narrowing things down while you reproduce the issue. Start looking after the last log message you see before the bug happens. If you’re very lucky something will be obviously wrong close to the log line you were looking for.

If you’re not quite as lucky, you’re going to need to run the code locally and start commenting things out. Assuming you have some idea where the problem is happening, start dividing the method it could be in, in half. Either comment half of it out or add a log line half way through and see if you see that log line before or after you reproduce the problem. Keep narrowing it down until you know exactly where the problem is. Once you know exactly where the problem is, you should now know what’s going wrong if not exactly why. It’s not unusual to have to trace back through your code to find the place that set up the issue that wasn’t triggered until later. Bad config, for example, may not cause an actual bug until long after it’s saved.

The most important things you can remember when you’re debugging are to be systematic and to not make assumptions. Don’t assume that your input is good. Don’t assume that a certain piece of code can’t be the problem because it hasn’t been changed in ages/just passed testing/doesn’t seem to be related. Don’t assume that your config is what you think it is. Don’t assume that you know what’s going on – if you did, you wouldn’t have written a bug in the first place :)

Let’s try it and see what happens

in development

On Saturday I mentored at the Ladies Learning Code workshop for National Learn to Code Day. As usual it was a great experience, although pretty draining for an introvert like me, but what I want to talk about is the single phrase I used most often with my learners: let’s try it and see what happens.

Now, I’ll freely admit that a good portion of the times I said that were because I didn’t have a lot of experience with the language we were using (python, if you’re curious) and just didn’t know what would happen :) But the bigger part was that I wanted to show the learners that it’s okay to not know what’s going to happen and to mess around and try stuff.

It’s normal to be worried that you’ll look stupid or completely break your program, but the only way you can learn a new programming language (or any language, for that matter), is by trying things and making mistakes. The great thing about programming languages is that failure is just about consequence free. If you get an error message, so what? It’s not even a waste of time, you learned something that doesn’t work. That gives you one less thing you need to try next time.

Even if you’re an experienced developer, you need to keep that willingness to try things and see what happens. Analysis paralysis is what happens when you’re afraid to try something. If you’re working on a large project it’s true that you need to plan it out carefully and run your idea past other developers, but sooner or later you’ve got to start building something even if you’re not absolutely sure it’ll work. The sooner you try it, the sooner you can figure out whether or not it’ll work and the sooner you can try something else. Thinking things through is great but you can’t let it stop you from doing something.

Sometimes no amount of planning can tell you more than a few hours of trying. This is especially true when you’re just starting out, but it doesn’t stop being true just because you’ve got a solid handle on how loops work. And honestly, even the most experienced developers get off by one errors sometimes :)

When you’ve been a developer for long enough, sometimes you lose track of what it was like to be a beginner. We start assuming that we should know exactly what will happen before we even write a line of code even though that’s not even slightly how it works. I feel like I’m raining on new devs parades here, but the uncertainty never goes away. You just work on a different scale when you’re more experienced. Instead of “what happens when I change this == to a >=?” it’s “what happens when I try to use this new library?” or “what happens when we try to make our system talk to this other system?”

The next time you catch yourself slipping into analysis paralysis (and I do all the time), just try something and see what happens.

The mythical man month

in development, opinion, project management

In the programming field, it’s pretty rare to find a book that’s still relevant even five years after it was published. The Mythical Man-Month is still useful forty years after it was first published, which is either amazing or depressing depending on how you look at it.

What depresses me about how useful the book still is so long after it was written is that in the forty years since, we clearly haven’t learned that much about how to run projects. I first tried to read The Mythical Man-Month ages ago, and I got so frustrated about how little we’ve learned since it was published that I had to set it down. Then I completely forgot about it for a few years because I’m just that organized :) I finally finished it this year, and it’s just as relevant as ever. By all rights everything in that book should be totally obvious and taken as a given by every project manager out there, but sadly it’s not. It is, however, kind of comforting to know that other people have run into the same problems I have.

The amazing part of The Mythical Man-Month is how clearly it shows that the actual programming is the easiest part. Code is simple compared to trying to coordinate a large team and hit a deadline, but as programmers we seem to get hung up on the easy part and largely ignore the hard part. One of the many terrible ironies of programming is that a field that attracts introverts who just want to be left alone to code actually requires huge amounts of communication if you want to get anything meaningful done. For me and probably for most other programmers the code is the fun part, so it’s understandable that we’re not as good at communication as we should be but eventually we’ll grow up as an industry, right?

As sarcastic as that sounds, I think a large part of what we need to do as an industry is accept that things just aren’t as easy as we wish they were and learn to work around it. It’s pretty similar to the way children grow up and understand that the world isn’t as simple as they thought and learn to work around it. I and every other programmer ever have massively underestimated how long something was going to take, and no doubt  we’ll all keep doing that. But understanding that underestimating tasks is common allows you to leave extra space in the schedule and/or prioritize features so you know what can be cut if you aren’t going to hit your deadline.

Basically we just need to learn to account for human nature. Should be easy :)

Underrated dev tool of the day

in development, tip of the day, underrated dev tool of the day

You might not expect it, but thesaurus.com is actually a really useful dev tool. Like they say, naming things is one of two hard problems in computer science. When I have an object that’s difficult to think of a good name for, I just plug whatever vague description I have into thesaurus.com and poke through synonyms until I find something I don’t hate. Try it, it really does make it easier to pick a name :)

The Visitor design pattern

in design patterns, development

Let’s talk about the visitor design pattern. This one is tough to find a real-world analog for, but once it clicks for you it can be really useful.

Basically the visitor pattern is used when you want to add functionality to an object or a set of objects without changing the object itself. If you have a list of objects and you want to perform a particular calculation on each one and might need to add more calculations later, you would use the visitor to ‘visit’ each one, get its state, and do the calculation. Headfirst design patterns uses the example of a menu item and an ingredient both needing a nutrition information report. It’s ugly to add essentially the same method to two classes and it violates the single responsibility principle to add a method that does something the base object shouldn’t know about.

Uncle Bob uses the example of building a report with data from hourly but not salaried employee objects. The employee object shouldn’t know anything about how to generate a line for that report or even whether it should be on the report at all, so it’s much cleaner to use a visitor to get each object’s data and use that to build the report.

So how does the visitor pattern actually work? First of all you need some interfaces or abstract classes for your other classes to inherit from. To keep using Uncle Bob’s employee example, you need an employee interface with an accept method that takes a visitor interface/base class as a parameter. Your visitor interface needs a visit method for each type of object it’s going to visit. Now that you have your interfaces set up, concrete employee classes like SalaryEmployee and HourlyEmployee can implement the accept method. All you need to do there is call visit(this) on the visitor that was passed in. Your concrete visitor class just needs to implement visit methods for each visited object and you’re done! There’s a nice clean class diagram here in case seeing it makes it easier to understand.

Why so many interfaces? Wouldn’t it be simpler just to call methods on the concrete objects directly? Not so much. We need an accept method on the employee interface so that the code that iterates over your list of employees and calls accept(visitor) on each one doesn’t have to know what type of employee it is. We need a visitor interface separate from the concrete class/es so that we can add more visitors without having to change the visited objects and because the visited objects shouldn’t be coupled to the visitor object. In the report example, the employee objects shouldn’t know anything about the visitor that compiles the report. To put the single responsibility principle another way, the report just isn’t any of the employee object’s business. If we decide to change the way that report is calculated or add another version of it, the employee object shouldn’t know or care.

Why not put the accept(visitor) method in the employee base class? Because then it would only be able to pass an Employee to the visitor, which is no good if different types of employees need to be treated differently. Yes, it’s duplicate code, but only a tiny little of it and in this case it’s necessary.

Aside from keeping your base objects from having multiple responsibilities, another thing the visitor pattern gives you is one class where all the related methods live. To stick with the report example a little longer, we’ve got all our report logic in one place instead of scattering it over different classes. Yay single responsibility! If we decide to change how we calculate rows in that report, we only have to change one class. If we only have two types of employees that may not sound like a big gain, but what if we need to add contractors and consultants and seasonal employees and part time employees?

The visitor pattern is a really weird concept at first but once you understand it, it can make your code a lot cleaner.

SICP: first impressions

in development, SICP

Not so long ago I decided to try working my way through Structure and Interpretation of Computer Programs, hereafter referred to as SICP because that’s an awful lot of typing. Why SICP? Because I hear it will make me a better programmer and because I’ve heard about it enough times that I want to know what I’m missing :)

Thanks to the wonders of the internets, you can get the book at the link above and MIT-Scheme or SCM scheme. If you’re running windows like me, I don’t so much recommend MIT-Scheme. I have heard that it’s the only one you can do all of the exercises from SICP in (I haven’t gotten far enough into the book yet to see if that’s true, but I’ll let you all know when I do), but the documentation for it is kind of painful and in general I’d rather take up cottage cheese sculpture than learn emacs.

SCM is much less painful to run once you get it installed, but that isn’t as easy as it looks. The quick start for windows section currently says to install slib-3b4-1.exe and then scm-5f2-1.exe, but that’s the wrong version of slib. You want slib-3b5-1.exe from http://people.csail.mit.edu/jaffer/SLIB.html, then you can install scm successfully. After that make sure scm scheme has been added to your path, then you can use whatever editor you want to work with your scheme files and run them from the command line with scm -f <filename>

For an editor, I personally like Sublime, although I’ll be honest, I’m cheap and I think $70 is a bit steep for a text editor. It does have excellent plugin support though, as well as a much nicer interface than free editors like Notepad++.

If you decide to go with Sublime, I recommend installing Package Control and then the scheme package to add scheme syntax highlighting. Once you have package control installed, go to preferences->package control and click “Package Control: Install Package” in the menu that will pop up. It lags a little bit on my machine but in a second or two a new popup will appear that lists all the packages you can install. Search for scheme and install that package. Once you restart you’ll have scheme syntax highlighting and paren matching, which is pretty handy.

When you’re editing your scheme file, it will probably be helpful to know that you print to standard out with (display <arg>) and print a carriage return with (newline). You can skip printing your output if you use the MIT-Scheme interpreter, but then you’d have to use the MIT-Scheme interpreter :)

My workflow so far is edit in sublime, run the file with scm from the command line (scm -f <filename>, don’t forget), then look at the output and go back to sublime.

Now that the environment setup is out of the way, let’s talk about scheme. It’s been almost ten years since I touched any dialect of Lisp and it’s incredibly weird working with it again. Normally I write methods/functions from the top down, but scheme forces me to write them from the center out. It’s also pretty strange to work in a language with so little syntax. There’s almost nothing to scheme but a few basic functions and shit-ton of parens. It’s kind of beautiful, but compared to java’s creepy obsession with boilerplate it’s really, really weird. I don’t know yet if SICP will actually make me a better programmer, but it’s definitely forcing me to look at problems from a different perspective.

So far SICP has covered pretty basic parts of programming, but if it’s been a while since you used scheme I recommend plodding through the refresher. And check your work with google, there are a ton of solutions out there for all the exercises, including this handy wiki. There are a couple of tricky questions about normal-order and applicative-order in the first chapter that I got completely backwards the first time, so check your work and run an example to make sure you’ve got a handle on it.

Expect to hear a lot more about SICP from me in the future, I have plenty of chapters to go.