melreams.com

Nerrrrd

Programming is actually a creative field

There’s this stereotype that programming isn’t a creative field, that programmers do nothing but mechanically assemble code all day. I find that really sad, I think if we did a better job of explaining how creative programming actually is many more people would be willing to give it a shot.

If you’ve only ever had a total beginner intro to programming, it might be really hard to see where the creativity comes in. Honestly, variables are pretty boring and conditionals aren’t much better. Programming gets a lot more interesting once you’ve mastered the basics, I swear :)

Programming is a bit like building things with lego blocks, except that you have to make all the blocks yourself. Building the blocks – that is, writing an if statement or creating a variable – isn’t that interesting, I’ll be honest. But once you get good at creating those blocks, that’s when you get to be creative. Just like you can build a castle or a siege engine or an entire lego Westeros out of simple little blocks, you can build amazing things out of variables and loops and conditionals if you’re patient.

It’s also a bit like pottery or wood working. Just because you’re constrained to making a usable cup or chair doesn’t mean you can’t be incredibly creative within those constraints. Even in visual art or writing you need sentences that make sense and a combination of shapes and colours that work. Jokes about modern art aside, you can’t just throw paint in the direction of the canvas and expect anyone to care what you’ve done.

There are always constraints on anything you make, programming just has particularly rigid ones. An imperfect sentence is still intelligible, but a missing semicolon will keep your code from compiling at all. For some people that’s more frustration than they care to deal with, for others it’s an interesting challenge.

Building your own project that does anything you want it to is pretty obviously creative, but what about programming at work where you’re given assignments?

Even the least interesting internal application still requires creative problem solving to add new features or fix bugs. There have been times when I’ve had to be very creative to change an application in a way that meets the new requirements without breaking anything that used to work and without making a horrifying mess of the code. There are often quick and dirty ways to make a change that just leave you with more trouble in the long run, and sometimes (in an emergency, for example) they’re the least bad option, but usually you think about not just how you can make the change that’s necessary right now but how you can set things up so that you can make more changes in the future without tearing your hair out.

There’s never just one way to do that, either. When you’re working through a beginner tutorial it will probably look like there’s one right way to build any application and you can’t be creative at all once you’ve decided what you’re going to build and what the user interface should look like. That’s totally untrue. Once you’re working on anything larger than a simple assignment to write a for loop, you enter the world of trade-offs. There’s never only one way to solve a problem and each solution has its own pros and cons.

For example, optimizing code so it runs faster involves a lot of decisions about trade-offs and a lot of creative problem solving. Optimizing code usually makes it harder to read, which makes it harder to update if you need a new feature or find a bug. This matters a whole lot when you’re running a business because programmer time is so expensive. On the other hand, if your program runs so slowly that no one wants to use it (and pay for it!), it doesn’t matter how nice the code is to work with. Sometimes you absolutely need your code to run as fast as possible and unclear code is worth it to get your game to run at a decent frame rate. Other times performance isn’t your highest priority and what you really need is to be able to read and change your code quickly because you get requests for new features all the time.

In short, I build things all day. How is that not creative?

Linux tip of the day

For Eclipse users new to Linux like me, it can be pretty frustrating when you try to install some updates and Eclipse just tells you “Insufficient access privileges to apply this update.” I don’t know about you, but I run Eclipse as admin on Windows, which should give you some idea what the problem is :) Yep, on Linux you just need to run “sudo eclipse” from the terminal. Thank you stackoverflow. Don’t forget to close Eclipse and restart normally once you’re done updating, running as root will make your sysadmin sad.

Windows to Linux: first impressions

Like I mentioned in my last post, I’ve been experimenting with Linux. Somehow I broke my development environment at work and stopped being able to run our whole unit test suite. Well, technically I could run it, but it took a minimum of 40 minutes and a good quarter of the tests failed, so effectively I couldn’t run the test suite at work. Even at home, where I hadn’t broken my environment, it took a good 18-20 minutes to run the whole thing. The other upside of running it at home was that I could tell the tests were finished running when my laptop stopped running its fans like it was trying to lift off. As an aside, if any of my readers just happen to be experts in running the Play Framework on Windows, particularly the test server used for unit tests, I would really love to hear from you. The test suite runs great on Linux, but it really bothers me not to understand why I couldn’t fix it on Windows.

a dirt road bordered by trees and grass leads around a corner
Unrelated image from pexels.com

So far, the switch to Linux has been eerily easy. Because I haven’t done much of anything with Linux since I graduated from Camosun in 2006, I was really worried that getting all three of my monitors at work to actually work would be a terrible experience. What actually happened was that my boss offered to give me the script he uses to get all three of his monitors configured, but by the time he got a cup of coffee and came back I had installed the proprietary drivers for my video card and restarted my window manager and everything just worked.

If you normally run Linux that proprietary drivers bit probably gave you pause. I’m running Linux Mint, which I’m told is a good distro for beginners, particularly if you’re used to Windows. It’s also convenient (as are a number of other low-maintenance distros) if you’ve been running Linux for years but you’re just not interested in screwing around getting things working again every time there’s an update.

The thing that makes Linux Mint so convenient for Windows users is the same thing that makes GNU so unhappy with them. To quote GNU’s explanation of why they don’t endorse other common distros:

Mint does not have a policy against including nonfree software, it includes nonfree binary blobs in drivers packaged with the kernel, and it includes nonfree programs in its repositories. It even includes proprietary codecs.

Totally open source and free software is a nice ideal, but I need my shit to work. My job is to build features, not to screw around trying to get stuff working. As much as Mint may offend free as in freedom software purists, it gives me the “one-click and it works” functionality I’m used to. Think of it as a gateway drug if you need to :)

Once I got all of my monitors working, Mint has been extremely easy to work with. My development environment includes java, eclipse, mongo, redis, and the play framework, all of which were incredibly easy to install and get running. Even as a newbie to Linux, I found mongo’s instructions for installing old versions easy to follow. I did hit a little bit of a snag when I discovered that the environment variables in my .profile need to be exported, but that wasn’t difficult to figure out and it was still less of a hassle than setting up environment variables on Windows.

With everything set up, our entire unit test suite runs in in less than half the time it takes on Windows and Eclipse starts much faster too. So far the minor inconvenience of installing Linux has been far outweighed by being able to run the stuff I need to.

And the inconveniences have been incredibly minor. I needed to install a driver to get all of my monitors working, and I found and installed Gnome Do because I can’t deal without a SlickRun alternative, and I’m still looking for an email/calendar client I really like (right now I’m trying Geary, which is okay but I’m not in love with it), but overall switching to Linux from Windows has largely been a non-issue.

Installing Mint on my laptop (an Alienware 17″ with a GeForce GTX 860M that came with Win 8.1) did kinda suck, to be fair. I had to mess around with my harddrive partitions a bit and it didn’t want to install from the USB drive I set up with UNetbootin so I had to resort to burning a Linux Mint iso on a dvd and using that, but at work I was able to install from the USB key to my desktop no problem. I had my environment mostly set up before our morning meeting, although it did take until after lunch to get all the fiddly little details sorted out. Things are still a little bit weird on my laptop – sometimes my window manager freaks out when I open the lid but it’s Linux on an SSD so it’s not as if a restart takes long.

All in all, I thought switching to Linux was going to be a lot harder and I’m still kind of waiting for the other shoe to drop. Isn’t this supposed to be more of a pain in the ass? But seriously, if you’re running Windows I really do recommend trying Linux.

Linux app of the day

I’ve been experimenting with switching to Linux (well mostly, I still need Windows for games), and so far it’s been weirdly easy. The only real problem I’ve run into so far, and it’s not much of a problem, is that I really love SlickRun and there’s no Linux version. Fortunately there is something SlickRun-like, Gnome Do. It works great for me on Linux Mint and it was really easy to change the settings so I can use the same keyboard shortcut I did to bring up SlickRun. I haven’t used it for anything besides launching programs yet, but it looks like it can do some really cool stuff.

Failure

Everybody screws up sometimes and I wish we talked about it more. One of many things that I think scares off new programmers is the belief that if you ever fuck up then you’re just not cut out to be a programmer. That’s totally not true, even experienced professional programmers make mistakes. All making a mistake means is that you tried to do something that’s actually difficult, it’s not the end of the world.

a relaxing view of rolling hills dotted with trees and a field with a flock of sheep and a shepherd
Look at some relaxing sheep from pexels.com while we talk about making mistakes.

In the spirit of helping other people not freak out about their own mistakes, let’s talk about some of mine.

Early in my career I came up with a truly terrible design for a portlet (remember portlets? No? Okay, I feel old now). If I remember correctly it was supposed to provide a friendly UI for people to create their own report queries, so there were all sorts of rules about which kinds of values applied to each field and how fields could be combined to keep track of. This was back before I really understood that requirements never ever stay the same, and after a couple rounds of changes my design was a real mess. If nothing had ever changed my house of cards might have been okay, but since things did it started to crack and warp under the strain.

What did I learn from that one? That requirements always change and your design needs to handle that, and that it’s always a good idea to have someone else review your design before you start building, especially if you don’t have a lot of experience. Designing software to handle requirement changes is something I’m still working on and something that could fill a whole book, let alone a blog post, so I’ll just say that asking yourself what happens if you need to add more rules or take some away is a good start.

Another thing I messed up I already mentioned in my post If it’s hard to explain, it’s probably a bad idea. To recap really quickly, we (the backend team on the project) came up with a scheme to sort players into rooms that was basically impossible to explain to non-programmers. Actually, I’m not sure it ever even made sense to the devs on the client team either. I bring that one up again to drive home the point that design is hard and you’re going to screw it up a lot before you get good at it.

What I learned from that mistake was that designing in isolation can really trip you up. That design seemed like a good idea to us backend devs in isolation, but if we had run it past the rest of the team first we could have figured out it was confusing and a bad experience for the users before we wasted weeks building something that ended up being confusing and frustrating for the people who actually used it.

I’ve also really fouled things up with databases. I once took down production by making a data model change that had run fine in test. Turns out things run differently when a) there’s actual load on the database, and b) there are orders of magnitude more rows in prod than in test. Now I know not to try to make data model changes in mysql cluster without taking the app down for maintenance.

Sadly, that’s not the only time a difference between prod and test has tripped me up. The amount of data in your database can change the way mysql optimizes its queries, which can lead to some really interesting results when you assume the database is using an index that it’s actually blissfully ignoring. And by interesting I mean unusably slow response times. Wheee.

What that one really drove home for me is that you can’t trust the test database unless there’s a full production dump in there and you’re simulating realistic load. Outside of a perfect clone of production, load included, you need to lean on explain plans. Explain plans are great because they tell you how things are going to run into production without actually migrating your whole application to prod and breaking everything. Next time, I’m running explain plans first in production instead of assuming things will be fine because it worked in a test database with a few hundred rows.

Another really entertaining screw up I’ve had was when we were preparing to turn off an online game that wasn’t performing well and was a maintenance nightmare on top of that. We thought that it would be a nice gesture to make the virtual game cards (kind of like a virtual scratch ticket) in the game free so they could play as much as they wanted until we turned the game off entirely. The design of the game didn’t allow us to make the cards entirely free, but it did allow us to make everything cost only one token. Normally higher level game cards cost hundreds of tokens, so we figured players were still getting a pretty good deal.

Unfortunately, I didn’t think through the consequences of players being able to buy hundreds more game cards than usual. On each card you could win credits (thankfully you couldn’t buy more cards with those credits or we really would have been in trouble) that you could use to buy prizes in the store. Normally the number of credits you could possibly win at one time was limited by how many cards you could afford, with the higher level ones that gave out lots of credits being so expensive that you could only buy a few at a time.

Does anyone see where this is going? Yes, overflows! We let players win so many credits that the fields in the database couldn’t hold their winnings. Once we fixed the database we discovered that parts of the game server code couldn’t handle those numbers either. It was a rough few days trying to get that game back into action.

There are two main things I learned from that experience, one specific to the problems we had and one less so.

One – do not make sweeping changes to a game’s economy! Just don’t do it, it’s a bad idea and will make you unhappy. The next time we sunsetted a game (the industry’s polite euphemism for killing it) we most certainly did not destroy the economy on the way out.

Two – the consequences of an action aren’t always obvious, you’ve got to put some work into figuring out what’s going to happen when you change things. If I had that particular change to do over again, I would look at how many daily tokens our top players were getting and calculate how many game cards they could buy with those tokens and how many credits they could win off of those game cards. Then I would try manually giving myself that many credits and tokens in test and see what happens. After watching everything blow up, I would go back to the product manager and say “1 credit game cards break the game, how about we just give players 50% off?”

Let’s mix things up a little. I have one more failure to talk about, but this time it’s not one of mine. Here’s the notification I got about a sad little troll trying to comment on my Shitty hackathon! post:

New comment on your post “Shitty hackathon!”
Date: 10 February, 2016 11:43
Author: Nancy Grace (IP: 72.39.146.43, d72-39-146-43.home1.cgocable.net)
Email: toolofoppression@gmail.com
URL:
Comment: I must say, that I am impressed. Keep it up, and one day you just might earn your official script kiddie participation award. In the meantime, don’t quit your night prostitution job just yet ;)

Now, it’s pretty clear that this shitheap is unable to learn (if he was, he would be good enough at his job not to be frightened of a woman talking about code), but other readers can learn something from this. First of all, it’s pretty clear that he’s terrified of women developers. That in turn makes it obvious he sucks at his job, because with the shortage of developers you have to be a serious fuckup to be afraid a girl will come and steal your job. It’s also pretty sad that he had to go digging three pages back from my front page. Poor bastard had to go past two of my posts about SOLID design principles and my post about Java synchronization and my post about different languages being good for different things before he found something that he thought he could smear his poop on like the shitty little baby he is.

If you’re going to try to shit on me for not being a perfect programmer, get it right! I’m embarrassed for you and I don’t even like you, fuckstick.

However, fuckstick does accidentally bring up something in the vague (astronomically vague) vicinity of a point: if you talk openly about your failures, it’s possible someone will give you shit for having the unbelievable gall to be a human instead of a robot. You should ignore those people because they’re stupid :) No seriously, you do not want to work with or for anyone who is so willfully ignorant of what software development actually is that they think it’s possible to do this without ever making mistakes. If god forbid your boss shows you that they know that little about development, they’ve given you the gift of complete certainty that you need to run. Everybody in Victoria is hiring right now and developers are in demand all over, there’s no reason to put up with a boss who doesn’t understand their industry.

Another reason to talk openly about your failures, especially on your blog, is that when the classic interview question comes up, you’ll have something to say already lined up. You could even share a link to your blog post if you’re less of a potty mouth than I am :)

How about you, readers? What have you screwed up and what did you learn?

Development is maintenance

Professional programming and the kind of programming you learn in college/university/bootcamp/etc are actually very different things. Despite what you learned in school, development is really maintenance. In other words, I’m here to crush your dreams :)

So, you know how in school you started new projects from the ground up all the time? Yeah, you’ll hardly ever do that at work.

Now, sometimes you will need to research new technologies and/or frameworks and starting a new prototype project is usually a part of that, and sometimes even the largest and slowest moving organization needs to start something completely new, but that’s generally a very small part of the job.

What you’ll actually spend most of your time doing as a professional developer is adding features and fixing bugs in an existing product. That’s such a large part of the job that I wish we’d done more (or any) of it in school. For anyone reading this who is learning to code, I strongly recommend taking sample projects or open source projects or whatever you can get your hands on and adding new features or fixing bugs. Learning to read other people’s code is hugely important and you may only barely touch on it in your studies.

To be fair, just learning to code takes a lot of time and you can only cram so much into any one program without keeping students there for years and years, but I wish we put a little more emphasis on what software developers actually do at work most of the time. I also wish we’d spent more time on why design is such a big deal.

One of the consequences of hardly ever starting completely new projects at work is that the few projects that do get started are extremely long lived. Instead of a tiny throw-away project that you spend maybe a week building and then never touch again, you’ll work with applications that live for years or even decades. This can be really weird to adjust to since the lifespan of those projects means every tiny decision you made in five minutes can come back to haunt you for years to come :)

On the other hand, long lived projects have a much greater impact than tiny little throw-aways. If you do a good job, the code you write can make people’s lives a little bit easier for years and years. You can also build much larger things, whether they’re applications, games, frameworks, or something else entirely, when you have years to work on something. Corporate software development isn’t all bad, you get to work on things that you could never build on your own.

Another way professional development is different from school projects is that requirements always, always change. Even if every feature you add is perfect and bug free (ha!), your users are going to ask for new things and/or discover that the feature they asked for isn’t actually what they needed and the business might expand into new areas and the laws that your business has to follow might change. Sometimes technical requirements even drive changes: if a new version of your database or framework or a library you depend on comes out, eventually you’re going to want to switch to that.

The requirements changes can be infuriating, I’m not going to sugar-coat that. But at least you get to work on something that people care enough about to ask for changes, even if sometimes it seems like they have no idea what they actually want. If you never had to change a piece of software, all that would mean was that absolutely nobody was using it. I  don’t know about you but I think it’s pretty cool that people actually use the stuff I build.

Real world software development is very, very different from what you do in school, so don’t be surprised if it takes you a little while to find your feet. As much as it can be frustrating sometimes, there are some really cool upsides too.

Dev tool of the day

You know what’s incredibly helpful? RequestBin! Why is it so great? Because testing webhooks sucks and RequestBin makes it easy. Logging your output is a good start but that can’t tell you which IP your request is actually coming from. RequestBin can, which is awesome when you’re trying to figure out whether the Elastic IP you set up in AWS is working correctly. It also shows you all of your output (in a nice human-readable format, no less), which is handy if you really want to know exactly what your client receives.

It’s also free! On the downside your destination url only lasts for 48 hours and your data may be wiped out at any time (if you need a permanent solution look at Runscope’s Request Captures – seems only fair to plug the paid version when I’m talking about how helpful the free one is), but the price is right :)

If it’s hard to explain, it’s probably a bad idea

One of the things I struggled with when I was new to programming was how to tell whether a given piece of code is good or not. When everything is new and confusing, how do you tell bad confusing from normal confusing?

One thing that will give you a very helpful hint is if you code is hard to explain. Like this Python style guide puts it:

If the implementation is hard to explain, it’s a bad idea. This is a general software engineering principle — but applies very well to Python code. Most Python functions and objects can have an easy-to-explain implementation. If it’s hard to explain, it’s probably a bad idea. Usually you can make a hard-to-explain function easier-to-explain via “divide and conquer” — split it into several functions.

Basically, if something you’re working on is hard to explain, that’s a sign that it needs to be re-thought. Some problems are just unavoidably complex, but it should still be possible to explain what you’re doing at a high level.

That applies to higher level application logic just as much as to individual functions. At a previous job I worked on a multiplayer game project that involved putting groups of players into rooms together for each round, then closing down that room when the round was over and creating a new one. Our first implementation seemed like a good idea at the time, but when we got the game to a point where the team could play together, we had a terrible time explaining how players were sorted into rooms the to the artists and project manager.

The non-programmers on the team were by no means stupid people and had been working on the game for quite a while by that point. The fact that we couldn’t explain our room selection scheme to them was a very strong sign that what we were doing just didn’t make sense. As we kept playing our in-progress game, it also turned out that it was extremely difficult to get the whole team into the same room. There were less than a dozen of us working on that game, so there was really no good reason for it to be that hard to play together.

In the end, we admitted our room selection logic wasn’t working and rewrote it to be much simpler. Players sometimes had to wait a bit longer for a round to start, but they could play with their friends more easily and stay in the same room with the people they played with last round. The simpler logic that was easier to explain was also a better experience for the players.

I’m not going to pretend you’ll never run into anyone who is invested in not understanding what you’re trying to explain to them, but if you give someone an overview of what you’re up to and they don’t follow it, think about whether you’re doing something overly complicated before you assume the person you’re trying to explain it to is just dumb. Complicated code is harder to test, harder to debug, and harder to change when you get a new feature request. It’s worth paying attention to seemingly unimportant signs like having a hard time explaining your code to someone else because it can save you so much time in the long run.

Android app of the day

One of my very favourite Android apps is actually one of the ones I use the least. Gentle Alarm does what it says on the tin, it wakes you up gently instead of startling you awake like normal alarms. I don’t know about you but I really hate starting my day grumpy about how I got jolted awake.

Gentle Alarm is a bit more complicated to set up than the usual “blare loud noises” alarms, but I think it’s worth it not to wake up angry. I also don’t completely trust the “safe alarm” option – last time I tested it, which to be fair was ages ago, it was barely loud enough to hear. I’d still set a regular alarm for when you seriously need to be upright – it’s still annoying, but not nearly as bad as starting with the “blare loud noises” option.

There’s also a free version to test it out. The free version doesn’t ring on Wednesdays, but since it’s Thursday you’ve got almost a week to test it out :)

Productivity is overrated

A little while ago I posted a productivity tip and immediately started worrying that I sounded like one of those awful productivity gurus who preach maximum productivity all day every day what do you mean leisure has value?

I want to be very clear here: my worth as a human being is not measured by how much stuff I get done and neither is yours. My sole interest in productivity is breaking the shame spiral.

The shame spiral I’m talking about is the horrible feedback loop where you procrastinate, feel ashamed of not getting enough done, procrastinate more because you associate the work with shame, feeling more ashamed because you’re still not getting anything done and on and on. I don’t want you to be the perfect [insert work here]-producing machine, I want you to be happy and to feel good about what you’ve done at the end of the day. In fact, I actively want you to not be a perfect work-producing machine because that’s no way to live. If you don’t enjoy your life and have time for friends and family (chosen or assigned) and hobbies it doesn’t matter how much you get done.

You will never ever hear productivity tips from me about shit like how much work you can get done if you start waking up at 5 am or how you should listen to professional development audiobooks on your commute. You are allowed to listen to music and relax on your way to work. You do not have to cram something useful into every minute of every day.

A question I hardly ever see anyone ask in articles or blog posts about productivity is why you want to be more productive. Everyone just assumes that getting more done must automatically be a good thing. Getting stuff done is a great feeling if it’s something important to you and you’re not pushing yourself into burnout, but you know what else is great? Downtime. Playing a boardgame with your friends or trying out a new restaurant or going for a walk or reading a novel.

Personally, I do feel unfulfilled if I do nothing but play computer games when I come home from work for too many days in a row. I like the sense of accomplishment that comes from making progress on personal projects. But just like computer games can’t be my whole life, neither can work. That’s just not healthy and that lack of perspective leads to shitty work anyway.

Before you look at a productivity tip, ask yourself why you want to get more done. If you have a project that you want to finish, great! But if you feel guilty if you’re not doing something useful every minute of every day or you don’t want your parents to be disappointed in you or you feel like you’ll be left behind if you don’t spend every minute working, you officially have my permission to say “Fuck it, I’m going to go play videogames.” Don’t be productive for the sake of being productive, do stuff that’s either personally meaningful to you or fun.