Mel Reams

Nerrrrd

Dealing with imperfect requirements

In a sunny green field, a wooden path curves into the distance. The sky is blue with fluffy white clouds.
Unrelated image from pexels.com to make this post look nicer in social media shares.

A very common question developers ask is what they can do about bad requirements. The internet definitely needs more opinions on it, so here’s mine!

The first thing you’ve got to do is accept you will never get perfect requirements. If anyone knew how to get perfect requirements every time no matter how large the feature was, you would know their name because they would be deservedly famous for finally fixing the requirements problem. Outside of school assignments, the requirements you get are practically always going to be incomplete in some way and probably wrong in at least one way too. It’s frustrating, I’m not going to lie, but that’s just how the job works.

I’m stressing that point so much because I want you to have realistic expectations about how much you can possibly improve your requirements gathering process. If you go in thinking you can ever get perfect requirements every time, you’re just going to be disappointed. On the other hand, if you have more realistic expectations about what kind of improvements you can make, you’ll end up a lot happier with your progress.

Now that I’ve gotten the disclaimer part of this post out of the way, there are some things you can do when you get incomplete requirements or when you work with people who often change their minds about what they want.

First of all ask questions! I touched on this briefly in my post about how communication can make you a better programmer, but it bears repeating. Communicating well is absolutely essential to being a professional programmer, and one of the many reasons it’s so important is that you need to make sure you completely understand the requirements before you build something. Don’t worry about looking stupid, ask questions about anything that you’re not sure you completely understand. Unless you work in a terrible toxic company, nobody is going to be mad that you cared enough about doing a good job to make sure you didn’t accidentally build the wrong thing.

Asking questions can get you quite a ways and is definitely worth doing, but it’s not always enough. Another really useful technique is mockups. When people can see an example and compare it to what they already have, they’ll understand it far better and will be able to give you much better direction about what they really need. It’s a lot of work for the human brain to have to imagine what the finished thing will look like, how it will work, and how you will use it all at the same time. Take a little cognitive load off of your requirements people by asking them to imagine fewer things and you’ll get better results.

There are tons of tools out there for mocking up UI, if nothing else you can throw together a quick slide deck or build a couple simple pages in HTML. Don’t forget to take a lot of notes when you walk your requirements people through your mockup, you’re working with a limited human brain too and you’re not going to remember every detail of the feedback they give you without notes.

After you’ve made some mockups and walked people through them, you may want to build a prototype before building the real feature. Once people can actually use something, it becomes real to people in a way written requirements just aren’t. It’s not because they’re dumb or bad at requirements, that’s just how humans work. Building a prototype will also bring up all the technicaly questions you didn’t realize weren’t answered in the requirements until you started building the feature. Don’t pretend you’re perfect at requirements either :)

There’s another really useful technique for gathering requirements, but unlike questions, mockups, and prototypes, it takes more buy-in from your requirements person or people. That technique is user stories (you may encounter some confusion about use cases and user stories, which seem to be similar but not quite the same thing. I just found out when I was looking for a good link to explain further that the things we call use cases at work, other people call user stories).

To very briefly define user stories, they’re a particular way of stating requirements. The general form is: as a_____ I want to _____  so I can_____. For example: as a blogger, I want to schedule posts for the future so my posts go up on a consistent schedule. It’s amazing what comes up when you think about what you users want to accomplish and why. That kind of context is really important to understand the intent of the feature, but it does take more work on the part of the person who comes up with your requirements and writes them down.

It’s pretty common for people to see all this question asking and mockup making and user story writing as a waste of time that just slows down development. If you’re willing to put in the effort, it can really help to record just how much development time gets wasted because the requirements are bad or get changed all the time. Developer time is really expensive, showing your manager exactly how much developer time gets wasted on building the wrong thing then throwing it out when you get better requirements will likely convince them that it’s worth putting a few more hours into getting better requirements.

Requirements are fundamentally a hard problem you can do everything right and still get a horrible surprise when you’re halfway through building a feature. Like everything else in development, there’s no silver bullet.

Linux tip of the day

Take out the trash! Unlike Windows, Linux (at least my distro, Mint) doesn’t have a recycling bin/trashcan icon on the desktop to remind you to empty your trash directory once in a while. That really adds up if you forgot to empty your trash for, uh, a while. Technically you can just hunt down your trash directory and delete everything inside it, but do you really want to have to remember the path? And remember how to find it if you delete a file on a USB drive?

trash-cli to the rescue! The terminal command to install it is sudo apt-get install trash-cli (at least on systems with apt-get), and to run it you just type trash-empty. Thanks as usual to stack overflow for that answer.

When I finally ran that on my laptop, I got 3.6gb back. Not bad for under a minute of work :) If you’re new to Linux like me, don’t forget to take out the trash.

 

Back to basics: advanced Java generics

a white fox sits in the snow, looking toward the camera
Unrelated image from pexels.com to make this post look nicer in social media shares.

Last time I mentioned that there are bounded generic types and wildcards. I’ve never actually used one of those if I remember correctly, which I why I left them for a followup post.

Bounded generics are really cool – they let you narrow down which types you can use with a generic class or method while still having the flexibility to work with different types in the range you’ve defined. There are lower bounded and upper bounded generics. Upper bounded generics specify a superclass your generic type must extend and lower bounded generics specify a class your generic class must be a superclass of.

This means that you can do more interesting stuff with your generic param because you know something about what it is. Let’s use WordPress as an example. If you had a theme superclass (or probably an interface) that was extended or implemented by, say, oneColumnTheme, twoColumnTheme, gridLayout, etc you could have a theme preview page that accepts anything that extends the base theme class and calls getPreviewImage and getDescription on every item passed in, even if they’re different subclasses.

That method signature would look like:

public Result createPreviewPage(List<? extends Theme> themeList) {
    //code goes here
}

You can’t add anything except null to a collection that has an upper bounded generic type, though. Why? Java inheritance – if you have a class hierarchy of Vehicle -> Car -> Sedan -> Toyota Corolla then that Toyota is definitely a Sedan, a Car, and a Cehicle, but a Vehicle could very well be a Motorcycle or a Bus, not any variety of Car.

On the lower bounded end of bounded generics, you could have a method that takes anything that’s a superclass of integer (which could be a number or object) and adds all the numbers from 1 to 10 to it. Yes, I stole this example (and the following code) from the official docs. The code for that method looks like:

public static void addNumbers(List<? super Integer> list) {
    for (int i = 1; i &lt;= 10; i++) {
        list.add(i);
    }
}

With a collection with a lower bounded type you can add things. Taking my Vehicle -> Toyota Corolla class hierarchy above, if you have a List<? super ToyotaCorolla> then you know what the most specific thing it can be is so you can safely add one of those.

You have have a lower bound or an upper bound but not both. You can have neither, though! That’s called an unbounded wildcard and it looks like:

public boolean removeAll(Collection<?> c);

That’s from the List source code. Unbounded wildcards are for when you really just don’t care what the thing is and when you want to make sure you don’t make any major changes to your parameter. The only thing you can do with a Collection<?> is add null, you just don’t know enough about what’s in there to add an object or any sort.

In my admittedly limited experience with bounded generics, upper bounded generics are much more common so you shouldn’t be too worried if you can’t find a use for a lower bounded generic parameter. If you’re not sure when to use which kind of wildcard/bounded generic, there’s a handy guide in the official docs. There are also some really in-depth articles about generics here and here if you’d like much, much more detail :)

Lastpass tip of the day

LastPass tries to be helpful by filling in form fields for you, but sometimes it gets it wrong. If you see it fill in the wrong data or if you just mysteriously struggle to sign into services that use 2FA (for me LastPass was helpfully inserting an old 2FA code when I tried to sign into AWS, which meant AWS thought my authenticator app was wildly out of sync and would make me resync it every time I tried to log in), follow these steps and see if it helps.

In case that link ever stops working, you’ve got to find the problem site in your vault, click the little wrench icon to edit that site, then click the little wrench icon again in the edit popup to edit form fields, then delete all the fields with bad values. There may be quite a few of these.

Credit where it’s due, one of my coworkers told me about this. Thanks Logan, I don’t know if I ever would have found that on my own!

Back to basics: Java generics

a crowd of people release paper lanterns with candles inside that make them float into the sky. the dark sky is full of small glowing lanterns floating away.
Unrelated image from pexels.com to make this post look nicer in social media shares.

Every so often you’ve got to go back to basics no matter how long you’ve been doing something. It’s amazing how much you forget when you haven’t done something in a while, even if it’s kind of a core feature of a language you use all the time. Like Generics in Java!

So let’s talk about Generics. First of all, what is a Generic? It’s just a placeholder for a type. Where you would normally put a concrete type like String or Integer, you can put T instead.

Why should we care, though?

In short, because class cast exceptions are a pain to deal with. Back in the dark time before Java 5, you could put any object you wanted in a Collection but for that to work with Java’s type system, all of the Collections classes had to work with Objects. That meant you could put anything in, but when you got it out all Java knew was that it was an Object, so you had to know what to cast it to to use it for anything interesting.

As a bit of an aside: yes you can use arrays (not to be confused with ArrayLists) to avoid dealing with casting, but then you miss out on all the handy stuff Collections do for you, like automatically resizing themselves when you add more items. Collections for the win!

Compared to having to resize arrays manually, having to cast your objects back to the object you really wanted when you take them out of a Collection doesn’t sound so bad, but here’s the big problem with that: what if you mess up somewhere along the line and try to cast that object to something it can’t be cast to? Then you get a ClassCastException, which is really irritating because it’s a runtime exception (I should write a post about exception handling in Java shouldn’t I) for something the compiler shouldn’t have let you do in the first place. Not finding out your code is wrong in a totally predictable way until you run it sucks.

Generics to the rescue! With Generics, you can tell a collection what kind of things go in it when you create it, and then not have to do any casts when you take them out because you can only put one (depending on how you count subclasses) type of thing in there.

Okay great, but how do you actually use Generics?

If you just want to put things in a Collection and get them back out, it’s really simple. In general using existing code that uses Generics is super easy.

List<String> names = new ArrayList<>();
names.add("Amy");
names.add("Brianna");
names.add("Cara");

String name = names.get(0);

While I’m at it the <> operator (aka the diamond operator) is great. Before Java 7 you had to specify the whole type in both the declaration and instantiation, which kinda sucked if you needed, say a Map<String, List<ReallyLongTypeHere>>.

Where things get a little more complicated is writing your own code using Generics. There are two places you can put generic types, on the class declaration and on the method declaration and the really fun part is you can put different types in each place. You could put a different generic type on each method if you wanted to, but that would probably be evil so don’t do it :)

Let’s look at the List source code for an example:

public interface List<E> extends Collection<E> {
    int size();
    boolean isEmpty();
    boolean add(E e);
    E get(int index);
    <T> T[] toArray(T[] a);
}

FYI that’s a tiny subset of all the methods on the List interface, I just didn’t want to list a ton of methods that aren’t relevant to this post.

When you put a generic type like E on the class (or interface!) declaration, what you’re saying is that this class primarily deals with Es. That way when you use the same generic in methods like add and get, it’s obvious what’s going on.

Why E instead of any other letter? It’s short for element. This post has a nice list of the naming conventions for generic types. I can’t link directly to that section, so just search for “naming convention” and you’ll find it.

The toArray method declaration shows how you can use another generic type just for one method even if the class already has a generic type. The <T> just means that method takes a generic type T, it’s separate from the return value. Basically, every time you use a generic type, you’ve got to have a <T> (or <E>, or <N>, etc) somewhere so Java knows you’re using a generic and doesn’t go looking for a class named T.

One thing that’s a little tricky with generic types is getting their Class object. You need a Class for things like using Jackson to convert Json into an object in your system, but you can’t do E.class. Luckily, there’s a way around that, you can use Class<E> like so:

//here's an example method using a generic Class
public static <T> T decode(String json, Class<T> destinationClass) { //code goes here }

//and here's how you call it
Result decoded = decodeUtil.decode(myString, Result.class);

I forgot about Class<T> once and made a real mess of my code, but at least you get to learn from my mistakes. A good rule of thumb for using generics is that if you still need to cast anything, something is wrong.

There are a bunch more fancy things you can do with bounded generic types and wildcards, but I’ll get to those in another post. What I’ve covered in this post is the majority of what you’re likely to need to do with Generics, you’ll need to get a handle on this stuff anyway before the advanced stuff makes sense.

App of the day

Today’s app is Authenticator Plus It’s available for both Android and iOS and will save you a lot of hassle if your phone dies on you like mine did. On an unrelated note, I do not recommend the Nexus 5x but you can’t buy new ones anyway so that’s kind of a wash.

Authenticator Plus, on the other hand, is great! It stores your accounts in your dropbox so you can simply sync them onto another device if you can’t use your usual device (don’t forget to factory reset it if you lost it!) instead of going through the slow and tedious process of using recovery codes to get back into your accounts. Don’t worry, you have to set a master password to access your accounts on a new device.

Full disclosure: Authy does the same thing and is free where Authenticator Plus is paid. One of my coworkers likes it and another one didn’t like the onboarding process, so you’re going to have to make your own decision there :) I started with Authenticator Plus and can’t be bothered to switch everything to Authy just to try it out so I’ll be no help there.

And while I’m at it, don’t use SMS for your two factor auth! Sure, odds are against anyone bothering to attack you personally but why take the risk?

What does reasoning about code even mean?

Unrelated image from pexels.com to make this post look nicer in social media shares.

Programmers talk about “reasoning about code” all the time, but what does that actually mean? Is it really just pretending to be a CPU and working out exactly when your for loop ends?

Yes and no. Figuring out exactly what your loops are up to is part of it, but at a higher level it’s also about how your methods, objects, modules, functions all work together. Like this stackoverflow answer says, writing code isn’t just about cranking out something that compiles, you’ve also got to be able to figure out if it’s actually right, if it performs well enough, if it can be scaled, if it’s vulnerable to bad data (whether that’s malicious or accidental).

Sadly, tools can only help you so much with everything that comes after getting your code to compile. The compiler can tell you whether your code will run, but it can’t tell you if it does what you meant. Automated tests are a huge help with that, but those tests are code themselves and you also need to be able to reason about them to make sure you’re testing what you meant to test and that the test itself isn’t broken.

To figure out what your code is doing, you need to be able to read and understand it well enough that you can predict what it will do for any given input. That’s basically all “reasoning about code” is. If you can reason about your code, you can change it or add more features without breaking things or spending hours and hours cursing at your computer and howling “whyyyyy won’t this work?!”

So being able to reason about your code is really useful, but how do you do it? Practice, mostly. If you’re a beginner programmer you should not worry even a little bit about understanding a whole codebase or heck, even a whole class. Focus on one method or one loop or one if statement at a time. When you get enough practice, you’ll be able to understand those really quickly and you can move up a level to figuring out what a whole method does with different inputs or how two methods in the same class work together.

Oh, and it really helps if the code you’re trying to reason about is good code (that is, well organized, has good names, etc). Some code is next to impossible to reason about because it’s full of giant methods or the variable names aren’t helpful at all or, and this is one of the worst problems, variables change all the time in unpredictable ways. A “total” variable that changes value each time through a loop isn’t so bad, it’s clear what it’s for and why it changes. On the other hand a “total” variable that gets reused for completely different totals at different places in a method is a real problem. Humans can only hold so many things in their working memory at once and “wait where does total change again?” takes up most of them. If you have a terrible time reasoning about a certain piece of code, it’s entirely possible the problem is not you but the code.

In cases like that when I don’t have time to refactor the code so it’s easier to read I find it really helpful to make a lot of notes and draw myself a map through the code. If you want to be really helpful you can even type up your notes / make a diagram out of your map and add it to your developer documentation.

Reasoning about code can be hard to do and takes practice, but it’s not some sort of magic that only “real” programmers can do.

How does a thread work, anyway?

three large spools of thread, one orange, one purple, and one blue, against a white background
See what I did there? Image from pexels.com to make this post look nicer in social media shares.

Threads are used a lot in java, so I should probably understand how they actually work. They’re one of those things you use all the time without thinking about what’s really happening under the covers. I know threads and processes are related, but not exactly how.

First of all, what’s a process? To understand threads inside a process you need to understand processes first.

A process is an individual thing that’s separate from all the other processes running on your computer. In most cases a process is a single program running on your computer, but some programs have many processes – Chrome, for example, has a process for each tab. To quote the official tutorial on concurrency in Java: “A process has a self-contained execution environment. A process generally has a complete, private set of basic run-time resources; in particular, each process has its own memory space.” As I understand it, the separate memory space is the really important part of a process – this keeps processes from overwriting each other’s memory, crashing your computer, and wrecking your day.

Threads, on the other hand, are just parts of the parent process that execute semi-separately and all use the same memory. This means they can change a variable another thread just set, cause wildly bizarre bugs, and wreck your day. Every process has at least one thread, but can have more.

Okay, so why would you want to use threads when you could use processes to keep everything separate? Because processes use up a lot more system resources than threads, it’s more work to get them to communicate with each other, and they’re just overkill for a lot of problems. If you’re building in a search feature in your app and you want to search by a few different things (like name, address, and ID number) from one search box, creating a whole process for each search is way more work than you need to do when you could just use threads. Do you really want to write a whole separate program for each search? I sure don’t.

So threads are convenient, but what are they really? Saying they’re a single thread of execution through a program is all well and good, but what does that really mean? To quote stackoverflow: “A thread is a basic unit of CPU utilization; it comprises a thread ID, a program counter, a register set, and a stack.” Those things are also called the execution context, because they’re everything you need to know about the executing program to stop it and start it again in exactly the same state it was in. The CPU does that a lot so it can appear to do two things at once (if you have a multicore processor, it actually can do more than one thing at once). If one thread is waiting for data to come back from the database, which takes forever from the perspective of a CPU, the CPU will essentially make a note of where the thread got to in the code and what values all of its variables had, then start executing another thread using the “notes” it made about that one.

Because threads all share the same address space, they can share information really easily by updating global variables. In a process, a memory address (aka a variable) will probably mean nothing to another process. In a thread, any variable that’s in scope can be used and (if it’s not final) changed. This is really handy for stuff like web programming – if a thread that’s handling a particular HTTP request needs a reference to the database service, it can just grab one from the controller/servlet/general parent object that contains methods for handling individual requests.

When a thread completes, its variables get garbage collected as usual (assuming nothing else has a reference to them), you don’t have to do anything special to clean up after it.

And finally, while the tutorial I linked above talks about Thread objects and Runnables, in modern Java you mostly use CompletableFutures and let them handle starting and stopping threads for you.

Let’s play CPU!

white ferris wheel against a blue sky with a few wisps of white clouds
Unrelated image from pexels.com to make this post look nicer in social media shares.

I think one of many reasons it’s so hard to learn to code is that the people doing the teaching have all forgotten what it was like not to know how to code, so we skip over some really important basic steps because we think they’re inherently obvious. Case in point, this poor redditor who nobody ever taught how to work through a piece of code with a pencil and paper. I had to read the question a couple of times to understand what they were even asking, it just didn’t occur to me that someone wouldn’t know how to trace through code manually.

The example that user posted was “x=0; y=0 While (x<7) { x=y-x; y=x+x; x= x+ y + 1;}”

First of all, let’s clean that up a little:

x=0;
y=0;
while (x < 7) {
    x = y - x;
    y = x + x;
    x = x + y + 1;
}

In the beginning x is 0, which is definitely less than 7 so we enter the loop.

Inside the loop, take each statement and swap out the variables for their current values and then do the math and put that value in the variable:

x = y – x becomes x = 0 – 0 becomes x = 0
y = x + x becomes y = 0 + 0 becomes y = 0
x = x + y + 1 becomes x = 0 + 0 + 1 becomes x = 1
At the end of the loop x = 1 and y = 0

Now we check the condition at the top of the loop again and x is still less than 7 so we go through the loop again

x = y – x becomes x = 0 – 1 becomes x = -1
y = x + x becomes y = -1 + -1 becomes y = -2
x = x + y + 1 becomes x = -1 + -2 + 1 becomes x = -2
At the end of the loop x = -2 and y = -2

At this point, I’m getting very suspicious this loop will never exit. I don’t think x is ever going to be greater than or equal to 7, but just to be sure let’s go through the code one more time

x is still less than 7 so we go through the loop again

x = y – x becomes x = -2 – -2 becomes x = 0
y = x + x becomes y = 0 + 0 becomes y = 0
x = x + y + 1 becomes x = 0 + 0 + 1 becomes x = 1

At the end of the loop x = 1 and y = 0. These are the same values we had after we went through the loop the first time, so now we can be completely sure the loop never ends. After each time through the loop x is either 1 or -2, there’s no way for it ever to be 7 or more.

That’s it. Working through code with a pencil and paper is just keeping track of what values all of your variables have and swapping out variables for their values. It’s really tedious, that’s why we make computers do it :) And to be clear, you’re never expected to be able to work through complicated code like that purely in your head. Everybody else needs a pencil and paper too, there are just too many details to keep them all in your working memory.

Okay, so if working through code with a pencil and paper sucks so much, why does anyone do it? Because it gives you a really solid understanding of what the computer is doing, which you’ll need later when you start building more complicated things. For a simple loop like the one above you can just throw that snippet of code into an IDE, run it, and see what happens, but you can’t even start building bigger things without a foundation of knowing what the computer is doing when it runs your code.

Project of the day

Somehow I came across a link to this one episode of a podcast called Reply All. That particular episode is about Anxiety Box, a delightfully odd project by Paul Ford. He was dealing with some pretty serious anxiety and being a nerd, decided to write a program to be anxious for him. It’s a really interesting story, and while Anxiety Box is now defunct, all the code is available so you can run your own external jerkbrain if you like.

WordPress Appliance - Powered by TurnKey Linux