Alright, I guess I can give some details :)

It’s amazing how helpful just turning up your log level can be when you’re working on a weird bug. If something you can’t immediately explain is happening, try turning up your log level. In java, where I have the most experience, it’s unusual to run your production logging at a level above warn or debug. Normally you wouldn’t want extremely verbose logs, which are what you get when you turn up the log level, but sometimes you really need that extra information.

I wouldn’t normally think of turning up the log level, but we happened to have some trace level logging in our code and when I ran into a weird bug. I thought it would be easier to turn up the logging than to change all the .traces to .debugs, and it turned out the debug level exception gave me much less information than the trace level exception which had been swallowed because we were logging at the debug level. The trace level exception pointed me at the real bug, which turned out to be an obscure issue to do with my particular version of java having a weird interaction with a couple of libraries we’re using. Just because there’s no good reason for it to break doesn’t mean it won’t break :)

The moral of the story is that your log doesn’t necessarily tell you everything and you should turn up your log level until you get answers.