Feel like starting with a quote, so here you go.

Quote: Brian Kernighan Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

In a certain way, there's a pretty big nugget of truth in this. Anyway, I'm in the middle of debugging a pretty complicated program, and I have ironed out most issues. I suspect that I'm down to the very last bug, but it is this programs 'big bug'. Not that it causes a giant explosion that crashes the program every time, but that it is subtle enough and rare enough that I have difficulty trusting the output of my program, even if it looks right. Something is amiss. It is driving me crazy. Everything *looks* right. Everything appears to be working in proper order, and occasionally I am wondering if I'm chasing a ghost here. I've been kicking this thing around for weeks now. This bug is the "big foot" of bugs. So here's the question, what do you do when you are working on a project and are confronted with a bug that has so confounded you that you're completely out of mental ammo to fire at it? Not *urgent* bugs... because those are usually pretty easy... *HARD* bugs. The rare stuff. The stuff that only happens one time in a million. Do you make a note, and brush it under the carpet with faith that inspiration will strike you later? Maybe a bug fix later will make this bug more obvious, or manifest more frequently... Hold a team meeting, and recruit other people to spend their time on it? Maybe you turn your attention to the user of your program, and start prodding them about what *they* did wrong with it? [if it even has users in the general sense]. Me, for now I'm going to shoot off some emails to the maintainers of the projects on which I am relying that could possibly be the cause of my headache. I'm going to be notifying them that I am having problems, and not accuse them of it. Ask if they have heard of issues manifesting similar symptoms as I am describing from others. At that point, I'm going to make a note and kick it down the road. I'll be hoping that I stumble upon a set of circumstances that makes things more easily reproducible. Or, rather since my problem crops up on a very long running program, an execution that manifests my problem quickly. For the time being, this bug is going to be an asterisk. A 'known issue' if you will.

Regardless of how a bug manifests, diagnosing and fixing it requires careful attention to the details of how and when it manifests - not necessarily how easy it is to reproduce.

In other words, we can't really even suggest a decent place to begin unless we know what exactly is going wrong and how you know there's a bug in the first place. A bug cannot be separated from its symptoms - they must always be treated as aspects of each other. To solve a bug without knowing its symptoms is a logically vacuous concept.

Not expecting you to fix my problem. Was just wondering how you react when you're stumped cold.

Quote: Original post by Drigovas
Not expecting you to fix my problem. Was just wondering how you react when you're stumped cold.

By clearing out all my assumptions, restating the nature of the problem as clearly and in as much detail as I can, and analyzing from there [wink]

This may or may not be an option for you, and I know it's not what you were looking for, but when I'm at work, and I hit one of those bugs that just seems completely out there (by the way chances are it's not your last bug ever in this program), I've found that if I pray for help, with the motivation of doing an effective job, the bug gets solved before the day's end. And some of these bugs definitely have the potential to take much longer than a day's worth of work to hammer out.

I don't know that it'll always work out this perfectly but so far it's been flawless.

Also, can you not rewrite some suspicious parts of code to be less "clever" and more clear. I know that some projects can be really huge, but sometimes I can't figure out what that 1200 line function (not even source file) is doing until I go through and refactor it.

Quote: Original post by ApochPiQ

Quote: Original post by Drigovas
Not expecting you to fix my problem. Was just wondering how you react when you're stumped cold.

By clearing out all my assumptions, restating the nature of the problem as clearly and in as much detail as I can, and analyzing from there [wink]

Basically, this ^

Sometimes, as a part of the above, I tend to move on to something else which needs doing and let my subconscious figure things out (which its pretty good at).

I've written my best code lately, and I can tell you, the more complex the program, the bigger the bug, no doubt about that.

But, if you got a bigger bug, just invest time designing a better mouse trap.

Create your own variation of a memory allocation system, of a function call-log, thread-lock manager, etc...

A particular suggestion might not be useful in your case, but something will be.

Embarrassingly enough asking someone else to have a thorough look at it usually solves it faster than if I continue with the same godawful bug chase I've at that point been doing for a couple of hours or in worst case days. I can only stand reading the same code over and over that many times before my mental precision drops to unacceptable levels. A lot of the time the problem is also in some code I'm intuitively assuming to work, when it doesn't. Someone else with a fresh mind can go in and see that much more clearly.

IMO there's a flaw in the original statement that debugging is twice as hard as writing code in the first place. True, some bugs - especially if they're hardware/workstation-specific things can be a real pain, but I'd say most of the (my) bugs are simple in nature - forgetting to initialize something to NULL, accidentally using a wrong iterator in a nested loop and writing outside of array bounds or accessing a NULL pointer without first checking it. All things that a debugger will happily tell me about.

The toughest cases I've had were throwing an exception and not having a try-catch block to intercept it - the behaviour of the program was really weird in that case and it took me hours to figure it out; and the occasional inevitable logic error, which simply means I have to rewrite my code, not debug it - the code wasn't at fault: it was wrong in the first place and shouldn't have worked (IMO a bug represents a piece of code that should work, but doesn't). Sometimes it just makes more sense to delete everything and start off a clean slate.

edit: oh, in C++ that I work with I guess the nastiest bug that you really cannot do anything against (other than read over your code until you spot it) is writing outside of array bounds on line X and then accessing a totally unrelated array 15000 lines later only to discover that one of the variables is changing completely randomly without any cause whatsoever. What makes it so nasty is that it could be a difference of || vs &&, < vs <= or == vs !=. I've had this twice and that was truly horrible. I guess I've grown aware of this now and can usually figure out what's going on before scraping my eyes out.

One of the best few weeks I ever spent was on developing a nigh-transparent exception handling mechanism - it's saved me days if not weeks worth of headaches already. Cuts a critical 1.5x off of that 2x - combine it with the debugger and there'll only be code you can write and code you can't write in the first place :)

Dont you love those bugs that dont happen when you put a breakpoint, or only happens on a PS3 Final Build?

I can think of 2 ways of dealing with those. One is to output a crapload of debug output and take the time to parse it. The other one is commenting/disabling just about everythink you can to reduce the complexity. If the bug/crah/leak disappear, you can re-enable the features one by one til you find the culprit :)