DebugBreak and ASSERTs that work always anywhere

Introduction

Are you sure that your assertions or DebugBreak's will always work when running outside a debugger? You write the code, place ASSERTs all through it, compile it, and test it usually starting it under a debugger. But for a number of reasons, you don't always run the application being tested under a debugger. It may be tested by your QA team or you're testing a plug-in DLL which is loaded by a host application. This way or the other, in the case your condition fails to be true, you would expect to see the assertion failure dialog so that you could easily break into a debugger and locate the problem. Usually it works. However sometimes, having seen the assertion failure window, you try to attach a debugger but it won't work...

Let's find out why it may happen and how it may be solved. I assume that you use Visual Studio .NET 2003 debugger and you are familiar with Win32 SEH (Structured Exception Handling). For information on SEH, read Jeffrey Richter's "Programming Applications for Microsoft Windows" or Matt Pietrek's article "A Crash Course on the Depths of Win32 Structured Exception Handling".

Solution

The problem actually goes wider then just assertions. To let you debug the code right on the spot, ASSERT macros use the breakpoint software exception. Unfortunately, you may not be able to break into a debugger even if you have hard-coded a breakpoint. And the reason may be exception handling. It is pretty a usual situation that the code from which your function is called could look something like that:

Collapse | Copy Code

// ... __try
{
    // calling your function     YourFunction ( );
}
__except ( EXCEPTION_EXECUTE_HANDLER )
{
    // handle an exception raised in your function }
// ...

The developer of this snippet intends to catch all exceptions that may be raised in your code and don't let them break his own. As a breakpoint is just another software exception (with code 0x80000003), it will be handled by this 'catch-all' exception handler leaving you no chance to break into a debugger. It is quite a widespread situation unfortunately. You will also encounter it if you develop an ATL COM out-of-proc server.

Compile as a console application and run outside a debugger the following code (for convenience #includes are excluded from the samples):

Collapse | Copy Code

void Foo()
{
    // ....     DebugBreak();
    //... }

int _tmain(int argc, _TCHAR* argv[])
{
    __try
    {
        Foo();
    }
    __except ( EXCEPTION_EXECUTE_HANDLER )
    {
        printf ( "In main handler/n" );
    }

    return 0;
}

In the output, you should see:

Collapse | Copy Code

Enter Foo
In main handler

That means that our breakpoint has been caught by SEH filter set up in the main function, the filter indicates that it can handle all the exceptions and hence the control is passed to the __except block and then the program finishes safely. Normally, you would expect the Application Error dialog box to show up that lets you to terminate the program and, if you have a debugger installed, to debug it.

So let's try to show this dialog to get back a chance to debug the code.

The Application Error dialog is launched in a standard API function called UnhandledExceptionFilter. This function is used as an exception filter for the default exception handler in the internal routine BaseProcessStart in the kernel32.dll. I'm not going to explain it here - for more information, look for the MSDN or better read Jeffrey Richter's "Programming Applications for Microsoft Windows".

Now let's just use this function! Rewrite the Foo as follows:

Collapse | Copy Code

void Foo()
{
    __try
    {
        printf( "Enter Foo/n" );
        DebugBreak();
        printf( "Leave Foo/n" );
    }
    __except ( UnhandledExceptionFilter( GetExceptionInformation() ) )
    {
        printf( "In Foo handler" );
    }
}

Compile and run the application in a command window again. Now the dialog box shows up stating that a breakpoint has been reached and giving a chance to push OK to terminate the application or Cancel to debug it (it may look a bit different on different Windows versions).

OK, let me first terminate the application. What I see in the output is:

Collapse | Copy Code

Enter Foo
In Foo handler

This is because when you press the OK button, the UnhandledExceptionFilter returns EXCEPTION_EXECUTE_HANDLER, so our exception handler in Foo gets the control. Run the app again and this time press Cancel. The Visual Studio debugger starts and seems like it's going to stop at our breakpoint. However, this does not happen. Instead, the program just finishes its execution and the debugger ceases. The reason is that when you press Cancel, the UnhandledExceptionFilter starts up the debugger, waits for it to attach to the application, and then returns EXCEPTION_CONTINUE_SEARCH. This forces the system to look for the next exception handler and hence again the __except block in the main function gets the control. You can make sure of it if you have a glance at the output. It looks like this:

Collapse | Copy Code

Enter Foo
In main handler

A nice question to be asked here is why then a debugger attaches properly when some other application fails and the default exception handler fires in the BaseProcessStart? The answer is that the default exception handler is the last handler in the list of all exception handlers and there's no more handlers to pass the control to. Even though the UnhandledExceptionFilter returns EXCEPTION_CONTINUE_SEARCH, there's nothing to search for. So the system assumes that a debugger has been attached and tries just to re-execute the fault-causing code again, and this time your breakpoint is caught by the debugger as a first-chance exception.

OK, we got the standard failure window showing up and notifying us about our breakpoint but we still can not break into a debugger. The solution may seem very obvious: writing a wrapper around the UnhandledExceptionFilter which returns EXCEPTION_CONTINUE_EXECUTION when the UnhandledExceptionFilter returns EXCEPTION_CONTINUE_SEARCH. This will make the system re-execute the faulting instruction. The wrapper function may be like this one:

Collapse | Copy Code

LONG New_UnhandledExceptionFilter( PEXCEPTION_POINTERS pEp )
{
    LONG lResult = UnhandledExceptionFilter ( pEp );
    if (EXCEPTION_CONTINUE_SEARCH == lResult)
        lResult = EXCEPTION_CONTINUE_EXECUTION;
    return lResult;
}

void Foo()
{
    __try
    {
        printf( "Enter Foo/n" );
        DebugBreak( );
        printf( "Leave Foo/n" );
    }
    __except ( New_UnhandledExceptionFilter( GetExceptionInformation() ) )
    {
        printf( "In Foo handler" );
    }
}

Rebuild and run the application outside a debugger. This time you successfully break into your debugger and it stops just right at the line with DebugBreak. Nice! We got our breakpoint working.

But how do we change the ASSERT macros to make them work with this DebugBreak which really always debugbreaks. We could write our own function which sets up __try/__except block and uses New_UnhandledExceptionFilter, then make an assertion with the help of, for example, _CrtDbgReport in DCRT. Yes, all that is possible but it is sure ugly! We need something pretty. OK, forget the New_UnhandledExceptionFilter. What we really need is a new function -let me name it- DebugBreakAnyway :).

I'd played a bit around with the code before I figured it out, and ended up with a helper function which I called PreDebugBreakAnyway and the macro DebugBreakAnyway that you can place in your code. First, I'd like to show the macro:

Collapse | Copy Code

#define DebugBreakAnyway()            /
    __asm    { call    dword ptr[PreDebugBreakAnyway]    }        /
    __asm    { int 3h                            }        /
    __asm    { nop                                }

As you can see, the first action is to make call to the PreDebugBreakAnyway function. Then int 3h initiates the breakpoint exception on x86 processors. Why did I write it using assembler? Well, I was playing quite enough time with the code and was trying to save the EAX register and... Whatever, it does not matter now. This is what remained after everything. Surely it's not portable across various CPUs, so you can define it like this:

Collapse | Copy Code

#define DebugBreakAnyway()    /
    PreDebugBreakAnyway();    /
    DebugBreak();

That's it. Now it's portable.

Let me show you the PreDebugBreakAnyway.

Collapse | Copy Code

void __stdcall PreDebugBreakAnyway()
{
    if (IsDebuggerPresent())
    {
        // We're running under the debugger.         // There's no need to call the inner DebugBreak         // placed in the two __try/__catch blocks below,         // because the outer DebugBreak will         // force a first-chance exception handled in the debugger.         return;
    }

    __try
    {
        __try
        {
            DebugBreak();
        }
        __except ( UnhandledExceptionFilter(GetExceptionInformation()) )
        {
            // You can place the ExitProcess here to emulate work             // of the __except block from BaseStartProcess             // ExitProcess( 0 );         }
    }
    __except ( EXCEPTION_EXECUTE_HANDLER )
    {
        // We'll get here if the user has pushed Cancel (Debug).         // The debugger is already attached to our process.         // Return to let the outer DebugBreak be called.     }
}

Though I've put some comments in it, let's examine what it does:

1. Checks whether a debugger is attached to the process (IsDebuggerPresent comes in handy here). If it is, the function just returns allowing the DebugBreak that just follows it to be called. So if you run under a debugger, it looks to you as if a usual breakpoint has been reached. One note to mention here is that the IsDebuggerPresent works only under Windows NT/2000/XP/2003. So if you need it under Win9x, consider this article.
2. Sets up two __try/__except blocks. The inner block allows the Application Error dialog to show up and ask the user if she wants to terminate or run a debugger. If she wants to debug, the UnhandledExceptionFilter returns EXCEPTION_CONTINUE_SEARCH, and the outer __except catches the breakpoint exception and doesn't let it fall through further. Then the control returns from PreDebugBreakAnyway and the outer DebugBreak fires which is handled by the already attached debugger as a first-chance exception, and you get right at the place in the source code where you've put DebugBreakAnyway. The DebugBreakAnyway macro just makes it appear as if breakpoint occurred right in your source file.

   If the user wants to terminate the faulting application, the inner __except will be executed so you may wish to place ExitProcess in there to avoid the application to continue running.

Place this DebugBreakAnyway wherever you like and you always get your breakpoints working. But I began this article with assertions. Well, to make your assertions work just as wonderful, you'll have to substitute DebugBreak in the standard ASSERTs macros. If you use DCRT, you can write your ASSERT macro in stdafx.h:

Collapse | Copy Code

#define  MY_ASSERT(expr) /
    do { if (!(expr) && /
        (1 == _CrtDbgReport(_CRT_ASSERT, __FILE__, __LINE__, NULL, NULL))) /
        DebugBreakAnyway(); } while (0)

#define ASSERT MY_ASSERT

In short, it depends on you and what assertions you use. As to me, I use the best assertion on earth ;-) - John Robbins' SUPERASSERT with his BugslayerUtil.dll. So I personally slightly modified it for my own use to make it work with the DebugBreakAnyway.