内存泄露检测

http://wyw.dcweb.cn/leakage.htm

 

A Cross-Platform Memory Leak Detector

 

Memory leakage has been a permanent annoyance for C/C++ programmers. Under MSVC, one useful feature of MFC is report memory leaks at the exit of an application (to the debugger output window, which can be displayed by the integration environment or a debugger). Under GCC, current available tools like mpatrol are relatively difficult to use, or have a big impact on memory/performance. This article details the implementation of an easy-to-use, cross-platform C++ memory leak detector (which I call debug_new), and discusses the related technical issues.

 

Basic usage

 

Let’s look at the following simple program test.cpp:

 

int main()

{

    int* p1 = new int;

    char* p2 = new char[10];

    return 0;

}

Our basic objectives are, of course, report two memory leaks. It is very simple: just compile and link debug_new.cpp. For example:

 

cl -GX test.cpp debug_new.cpp (MSVC)

g++ test.cpp debug_new.cpp -o test (GCC)

The running output is like follows:

 

Leaked object at 00341008 (size 4, <Unknown>)

Leaked object at 00341CA0 (size 10, <Unknown>)

If we need clearer reports, it is also trivial: just put this at the front of test.cpp:

 

#include "debug_new.h"

The output after adding this line is:

 

Leaked object at 00340FB8 (size 10, test5.cpp:5)

Leaked object at 00340F80 (size 4, test5.cpp:4)

Very simple, isn’t it?

 

Background knowledge

 

In a new/delete operation, C++ compilers generates calls to operator new and operator delete (allocation and deallocation functions) for the user. The prototypes of operator new and operator delete are as follows:

 

void* operator new(size_t) throw(std::bad_alloc);

void* operator new[](size_t) throw(std::bad_alloc);

void operator delete(void*) throw();

void operator delete[](void*) throw();

For new int, the compiler will generate a call to “operator new(sizeof(int))”, and for new char[10], “operator new(sizeof(char) * 10)”. Similarly, for delete ptr and delete[] ptr, the compiler will generate calls to “operator delete(ptr)” and “operator delete[](ptr)”. When the user does not define these operators, the compiler will provide their definitions automatically; when the user do define them, they will override the ones the compiler provides. And we thus get the ability to trace and control dynamic memory allocation.

 

In the meanwhile, we can adjust the behaviour of new operators with new-placements, which are to supply additional arguments to the allocation functions. E.g., when we have a prototype

 

void* operator new(size_t size, const char* file, int line);

we may use new ("hello", 123) int to generate a call to “operator new(sizeof(int), "hello", 123)”. This can be very flexible. One placement allocation function that the C++ standard ([C++1998]) requires is

 

void* operator new(size_t size, const std::nothrow_t&) throw();

in which nothrow_t is usually an empty structure (defined as “struct nothrow_t {};”), whose sole purpose is to provide a type that the compiler can identify for overload resolution. Users can call it via new (std::nothrow) type (nothrow is a constant of type nothrow_t). The difference from the standard new is that when memory allocation fails, new will throw an exception, but new(std::nothrow) will return a null pointer.

 

One thing to notice is that there is not a corresponding syntax like delete(std::nothrow) ptr. However, a related issue will be mentioned later in this article.

 

For more information about the above-mentioned C++ language features, please refer to [Stroustrup1997], esp. sections 6.2.6, 10.4.11, 15.6, 19.4.5, and B.3.4. These features are key to understanding the implementation described below.

 

Principle and basic implementation

 

Similar to some other memory leakage detectors, debug_new overrides operator new, and provides macros to do substitues in user’s programs. The relevant part in debug_new.h is as follows:

 

void* operator new(size_t size, const char* file, int line);

void* operator new[](size_t size, const char* file, int line);

#define DEBUG_NEW new(__FILE__, __LINE__)

#define new DEBUG_NEW

Let’s look at the test.cpp after including debug_new.h: new char[10] will become “new("test.cpp", 4) char[10]” after preprocessing, and the compiler will generate a call to “operator new[](sizeof(char) * 10, "test.cpp", 4)” accordingly. If I define “operator new(size_t, const char*, int)” and “operator delete(void*)” (as well as “operator new[]...” and “operator delete[]...”; for clarity, my discussions about operator new and operator delete also cover operator new[] and operator delete[] without mentioning specifically, unless noted otherwise) in debug_new.cpp, I can trace all dynamic memory allocation/deallocation calls, and check for unmatched news and deletes. The implementation may be as simple as using just a map: add a pointer to map in new, and delete the pointer and related information in delete; report wrong deleting if the pointer to delete does not exist in the map; report memory leaks if there are still pointers to delete in the map at program exit.

 

However, it will not work if debug_new.h is not included. And the case that some translation units include debug_new.h and some do not are unacceptable, for although two operator news are used — “operator new(size_t, const char*, int)” and “operator new(size_t)” — there is only one operator delete! The operator delete we define will consider it an invalid pointer, when given a pointer returned by “operator delete(void*)” (no information about it exists in the map). We are facing a dilemma: either to misreport in this case, or not to report when deleting a pointer twice: none is satisfactory behaviour.

 

So defining the global “operator new(size_t)” is inevitable. In debug_new.h, I have

 

void* operator new(size_t size)

{

    return operator new(size, "<Unknown>", 0);

}

Implement the memory leak detector as I have described, you will find it works under some environments (say, GCC 2.95.3 w/ SGI STL), but crashes under others (MSVC 6 is among them). The reason is not complicated: memory pools are used in SGI STL, and only large chunks of memory will be allocated by operator new; in STL implementations which do not utilize such mechanisms, adding data to map will cause a call to operator new, which will add data to map, and this dead loop will immediately cause a stack overflow that aborts the application. Therefore I have to stop using the convenient STL container and resort to my own data structure:

 

struct new_ptr_list_t

{

    new_ptr_list_t*     next;

    const char*         file;

    int                 line;

    size_t              size;

};

Every time one allocates memory via new, sizeof(new_ptr_list_t) more bytes will be allocated when calling malloc. The memory blocks will be chained together as a linked list (via the next field), the file name, line number, and object size will be stored in the file, line, and size fields, and return (pointer-returned-by-malloc + sizeof(new_ptr_list_t)). When one deletes a pointer, it will be matched with those in the linked list. If it does match — pointer-to-delete == (char*)pointer-in-linked-list + sizeof(new_ptr_list_t) — the linked list will be adjusted and the memory deallocated. If no match is found, a message of deleting an invalid pointer will be printed and the application will be aborted.

 

In order to automatically report memory leaks at program exit, I construct a static object (C++ ensures that its constructor will be called at program initialization, and the destructor be called at program exit), whose destructor will call a function to check for memory leaks. Users are also allowed to call this function manually.

 

Thus is the basic implementation.

 

Improvements on usability

 

The above method worked quite well, until I began to create a large number of objects. Since each delete needed to search in the linked list, and the average number of searches was a half of the length of the linked list, the application soon crawled. The speed was too slow even for the purpose of debugging. So I made a modification: the head of the linked list is changed from a single pointer to an array of pointers, and which element a pointer belongs to depends on its hash value. — Users are allowed to change the definitions of _DEBUG_NEW_HASH and _DEBUG_NEW_HASHTABLESIZE (at compile-time) to adjust the behaviour of debug_new. Their current values are what I feel satisfactory after some tests.

 

I found in real use that under some special circumstances the pointers to file names can become invalid (check the comment in debug_new.cpp if you are interested). Therefore, currently the default behaviour of debug_new is copying the first 20 characters of the file name, instead of storing the pointer to the file name. Also notice that the length of the original new_ptr_list_t is 16 bytes, and the current length is 32 bytes: both can ensure correct memory alignments.

 

In order to ensure debug_new can work with new(std::nothrow), I overloaded “void* operator new(size_t size, const std::nothrow_t&) throw()” too; otherwise the pointer returned by a new(std::nothrow) will be considered an invalid pointer to delete. Since debug_new does not throw exceptions (the program will report an alert and abort when memory is insufficient), this overload just calls operator new(size_t). Very simple.

 

It has been mentioned previously that a C++ file should include debug_new.h to get an accurate memory leak report. I usually do this:

 

#ifdef _DEBUG

#include "debug_new.h"

#endif

The include position should be later than the system headers, but earlier than user’s own header files if possible. Typically debug_new.h will conflict with STL header files if included earlier. Under some circumstances one may not want debug_new to redefine new; it could be done by defining _DEBUG_NEW_REDEFINE_NEW to 0 before including debug_new.h. Then the user should also use DEBUG_NEW instead of new. Maybe one should write this in the source:

 

#ifdef _DEBUG

#define _DEBUG_NEW_REDEFINE_NEW 0

#include "debug_new.h"

#else

#define DEBUG_NEW new

#endif

and use DEBUG_NEW where memory tracing is needed (consider global substitution).

 

Users might choose to define _DEBUG_NEW_EMULATE_MALLOC, and debug_new.h will emulate malloc and free with debug_new and delete, causing malloc and free in a translation unit including debug_new.h to be traced. Three global variables are used to adjust the behaviour of debug_new: new_output_fp, default to stderr, is the stream pointer to output information about memory leaks (traditional C streams are preferred to C++ iostreams since the former is simpler, smaller, and has a longer and more predictable lifetime); new_verbose_flag, default to false, will cause every new/delete to output trace messages when set to true; new_autocheck_flag, default to true (which will cause the program to call check_leaks automatically on exit), will make users have to call check_leaks manually when set to false.

 

One thing to notice is that it might be impossible to ensure that the destruction of static objects occur before the automatic check_leaks call, since the call itself is issued from the destructor of a static object in debug_new.cpp. I have used several techniques to better the case. For MSVC, it is quite straightforword: “#pragma init_seg(lib)” is used to adjust the order of object construction/destruction. For other compilers without such a compiler directive, I use a counter class as proposed by Bjarne ([Stroustrup1997], section 21.5.2) and can ensure check_leaks will be automatically called after the destruction of all objects defined in translation units that include debug_new.h. For static objects defined in C++ libraries instead of the user code, there is a last resort: new_verbose_flag will be set to true after the automatic check_leaks call, so that all later delete operations along with number of bytes still allocated will be printed. Even if there is a misreport on memory leakage, we can manually confirm that no memory leakage happens if the later deletes finally report that “0 bytes still allocated”.

 

Debug_new will report on deleteing an invalid pointer (or a pointer twice), as well as on mismatches of new/delete[] or new[]/delete. A diagnostic message will be printed and the program will abort.

 

Exception safety and thread safety are worth their separate sections. Please read on.

 

Exception in the constructor

 

Let’s look at the following simple program:

 

#include <stdexcept>

#include <stdio.h>

 

void* operator new(size_t size, int line)

{

    printf("Allocate %u bytes on line %d/n", size, line);

    return operator new(size);

}

 

class Obj {

public:

    Obj(int n);

private:

    int _n;

};

 

Obj::Obj(int n) : _n(n)

{

    if (n == 0) {

        throw std::runtime_error("0 not allowed");

    }

}

 

int main()

{

    try {

        Obj* p = new(__LINE__) Obj(0);

        delete p;

    } catch (const std::runtime_error& e) {

        printf("Exception: %s/n", e.what());

    }

}

Any problems seen? In fact, if we compile it with MSVC, the warning message already tells us what has happened:

 

test.cpp(27) : warning C4291: 'void *__cdecl operator new(unsigned int,int)' : no matching operator delete found; memory will not be freed if initialization throws an exception

Try compiling and linking debug_new.cpp also. The result is as follows:

 

Allocate 4 bytes on line 27

Exception: 0 not allowed

Leaked object at 00341008 (size 4, <Unknown>)

There is a memory leak!

 

Of course, this might not be a frequently encountered case. However, who can ensure that the constructors one uses never throw an exception? And the solution is not complicated; it just asks for a compiler that conforms well to the C++ standard and allows the definition of a placement deallocation function ([C++1998], section 5.3.4; drafts of the standard might be found on the Web, such as here). Of compilers I have tested, GCC (2.95.3 or higher) and MSVC (6.0 or higher) support this feature quite well, while Borland C++ Compiler 5.5.1 and Digital Mars C++ compiler (all versions up to 8.38) do not. In the example above, if the compiler supports, we should declare and implement an “operator delete(void*, int)” to recycle the memory allocated by new(__LINE__); if the compiler does not, macros need to be used to make the compiler ignore the relevant declarations and implementations. To make debug_new compile under such a non-conformant compiler, users need to define the macro HAS_PLACEMENT_DELETE (Update: The macro name is HAVE_PLACEMENT_DELETE from Nvwa version 0.8) to 0, and take care of the exception-in-constructor problem themselves. I wish you did not have to do this, since in that case your compiler is really out of date!

 

Thread safety

 

My original version of debug_new was not thread-safe. There were no synchronization primitives in the standard C++ language, and I was unwilling to rely on a bulky third-party library. At last I decided to write my own thread-transparency layer, and the current debug_new relies on it. This layer is thin and simple, and its interface is as follows:

 

class fast_mutex

{

public:

    void lock();

    void unlock();

};

It supports POSIX threads and Win32 threads currently, as well as a no-threads mode. Unlike Loki ([Alexandrescu2001]) and some other libraries, threading mode is not to be specified in the code, but detected from the environment. It will automatically switch on multi-threading when the -MT/-MD option of MSVC, the -mthreads option of MinGW GCC, or the -pthread option of GCC under POSIX environments, is used. One advantage of the current implementation is that the construction and destruction of a static object using a static fast_mutex not yet constructed or already destroyed are allowed to work (with lock/unlock operations ignored), and there are re-entry checks for lock/unlock operations when the preprocessing symbol _DEBUG is defined.

 

Directly calling lock/unlock is error-prone, and I generally use an RAII (resource acquisition is initialization; [Stroustrup1997], section 14.4.1) helper class. The code is short and I list it here in full:

 

class fast_mutex_autolock

{

    fast_mutex& _M_mtx;

public:

    explicit fast_mutex_autolock(fast_mutex& __mtx) : _M_mtx(__mtx)

    {

        _M_mtx.lock();

    }

    ~fast_mutex_autolock()

    {

        _M_mtx.unlock();

    }

private:

    fast_mutex_autolock(const fast_mutex_autolock&);

    fast_mutex_autolock& operator=(const fast_mutex_autolock&);

};

I am quite satisfied with this implementation and its application in the current debug_new.

 

Special improvement with gcc/binutils

 

Using macros has intrinsic problems: it cannot work directly with placement new, for it is not possible to expand an expression like “new(special) MyObj” to record file/line information without prior knowledge of the “special” stuff. What is more, the definition of per-class operator new will not work since the preprocessed code will be like “void* operator new("some_file.cpp", 123)(size_t ...)” — the compiler will not love this.

 

The alternative is to store the instruction address of the caller of operator new, and look up for the source line if a leak is found. Obviously, there are two things to do:

 

Get the caller address of operator new;

Convert the caller address to a source position.

There is no portable way to achieve these, but the necessary support has already been there for ready use if the GNU toolchain is used. Let’s just look at some GNU documentation:

 

`__builtin_return_address (LEVEL)'

     This function returns the return address of the current function,

     or of one of its callers.  The LEVEL argument is number of frames

     to scan up the call stack.  A value of `0' yields the return

     address of the current function, a value of `1' yields the return

     address of the caller of the current function, and so forth.

 

     The LEVEL argument must be a constant integer.

 

     On some machines it may be impossible to determine the return

     address of any function other than the current one; in such cases,

     or when the top of the stack has been reached, this function will

     return `0'.

(gcc info page)

addr2line

*********

 

     addr2line [ -b BFDNAME | --target=BFDNAME ]

               [ -C | --demangle[=STYLE ]

               [ -e FILENAME | --exe=FILENAME ]

               [ -f | --functions ] [ -s | --basename ]

               [ -H | --help ] [ -V | --version ]

               [ addr addr ... ]

 

   `addr2line' translates program addresses into file names and line

numbers.  Given an address and an executable, it uses the debugging

information in the executable to figure out which file name and line

number are associated with a given address.

 

   The executable to use is specified with the `-e' option.  The

default is the file `a.out'.

(binutils info page)

So the implementation is quite straightforward and like this:

 

void* operator new(size_t size) throw(std::bad_alloc)

{

    return operator new(size, __builtin_return_address(0), 0);

}

When a leak is found, debug_new will try to convert the stored caller address to the source position by popening an addr2line process, and display it if something useful is returned (it should be the case if debugging symbols are present); otherwise the stored address is displayed. One thing to notice is that one must tell debug_new the path/name of the process to make addr2line work. I have outlined the ways in the doxygen documentation.

 

If you have your own routines to get and display the caller address, it is also easy to make debug_new work with it. You may check the source code for details. Look for _DEBUG_NEW_CALLER_ADDRESS and print_position_from_addr.

 

Important update in 2007

 

With an idea coming from Greg Herlihy’s post in comp.lang.c++.moderated, a better solution is implemented. Instead of defining new to “new(__FILE__, __LINE__)”, it is now defined to “__debug_new_recorder(__FILE__, __LINE__) ->* new”. The most significant result is that placement new can be used with debug_new now! Full support for new(std::nothrow) is provided, with its null-returning error semantics (by default). Other forms (like “new(buffer) Obj”) will probably result in a run-time warning, but not compile-time or run-time errors — in order to achieve that, magic number signatures are added to detect memory corruption in the free store. Memory corruption will be checked on freeing the pointers and checking the leaks, and a new function check_mem_corruption is added for your on-demand use in debugging. You may also want to define _DEBUG_NEW_TAILCHECK to something like 4 for past-end memory corruption check, which is off by default to ensure performance is not affected.

 

The code was heavily refactored during the modifications. I was quite satisfied with the new code, and I released Nvwa 0.8 as a result.

 

Summary

 

So I have presented my small memory leakage detector. I’ll make a summary here, and you can also consult the online doxygen documentation for the respective descriptions of the functions, variables, and macros.

 

This implementation is relatively simple. It is lacking in features when compared with commercial applications, like Rational Purify, or even some open-source libraries. However, it is

 

Cross-platform and portable: Apart from the code handling threading (which is separated from the main code) and providing special GCC support (which is automatically on when GCC is detected), only standard language features are used. It should compile under modern C++ compilers. It is known to work with GCC (2.95.3 and later), MSVC 6/7.1, and Borland C++ Compiler 5.5.1.

Easy to use: Because “void* operator new(size_t)” is overloaded too, memory leaks could be detected without including my header file. — I myself use it this way habitually in nearly every C++ program. — Generally, I check for the leak position only after I see memory leaks reported by debug_new.

Flexible: Its behaviour can be tailored by macros at compile time.

Efficient: It has a very low overhead, and it can be used in debugging applications that require high performance.

Open-source: It is released in the zlib/libpng licence and you have the freedom to use, change, or redistribute as you like.

With the recent improvements, some of the old restrictions are gone. The macro new or DEBUG_NEW in debug_new.h can mostly work if the newed object has operator news as class member functions, or if new(std::nothrow) is used in the code, though the macro new must be turned off when defining any operator news. Even in the worst case, linking only debug_new.cpp should always work, as long as the allocation operation finally goes to the global operator new(size_t) or operator new(size_t, std::nothrow_t).

 

Source is available, for your programming pleasure, in the CVS (most up to date) or download of Stones of Nvwa.

 

May the Source be with you!

 

Bibliography

 

[Alexandrescu2001] Andrei Alexandrescu. Modern C++ Design: Generic Programming and Design Patterns Applied. Addison-Wesley, 2001.

[C++1998] ISO/IEC. Programming Languages — C++. Reference Number ISO/IEC 14882:1998(E), 1998.

[Stroustrup1997] Bjarne Stroustrup. The C++ Programming Language (Third Edition). Addison-Wesley, 1997.

HTML for code syntax highlighting is generated by Vim

 

Note: Although I would love to, I did not succeed in making this page conform to the HTML 4.01 specification. I guess W3C is to blame. Why should <font color=...> be forbidden inside a <pre> block (Vim currently generate HTML code this way), while the clumsier <span style="color: ..."> is allowed? However, this is fixable after all (I have already written a converter indeed). There is worse to come: <nobr> is not a valid tag. When a major browser could break a line after a “-” and no mechanisms are provided by the standard to achieve the no-breaking effect, using something like <nobr> is inevitable (and I can name other cases where <nobr> is needed). I cannot fix the browser, so I have to choose to break the standard. Detailed online information about this problem can be found here.

 

2004-3, Chinese version first published here at IBM developerWorks China 

2004-11-28, rewritten in English (at last) by Wu Yongwei 

2007-12-31, last updated by Wu Yongwei

 

 

This work is licensed under a Creative Commons Attribution-Share Alike 2.5 Licence.

 

Return to Main

 

 

 

 

http://msdn.microsoft.com/en-us/library/x98tx3cf(v=VS.80).aspx

Memory Leak Detection and Isolation 

 

http://msdn.microsoft.com/en-us/library/e5ewb1h3(v=VS.80).aspx

Memory Leak Detection Enabling

The primary tools for detecting memory leaks are the debugger and the C Run-Time Libraries (CRT) debug heap functions. To enable the debug heap functions, include the following statements in your program:

 

Copy

#define _CRTDBG_MAP_ALLOC

#include <stdlib.h>

#include <crtdbg.h>

Note

The #include statements must be in the order shown here. If you change the order, the functions you use may not work properly.

By including crtdbg.h, you map the malloc and free functions to their debug versions, _malloc_dbg and _free_dbg, which keep track of memory allocation and deallocation. This mapping occurs only in a debug build (in which _DEBUG is defined). Release builds use the ordinary malloc and free functions.

 

The #define statement maps the base versions of the CRT heap functions to the corresponding debug versions. You do not absolutely need this statement, but without it, the memory leak dump will contain less useful information.

 

Once you have added the previous statements, you can dump memory leak information by including the following statement in your program:

 

Copy

_CrtDumpMemoryLeaks();

When you run your program under the debugger, _CrtDumpMemoryLeaks displays memory leak information in the Output window. The memory leak information looks like this:

 

Copy

Detected memory leaks!

Dumping objects ->

C:/PROGRAM FILES/VISUAL STUDIO/MyProjects/leaktest/leaktest.cpp(20) : {18} 

normal block at 0x00780E80, 64 bytes long.

 Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD

Object dump complete.

If you do not use the #define _CRTDBG_MAPALLOC statement, the memory leak dump would look like this:

 

Copy

Detected memory leaks!

Dumping objects ->

{18} normal block at 0x00780E80, 64 bytes long.

 Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD

Object dump complete.

Without _CRTDBG_MAP_ALLOC defined, the display shows:

 

The memory allocation number (inside the curly braces).

 

The block type, which is normal, client, or CRT.

 

The memory location in hexadecimal form.

 

The size of the block in bytes.

 

The contents of the first 16 bytes, also in hexadecimal form.

 

With _CRTDBG_MAP_ALLOC defined, the display also shows you the file where the leaked memory was allocated. The number in parentheses following the file name (20, in this example) is the line number within the file.

 

To go to the line in the source file where the memory is allocated

 

Double-click on the line in the Output window that contains the file name and line number.

 

-or-

 

Select the line in the Output window that contains the file name and line number, and press F4.

 

_CrtSetDbgFlag

Calling _CrtDumpMemoryLeaks is easy enough if your program always exits in the same place. If your program can exit from multiple locations, instead of putting a call to _CrtDumpMemoryLeaks at each possible exit, you can include the following call at the beginning of your program:

 

Copy

_CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );

This statement automatically calls _CrtDumpMemoryLeaks when your program exits. You must set both bit fields, _CRTDBG_ALLOC_MEM_DF and _CRTDBG_LEAK_CHECK_DF, as shown previously.

 

Setting the CRT Report Mode

By default, _CrtDumpMemoryLeaks dumps memory leak information to the Debug pane of the Output window, as described previously. You can reset this to dump to another location using _CrtSetReportMode. If you use a library, it may reset the output to another location. In that case, you can set the output location back to the Output window using the following statement:

 

Copy

_CrtSetReportMode( _CRT_ERROR, _CRTDBG_MODE_DEBUG );

 

http://msdn.microsoft.com/en-us/library/htdyz80k(v=VS.80).aspx

Memory Block Type Interpretation 

 

As seen in Enabling Memory Leak Detection, the memory leak information identifies each block of leaked memory as a normal block, a client block, or a CRT block. In practice, normal blocks and client blocks are the only types you are likely to see.

 

A normal block is ordinary memory allocated by your program.

 

A client block is a special type of memory block used by MFC programs for objects that require a destructor. The MFC new operation creates either a normal block or a client block, as appropriate for the object being created.

 

A CRT block is a block of memory allocated by the CRT library for its own use. The CRT library handles the deallocation for these blocks, so it is unlikely you will see these in the memory leak report unless something is seriously wrong (for example, the CRT library is corrupted).

 

There are two block types you will never see in the memory leak information:

 

A free block is a block of memory that has been released.

 

An ignore block is a block that you have specifically marked so it doesn't appear in the memory leak report.

 

 

http://msdn.microsoft.com/en-us/library/w2fhc9a3(v=VS.80).aspx

Set Breakpoints on a Memory Allocation Number 

 

The file name and line number in the memory leak report tell you where leaked memory is allocated, but knowing where the memory is allocated is not always sufficient to identify the problem. Often an allocation will be called many times during a run of the program, but it may leak memory only on certain calls. To identify the problem, you need to know not only where the leaked memory is allocated but also the conditions under which the leak occurs. The piece of information that allows you to do this is the memory allocation number. This is the number that appears in braces after the filename and line number when those are displayed. For example, in the following output, the memory allocation number is 18. It means that the leaked memory is the 18th block of memory allocated in your program.

 

Copy

Detected memory leaks!

Dumping objects ->

C:/PROGRAM FILES/VISUAL STUDIO/MyProjects/leaktest/leaktest.cpp(20) : {18} 

normal block at 0x00780E80, 64 bytes long.

 Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD

Object dump complete.

The CRT library counts all memory blocks allocated during a run of the program, including memory allocated by the CRT library itself or by other libraries such as MFC. Therefore, an object with allocation number N will be the Nth object allocated in your program but may not be the Nth object allocated by your code. (In most cases, it will not be.)

 

You can use the allocation number to set a breakpoint at the location where memory is allocated. To do this, set a location breakpoint near the start of your program. When your program breaks at that point, you can set such a memory-allocation breakpoint from the QuickWatch dialog box or the Watch window.

 

Procedure

To set a memory-allocation breakpoint in the Watch window

 

In the Watch window, type the following expression in the Name column:

 

Copy

_crtBreakAlloc

If you are using the multithreaded DLL version of the CRT library (the /MD option), include the context operator, as shown here:

 

Copy

{,,msvcr71d.dll}_crtBreakAlloc

Press RETURN.

 

The debugger evaluates the call and places the result in the Value column. This value will be –1 if you have not set any breakpoints on memory allocations.

 

Replace the value in the Value column with the allocation number of the memory allocation where you want to break. For example, 18 to break at the allocation shown in the output above.

 

After you set breakpoints on the memory allocations you are interested in, you can continue debugging. Be careful to run the program under the same conditions as the previous run so that the allocation order does not change. When your program breaks at the specified memory allocation, you can look at the Call Stack window and other debugger information to determine the conditions under which the memory was allocated. If necessary, you can continue execution of the program from that point to see what happens to the object and perhaps determine why it is not properly deallocated.

 

Note

Setting a data breakpoint on the object may be helpful. For more information, see How to: Set a Data Breakpoint (Native Only).

Although it is usually easier to set memory-allocation breakpoints in the debugger, you can set them in your code, if you prefer.

 

To set a memory-allocation breakpoint in your code

 

Add a line like this (for the 18th memory allocation):

 

Copy

_crtBreakAlloc = 18;

Alternately, you can use the _CrtSetBreakAlloc function, which has the same effect:

 

Copy

_CrtSetBreakAlloc(18);

 

 

http://msdn.microsoft.com/en-us/library/5tz9b54s(v=VS.80).aspx

Memory State Comparison 

 

Another technique for locating memory leaks involves taking snapshots of the application's memory state at key points. The CRT library provides a structure type, _CrtMemState, which you can use to store a snapshot of the memory state:

 

Copy

_CrtMemState s1, s2, s3;

 

To take a snapshot of the memory state at a given point, pass a _CrtMemState structure to the _CrtMemCheckpoint function. This function fills in the structure with a snapshot of the current memory state:

 

Copy

_CrtMemCheckpoint( &s1 );

 

You can dump the contents of a _CrtMemState structure at any point by passing the structure to the _CrtMemDumpStatistics function:

 

Copy

_CrtMemDumpStatistics( &s1 );

 

This function prints a dump of memory allocation information that looks like this:

 

Copy

0 bytes in 0 Free Blocks.

0 bytes in 0 Normal Blocks.

3071 bytes in 16 CRT Blocks.

0 bytes in 0 Ignore Blocks.

0 bytes in 0 Client Blocks.

Largest number used: 3071 bytes.

Total allocations: 3764 bytes.

 

To determine whether a memory leak has occurred in a section of code, you can take snapshots of the memory state before and after the section, and then use _CrtMemDifference to compare the two states:

 

Copy

_CrtMemCheckpoint( &s1 );

// memory allocations take place here

_CrtMemCheckpoint( &s2 );

 

if ( _CrtMemDifference( &s3, &s1, &s2) )

   _CrtMemDumpStatistics( &s3 );

As the name implies, _CrtMemDifference compares two memory states (s1 and s2) and produces a result (s3) that is the difference of the two states. Placing _CrtMemCheckpoint calls at the beginning and end of your program and using _CrtMemDifference to compare the results provides another way to check for memory leaks. If a leak is detected, you can use _CrtMemCheckpoint calls to divide your program and locate the leak using binary search technique.

 

 

 

 

 

http://www.codeproject.com/KB/applications/visualleakdetector.aspx

 

Introduction

 

Visual C++ provides built-in memory leak detection, but its capabilities are minimal at best. This memory leak detector was created as a free alternative to the built-in memory leak detector provided with Visual C++. Here are some of Visual Leak Detector's features, none of which exist in the built-in detector:

 

Provides a complete stack trace for each leaked block, including source file and line number information when available.

Provides complete data dumps (in hex and ASCII) of leaked blocks.

Customizable level of detail in the memory leak report.

Other after-market leak detectors for Visual C++ are already available. But most of the really popular ones, like Purify and BoundsChecker, are very expensive. A few free alternatives exist, but they're often too intrusive, restrictive, or unreliable. Here are some key advantages that Visual Leak Detector has over many other free alternatives:

 

Visual Leak Detector is cleanly packaged as an easy-to-use library. You don't need to compile its source code to use it. And you only need to make minor additions to your own source code to integrate it with your program.

In addition to providing stack traces with source files, line numbers, and function names, Visual Leak Detector also provides data dumps.

It works with both C++ and C programs (compatible with both new/delete and malloc/free).

The full source code to the library is included and it is well documented, so it is easy to customize it to suit your needs.

Visual Leak Detector is licensed free of charge as a service to the Windows developer community.

 

Using Visual Leak Detector

 

This section briefly describes the basics of using Visual Leak Detector (VLD). For a more in-depth discussion of the configuration options, runtime APIs, and a discussion of the more advanced usage scenarios (such as using VLD with DLLs), please see the full documentation included in the downloadable Zip files.

 

To use VLD with your project, follow these simple steps:

 

Copy the VLD library (*.lib) files to your Visual C++ installation's "lib" subdirectory.

Copy the VLD header files (vld.h and vldapi.h) to your Visual C++ installation's "include" subdirectory.

In the source file containing your program's main entry point, include the vld.h header file. It's best, but not absolutely required, to include this header before any other header files, except for stdafx.h. If the source file, include stdafx.h, then vld.h should be included after it.

If you are running Windows 2000 or earlier, then you will need to copy dbghelp.dll to the directory where the executable being debugged resides.

Build the debug version of your project.

VLD will detect memory leaks in your program whenever you run the debug version under the Visual C++ debugger. A report of all the memory leaks detected will be displayed in the debugger's output window when your program exits. Double-clicking on a source file's line number in the memory leak report will take you to that file and line in the editor window, allowing easy navigation of the code path leading up to the allocation that resulted in a memory leak.

 

Making a Memory Leak Detector

 

The goal of Visual Leak Detector was to build a better replacement for the memory leak detector built-in to Visual C++. With that in mind, I set out to use the same method used by the built-in detector, namely the CRT Debug Heap. But this new detector would provide enhancements -- primarily full stack traces, which can be extremely helpful for finding and fixing leaks.

 

The Built-In Detector

 

The built-in detector is pretty simple really. When a program is exiting, the CRT runs a bunch of cleanup code after main returns. If the built-in detector is enabled, then it runs a memory leak check as part of the cleanup procedure. The memory leak check simply looks at the debug heap: if there are any user blocks still allocated on the debug heap, then they must be memory leaks. The debug version of malloc stores the file and line number that allocated each block in the block's header at the time it is allocated. When the built-in detector identifies a memory leak, it simply peers inside the block header to get the file and line number. It then reports that information to the debugger where it is displayed.

 

Note that the built-in detector detects leaks without doing any monitoring of allocations or frees. It simply takes a snapshot of the heap just before the process terminates and determines if there are any leaks based on that snapshot. A snapshot of the heap only tells us if there are leaks; it does not tell us how they were leaked. Clearly, to determine the "how" we also need to obtain a stack trace. But to obtain a stack trace, we need to be able to monitor every allocation on-the-fly at runtime. This is what will distinguish our leak detector from the built-in one.

 

Allocation Hooking

 

Luckily for us, Microsoft has provided an easy way to monitor every allocation made from the debug heap: allocation hooks. An allocation hook is simply a user-supplied callback function that will be called just before each allocation is made from the debug heap. Microsoft has provided a function, _CrtSetAllocHook, which registers the allocation hook function with the debug heap. When the debug heap calls the allocation hook, one of the arguments passed is an ID number that uniquely identifies each allocation -- it's basically a serial number for each memory block allocated. There's not enough room in the memory block header for us to record any information directly in it, but we can use this unique ID number as a key to map each block to any data that we want to record.

 

Walking the Stack

 

Now that we have a way to be notified every time a block is allocated, as well as a way to uniquely identify each allocation, all that's left to do is to record the call stack each time an allocation occurs. We could conceivably attempt to unwind the stack ourselves using inline assembly. But stack frames can be organized in different ways, depending on compiler optimizations and calling conventions, so it could become complicated to do it that way. Once again, Microsoft has provided us with a tool to help us out. This time it is a function that we can call iteratively to walk the stack, frame by frame. That function is StackWalk64. It is part of the Debug Help Library (dbghelp.dll). As long as we provide it with the information that it needs to establish a starting "frame of reference", so to speak, it can examine our stack from there and reliably unwind it for us. Each time StackWalk64 is called, it gives back a STACKFRAME64 structure that can be reused as input for the next call to StackWalk64. It can be repeatedly called this way until the end of the stack is reached.

 

Initializing the Memory Leak Detector

 

We now have the beginnings of a better memory leak detector. We can monitor every allocation, and for each allocation monitored, we can obtain and record a stack trace. The only challenge that remains is to ensure that the allocation hook function is registered with the debug heap as soon as the program starts executing. This can be very simply solved by creating a global instance of a C++ class object. The constructor will run when the program is initialized. From the constructor, we can call _CrtSetAllocHook to register our allocation hook function. But wait, what if the program we are debugging already has other global C++ class objects that allocate memory? How can we ensure that our constructor will be called first, and that our allocation hook function will be installed before any other global objects are constructed? Unfortunately, the C++ specification does not spell out any rules for deciding in which order to construct global objects. So there are no absolute guarantees that our constructor will be called first. But we can come very close to guaranteeing it. We can leverage a compiler-specific preprocessor directive that explicitly tells the compiler to ensure that our global variable is constructed as soon as possible: #pragma init_seg (compiler). This directive tells the compiler to place our global object in the "compiler" initialization segment. Objects in this segment are the first to be constructed. Next, objects in the "library" segment are constructed, and objects in the "user" segment are constructed last. The "user" segment is the default segment for global objects. Generally speaking, no normal user objects should ever be placed in the "compiler" segment, so this provides a reasonable amount of certainty that our global object will be constructed before any user objects.

 

Detecting Memory Leaks

 

Because global objects are destroyed in the inverse order they are constructed, our global object will be destroyed after any user objects. We can then examine the heap, just like the built-in detector does. If we find a block on the heap that has not been freed, it is a leak and we can look up its call stack using the unique ID number recorded by our allocation hook function. An STL map would work fine here for mapping the ID number to the call stacks. I didn't use an STL map because I wanted my library to be compatible with both new and old versions of Visual C++. The STL from older versions is incompatible with the newer versions, so I couldn't use STL components. But the good news is that this gave me the opportunity to create a data structure similar in concept to the STL map, but with specific optimizations for use with my memory leak detector.

 

Do you remember that the built-in leak detector peers inside the memory block to get the name of the file and the line number where the block was allocated? Well, all we have for our call stack is a bunch of program addresses. Dumping all those hex numbers to the debugger wouldn't be of much use. To make those addresses more meaningful, we need to translate them to human readable information: files and line numbers (and function names too). Once again, Microsoft comes through with the tools that will help us do our job: the symbol handler APIs. Like StackWalk64, they also happen to be part of the Debug Help Library. I won't dwell on them in detail here, because there are a lot of them and they're pretty simple to use. They don't require as much ingenuity to use as StackWalk64 does. We can use two of the symbol handler APIs to get the filenames, line numbers, and function names that we want. The aptly named SymGetLineFromAddr64 translates addresses into source filenames and line numbers. Its sister API, SymFromAddr translates addresses into symbol names. For program addresses, which are what we have, the corresponding symbol name will be the name of the function containing that program address.

 

Key Parts of the Source Code

 

In case you got bored with the above section and skipped ahead, I'll summarize it here. In a nutshell, this memory leak detector works like this:

 

A global object is automatically constructed. It is the first object constructed. The constructor registers our allocation hook function.

Every allocation eventually calls our allocation hook function. The allocation hook function obtains and records the call stack for each allocation. The call stack information is recorded in a specialized STL-like map.

When the program terminates, the global object is the last object destroyed. It examines the heap and identifies leaks. Leaked blocks are looked up in the map and matched with their corresponding call stack. The resulting data is sent to the debugger to be displayed.

Step 1: Registering the Allocation Hook

 

Here is the VisualLeakDetector class constructor. Note the call to _CrtSetAllocHook. This is where our callback function, allochook, is registered with the debug heap. The call to linkdebughelplibrary performs an explicit dynamic link with the Debug Help Library (dbghelp.dll). Because VLD is itself a library, implicitly linking with the Debug Help Library through the import library dbghelp.lib is undesirable; it would make the VLD library dependent on dbghelp.lib at link-time. dbghelp.lib will not be present on many Windows computers and it's not redistributable, so we need to link with the DLL at runtime in order to bypass the import library. There is a lot of other stuff going on in here as well, but most of it has to do with custom configuration options that VLD supports.

 

 Collapse

// Constructor - Dynamically links with the Debug Help Library and installs the

 

//   allocation hook function so that the C runtime's debug heap manager will

 

//   call the hook function for every heap request.

 

//

 

VisualLeakDetector::VisualLeakDetector ()

{

    // Initialize private data.

 

    m_mallocmap    = new BlockMap;

    m_process      = GetCurrentProcess();

    m_selftestfile = __FILE__;

    m_status       = 0x0;

    m_thread       = GetCurrentThread();

    m_tlsindex     = TlsAlloc();

 

    if (_VLD_configflags & VLD_CONFIG_SELF_TEST) {

        // Self-test mode has been enabled.

 

        // Intentionally leak a small amount of

 

        // memory so that memory leak self-checking can be verified.

 

        strncpy(new char [21], "Memory Leak Self-Test", 21);

        m_selftestline = __LINE__;

    }

 

    if (m_tlsindex == TLS_OUT_OF_INDEXES) {

        report("ERROR: Visual Leak Detector:" 

               " Couldn't allocate thread local storage./n");

    }

    else if (linkdebughelplibrary()) {

        // Register our allocation hook function with the debug heap.

 

        m_poldhook = _CrtSetAllocHook(allochook);

        report("Visual Leak Detector " 

               "Version "VLD_VERSION" installed ("VLD_LIBTYPE")./n");

        reportconfig();

        if (_VLD_configflags & VLD_CONFIG_START_DISABLED) {

            // Memory leak detection will initially be disabled.

 

            m_status |= VLD_STATUS_NEVER_ENABLED;

        }

 

        m_status |= VLD_STATUS_INSTALLED;

        return;

    }

 

    report("Visual Leak Detector is NOT installed!/n");

}

Step 2: Walking the Stack

 

Here is the function responsible for obtaining call stacks. This is perhaps the trickiest part of the entire program. Setting up for the first call to StackWalk64 is where the tricky bit is. To start the stack trace, StackWalk64 needs to know exactly where on the stack to begin walking. It never assumes that we want to start tracing from the current stack frame. This requires that we provide it with the address of the current frame, as well as the current program address. I've seen other examples that attempt to get this information by calling GetThreadContext to retrieve the current thread's context, which would contain both of the required addresses. But, as its documentation clearly states, GetThreadContext can't be relied upon to get valid information for a running thread. By definition, this means that GetThreadContext can't get a valid context for the current thread. A better approach is to get the required addresses directly, and the only way to do that is with inline assembly.

 

Obtaining the address of the current frame is easy: it's stored in a CPU register (EBP) that we can directly read it from. The program address is a little harder to obtain. Though there is a CPU register (EIP) that always contains the current program address, on Intel x86 CPUs, it can't be read by software. But we can get the same address in a round-about way, by calling another function and from within that function obtaining the return address. The return address is the same as the program address that called the function. For this, I've created a separate function, getprogramcounterx86x64. Since we're already doing inline assembly, we could write a simple function call in assembly, instead of writing another C++ function, but to keep it easier to understand, I've used C++ wherever it's possible to do so.

 

In the following code, pStackWalk64, pSymFunctionTableAccess64 and pSymGetModuleBase64 are all pointers to the functions exported by dbghelp.dll.

 

 Collapse

// getstacktrace - Traces the stack, starting from this function, as far

 

//   back as possible.

 

//

 

//  - callstack (OUT): Pointer to an empty CallStack to be populated with

 

//    entries from the stack trace.

 

//

 

//  Return Value:

 

//

 

//    None.

 

//

 

void VisualLeakDetector::getstacktrace (CallStack *callstack)

{

    DWORD        architecture;

    CONTEXT      context;

    unsigned int count = 0;

    STACKFRAME64 frame;

    DWORD_PTR    framepointer;

    DWORD_PTR    programcounter;

 

    // Get the required values for initialization of the STACKFRAME64 structure

 

    // to be passed to StackWalk64(). Required fields are AddrPC and AddrFrame.

 

#if defined(_M_IX86) || defined(_M_X64)

    architecture = X86X64ARCHITECTURE;

    programcounter = getprogramcounterx86x64();

    __asm mov [framepointer], BPREG // Get the frame pointer (aka base pointer)

 

#else

// If you want to retarget Visual Leak Detector to another processor

 

// architecture then you'll need to provide architecture-specific code to

 

// retrieve the current frame pointer and program counter in order to initialize

 

// the STACKFRAME64 structure below.

 

#error "Visual Leak Detector is not supported on this architecture."

#endif // defined(_M_IX86) || defined(_M_X64)

 

 

    // Initialize the STACKFRAME64 structure.

 

    memset(&frame, 0x0, sizeof(frame));

    frame.AddrPC.Offset    = programcounter;

    frame.AddrPC.Mode      = AddrModeFlat;

    frame.AddrFrame.Offset = framepointer;

    frame.AddrFrame.Mode   = AddrModeFlat;

 

    // Walk the stack.

 

    while (count < _VLD_maxtraceframes) {

        count++;

        if (!pStackWalk64(architecture, m_process, m_thread, 

             &frame, &context, NULL, pSymFunctionTableAccess64, 

             pSymGetModuleBase64, NULL)) {

            // Couldn't trace back through any more frames.

 

            break;

        }

        if (frame.AddrFrame.Offset == 0) {

            // End of stack.

 

            break;

        }

 

        // Push this frame's program counter onto the provided CallStack.

 

        callstack->push_back((DWORD_PTR)frame.AddrPC.Offset);

    }

}

And here is the function that retrieves the EIP register. Again, this has to be done as a separate function call because there is no way for the software to directly read the EIP register. But the same value can be obtained by making a function call, and then from within the called function getting the return address. The return address is the program address that made the function call, and it is pushed onto the stack when the function call is made. We get it by copying it from the stack.

 

 Collapse

// getprogramcounterx86x64 - Helper function that retrieves the program counter

 

//   for getstacktrace() on Intel x86 or x64 architectures.

 

//

 

//  Note: Inlining of this function must be disabled. The whole purpose of this

 

//    function's existence depends upon it being a *called* function.

 

//

 

//  Return Value:

 

//

 

//    Returns the caller's program address.

 

//

 

#if defined(_M_IX86) || defined(_M_X64)

#pragma auto_inline(off)

DWORD_PTR VisualLeakDetector::getprogramcounterx86x64 ()

{

    DWORD_PTR programcounter;

 

    // Get the return address out of the current stack frame

 

    __asm mov AXREG, 

    // Put the return address into the variable we'll return

 

    __asm mov [programcounter], AXREG

 

    return programcounter;

}

#pragma auto_inline(on)

#endif // defined(_M_IX86) || defined(_M_X64)

Step 3: Generating a Better Memory Leak Report

 

Finally, here is the function that converts the program addresses obtained while walking the stack into useful symbol names. Note that the address-to-symbol conversion code is only run if memory leaks are detected. This avoids having to do symbol lookups on-the-fly while the program is running, which would add considerable additional overhead. Not to mention that it just doesn't make sense to store (large) symbol names for later retrieval when you can store (small) addresses instead.

 

The CRT doesn't expose any documented method for gaining access to its internal linked-list of allocated memory blocks. This linked list is what is used by the built-in detector for taking a "snapshot" of the heap to determine if there are any memory leaks. I've come up with a very simple trick to gain access to the list. Any time a new memory block is allocated, it happens to be placed at the beginning of the linked-list. So, to get a pointer to the head of the list, I just allocate a temporary memory block. That block's address can be converted to the address of the containing _CrtMemBlockHeader structure and now I have a pointer to the beginning of the linked list.

 

In the following code, pSymSetOptions, pSymInitialize, pSymGetLineFromAddr64, and pSymFromAddr are all pointers to the functions exported by dbghelp.dll. The report function is just a custom wrapper around OutputDebugString which sends messages to the debugger for display in the debugger's output window.

 

This function is quite long. To cut down on clutter, I've removed all of the uninteresting and trivial parts. To see this function in its entirety, please download the source ZIP file.

 

 Collapse

// reportleaks - Generates a memory leak report when the program terminates if

 

//   leaks were detected. The report is displayed in the debug output window.

 

//

 

//  Return Value:

 

//

 

//    None.

 

//

 

void VisualLeakDetector::reportleaks ()

{

 

    ...

 

    // Initialize the symbol handler. We use it for obtaining source file/line

 

    // number information and function names for the memory leak report.

 

    symbolpath = buildsymbolsearchpath();

    pSymSetOptions(SYMOPT_LOAD_LINES | SYMOPT_DEFERRED_LOADS | SYMOPT_UNDNAME);

    if (!pSymInitialize(m_process, symbolpath, TRUE)) {

        report("WARNING: Visual Leak Detector: The symbol handler" 

               " failed to initialize (error=%lu)./n"

               "    Stack traces will probably not be available" 

               " for leaked blocks./n", GetLastError());

    }

 

    ...

 

#ifdef _MT

    _mlock(_HEAP_LOCK);

#endif // _MT

 

    pheap = new char;

    pheader = pHdr(pheap)->pBlockHeaderNext;

    delete pheap;

    while (pheader) {

 

        ...

 

        callstack = m_mallocmap->find(pheader->lRequest);

        if (callstack) {

 

            ...

 

            // Iterate through each frame in the call stack.

 

            for (frame = 0; frame < callstack->size(); frame++) {

                // Try to get the source file and line number associated with

 

                // this program counter address.

 

                if (pSymGetLineFromAddr64(m_process, 

                   (*callstack)[frame], &displacement, &sourceinfo)) {

 

                    ...

 

                }

 

                // Try to get the name of the function containing this program

 

                // counter address.

 

                if (pSymFromAddr(m_process, (*callstack)[frame], 

                    &displacement64, pfunctioninfo)) {

                    functionname = pfunctioninfo->Name;

                }

                else {

                    functionname = "(Function name unavailable)";

                }

 

                ...

 

            }

 

            ...

 

        }

        pheader = pheader->pBlockHeaderNext;

    }

#ifdef _MT

    _munlock(_HEAP_LOCK);

#endif // _MT

 

 

    ...

 

}