内存泄漏及检测（英文）

Copyright by 云破日出

 

 

1.     Introduction

1.1       What we are dealing with

Before finding memory leak, we should have an overview of what we are dealing with. Memory leak on Windows platform contains following aspects:

§  Memory leak on CRT, Global, Heap, Virtual, COM, …

§  Handles leak when using GDI, Kernel, Common control, winsock, winhttp, advapi, …

Many memory leak detection tools also detect errors when using memory, which include memory conflict, memory overrun, etc.

 

1.2       Leak detection tools

Figure 1  Leak detection tools

 

1.3       Leak category

Figure 2  Leak category

 

1.4       Work flow

Although we can use different tools to detect memory leak, but there are some common steps to do.

       

Figure 3  Leak detection work flow

 

2.     Tools

2.1       CRT Heap Check Routine

Advantage:

Simple

Speedy

Low cost

Disadvantage:

Can only detect memory leaks allocated by new and malloc

Can only run under Visual Studio IDE

No call stack information, hard to find the leak especially when multi-allocation in same place

Detail:

                Write the following code in the firstly included .h file

                #define _CRTDBG_MAP_ALLOC

                #include

                #include

                Write the following code at last to print the result to Debug window

_CrtDumpMemoryLeaks();

Demo:

Figure 4  CRT Heap Check Routing

Notice:

                If _CRTDBG_MAP_ALLOC is not defined before including crtdbg.h, the output imformation will not contain the leak file.

Analyse:

                The output string in Debug window between Dumping objects and Object dump complete has the following meanings (marks are in the window above):

(1)     Leak file and line number

(2)     Memory allocation sequence

(3)     Type of leak block

a.       normal block       allocated by your code

b.       client block                          for MFC destructor

c.       CRT block                            CRT allocated

d.       free block                            freed block (will not appear in the output of the leak dump)

e.       ignore block        programmer can claim blocks ignored by CRT check routine

(4)     The leak address

(5)     ASCII and Binary version of leak content

 

 

2.2       Visual Leak Detector

This is a wrap for CRT heap check routines.

Advantage:

                Same with CRT check routines, but provides call stack information

Disadvantage:

                Same with CRT check routines, but provides call stack information

Detail:

§  Download the zip from CodeProject

§  Copy the VLD library (*.lib) files to your Visual C++ installation's "lib" subdirectory.

§  Copy the VLD header files (vld.h and vldapi.h) to your Visual C++ installation's "include" subdirectory.

§  In the source file containing your program's main entry point, include the vld.h header file. It's best, but not absolutely required, to include this header before any other header files, except for stdafx.h. If the source file, include stdafx.h, then vld.h should be included after it.

§  If you are running Windows 2000 or earlier, then you will need to copy dbghelp.dll to the directory where the executable being debugged resides.

§  Build the debug version of your project.

Demo:

Figure 5  Visual Leak Detector

Notice:

                The library works only when defines _DEBUG, so the release version will not be effected even if including

Analyse:

                The stack trace information and entire leaked data content was print to the output window in VC IDE.

 

2.3       Memory Validator

Memory Validator is a speedy and powerful tool for memory error detecting. It is able to monitor the following kinds of resources :

Figure 6  Memory Validator – Setting Dialog 1

Advantage

                Powerful and handy

Disadvantage

                Sometimes not accurate

Demo

§  Settings

o    Set to monitor one specific resource one time on large scale software. See the collect setting above, and the display setting below.

Figure 7  Memory Validator - Setting Dialog 2

o    Set to monitor only some module of the large scale software

Figure 8  Memory Validator - Setting Dialog 3

§  Watermark

o        Check the leak during the two specific scenarios. We can take a snapshot of memory usage before doing suspicious operation and then take another snapshot when finishing the operation. Check the Compared result data provided by the tool.

Figure 9  Memory Validator – Adding watermark

Figure 10  Memory Validator - All the allocation between two watermarks

§  Objects : Tells us which kind and how many objects existed in memory or leaked after exiting the program. Sort by Number is recommended.

Figure 11  Memory Validator - Objects

§  Filters : Use filters to select the data you are interested in.

Figure 12  Memory Validator - Filters

§  For icons and other view usage, please check the help manual of Memory Validator

 

2.4       BoundsChecker

BoundsChecker is a memory detect tool developed by Compuware. Like memory validator, it uses hook to detect memory leaks. They are almost the same.

Here is some pictures taken from BoundsChecker

Figure 13  Setting Dialog

Figure 14  Boundschecker - Summary Dialog

Figure 15  Leak Detail

 

2.5       UMDH & LeakExplorer

UMDH is a tool provide by Microsoft. Through comparing two memory dump, UMDH tells us how much memory is allocated between the dump. Since not all the memory allocation is leak, so we should do the same operation between the two dump, and get the third dump. If memory increase steadily in the same context, it is quite suspicious. Since there are so much allocations information from kernel, and the output of UMDH is txt file, we have great difficulty in finding leaks. Autocad team developed a tool to analyze the output txt files by UMDH, this is called LeakExplorer.

 

Advantage

                Accurate

                Speedy

                Not using hook, no impact on the program to be tested

Disadvantage

                Not handy, Cost a lot of time to find the leak

Demo

                UMDH.wmv

 

 

 

 

Appendix: Background

1.1       Layered Memory Management in Win32

 

Figure 16  Memory Layer in Windows NT based system

Local, Global memory API are provided by previously released windows api. There used to be a global heap for all the applications, which shares it concurrently, and multiple, private local heaps, one for each application. After win 2000, there is no difference between local heap and global heap. Actually there is no global heap now.

 

1.2       Memory allocation pair

 

	Allocator	Deallocator
CRT	new	delete
	malloc	Free
Virtual Memory	VirtualAlloc	VirtualFree
Heap	HeapAlloc	HeapFree
Global	GlobalAlloc	GlobalFree
Local	LocalAlloc	LocalFree
Com	CoTaskMemAlloc	CoTaskMemFree
	SysAllocString*	SysFreeString
	SafeArrayCreate	SafeArrayDestroy

Figure 17  Memory allocation pair

Most memory leak happens because of misusing the pair of allocator and free functions. For example, use free() to new(), or use delete() to malloc(), or forget to use free function after allocating memory. But there is an exception, that is Global heap and local heap memory api, as described above.

 

There two kinds of heaps for each application now. One is called default heap, the CRT memory API and Local, Global API allocate memory on it by default. Besides, we can also use HeapCreate to generate a dynamic heap. By using HeapAlloc and HeapFree, we can allocate and free memory on the dynamic heap. The most important difference between these two kinds of heaps is that the default heap is thread-safe. That is to say, we have to do synchronizations for the heap created by HeapCreate.

 

Since we are familiar with the CRT memory management, I will skip that. You can get the leak scenarios for CRT leaks in appendix. Those are tutorials from memory validator, which will be introduced in chapter 2.

 

1.3       Memory management in COM

 

1.3.1          COM pointer

 

All COM components use reference counting technology, they all have AddRef() to increase the count number, and Release() to decrease the count number. If the count number falls to 0, Release() will destroy the COM point and free its memory. So we should use a COM pointer like this.

pCOMNew = pCOM;

pCOM->AddRef();

…

If ( NULL != pCOM)

{

    pCOM->Release();

    pCOM = NULL;

}

If ( NULL != pCOMNEW)

{

    pCOMNEW->Release()

    pCOMNEW = NULL;

}

 

                In most cases we don’t need to write such low level code, for example, we can use macro to replace code of xxx->Release(); xxx = NULL (like SAFEDELETE in directx). Or, we can use a simple smart pointer like this:

 

template

class CSmartPtr

{

    P* p;

public:

    CSmartPtr(P* ptr) { p = ptr; if(p) p->AddRef(); }

    ~CSmartPtr() { if(p) p->Release(); }

};

 

                And for ATL : CComPtr and CComQIPtr, which are also smart pointer. Just remember when these COM objects goes out of their scope, they will destroy automatically, we needn’t destroy them again. But a lot of memory detection tools report these are memory leaks.

 

1.3.2          BSTR

 

The string operation leaks memory easily, not only because the string often used as a parameter passed everywhere, which would be easily forgotten to free, but also because not understand the library we use.

BSTR is null-terminated, two-bytes string, so its null-termination also has 2 bytes. The name of BSTR means “Basic STRing”, it can be used in Visual Basic and java.lang.string, and any other language support Automation. We usually use CComBSTR in ATL to manipulate the string. I mention it here just because it use SysAllocString and SysFreeString to allocate and free the memory, not malloc and free in CRT. So when there is a memory leak, we can’t expect the CRT memory check routine to find it.

For more information about CComBSTR, you can see Chapter 5, Data Conversion Classes in .

 

1.3.3          SafeArray

 

Like BSTR, SafeArray can also be recognized in VB and Java. We use SafeArrayCreate to allocate memory, and use SafeArrayDestroy to free the memory. In ATL, there is an alternative way: CComSafeArray. It’s a template class of SafeArray. They need special attention also because SafeArrayCreate and SafeArrayDestory are not CRT functions. Some memory detection tool will miss the real leaks, and some tools will report a correct scenario as memory leak.

 

1.3.4          CoTaskMemAlloc and CoTaskMemFree

 

As mentioned in 1.3.2, there are two kinds of heaps. The default heap meets our needs in most cases, but for COM, we usually allocate memory in one process, and free memory in another process. Clearly, a conventional heap has no way to handle this, so CoTaskMemAlloc and CoTaskMemFree is used instead. The automatically generated proxy-stub code can properly allocate and free memory across COM boundaries. COM's remoting infrastructure does all the dirty work needed to create the illusion of a single heap that spans processes.

In ATL, there is an interface called IAtlMemMgr, which has the following members:

__interface IAtlMemMgr {                       

public:                                         

    void* Allocate( size_t nBytes ) ;          

    void Free( void* p ) ;                     

    void* Reallocate( void* p, size_t nBytes ) ;

    size_t GetSize( void* p ) ;                

}; 

We can implement it for special usage. ATL also has 5 concrete class implemented IAtlMemMgr, they can be used under different situation, as described in the following table, the left column is classes in ATL, the right column is the low level functions each class used.

 

Memory Manager Class	Heap Functions Used
CComHeap	CoTaskMemAlloc, CoTaskMemFree, CoTaskMemRealloc, IMalloc::GetSize
CCRTHeap	malloc, free, realloc
CLocalHeap	LocalAlloc, LocalFree, LocalReAlloc, LocalSize
CGlobalHeap	GlobalAlloc, GlobalFree, GlobalReAlloc, GlobalSize
CWin32Heap	HeapAlloc, HeapFree, HeapReAlloc, HeapSize

 

1.3.5          COM objects’ Reference count

If we found a lot leak within one class, we can infer that one instance of that class wasn’t released, and its destructor wasn’t invoked at all. Usually this can be caused by three scenarios:

§  Object pointer overwriting

§  Forgetting invoke delete operator on the pointer passed by another function

§  Incorrect com object reference count

The first two scenario is easy to detect by using memory check tools, but the third one is hard to locate where has gone wrong. Let’s see why this is difficult. The reference count itself is simple, we only should make sure that AddRef() and Release() matches on a non-smart com pointer, just like new() and delete().

There are 3 kinds of smart pointer in COM, which manipulate reference count automatically.

§  Smart pointer provided by ATL template ( CComPtr and CComQIPtr )

§  Smart pointer provided by compiler ( _com_ptr_t )

§  Smart pointer defined by ourselves

Let’s pick smart pointer from ATL template, other smart pointers are the same. CComPtr and CComQIPtr are subclass of CComPtrBase, its release method is defined like this:

      // Release the interface and set to NULL

      void Release() throw()

      {

            T* pTemp = p;

            if (pTemp)

            {

                  p = NULL;

                  pTemp->Release();

            }

   }

Usually we needn’t call release on a smart pointer, because it can be released automatically. But if we do call release on a class member, let’s see what happens. After calling release, the class member is assigned to be NULL. But actually its reference count may still greater than 0. When the object goes out of scope, the auto-release action will not perform, since the pointer is NULL. Finally the COM object will have no change to release because its reference count is greater than 0.

We also should pay great attention on using non-smart pointer and smart point together. “=” operator is only overloaded for assignments between two smart pointers. When the left side of “=” is just an interface, AddRef() will not be called. We should call AddRef() and Release() manually in this case.

 

1.4       Handle Management

Like memory allocation pairs, handles also has their creator and destroy function pair. We should use them together and make them match.

Appendix: CRT Leak scenario

n  incorrect usage: double delete

    1284 :     char    *ptr;

    1285 :

    1286 :     ptr = new char [10];

    1287 :     delete [] ptr;              // correct cleanup

    1288 :     delete [] ptr;              // incorrect doublecleanup

    1289 : }

 

n  incorrect usage: double free

    1082 :

   1083 :     char    *ptr;

    1084 :

    1085 :     ptr = (char *)malloc(10);

    1086 :     free(ptr);                  // correct cleanup

    1087 :     free(ptr);                  // incorrect doublecleanup

    1088 : }

 

n  free the wrong location

    1107 :     // allocate some memory then free the wrong location, inside the block

    1108 :

    1109 :     ptr = (char *)malloc(3);

    1110 :     for(i = 1; i < 3; i++)

    1111 :         free(ptr + i);          // incorrect cleanup

    1112 :

    1113 :     // correct cleanup

    1114 :

    1115 :     free(ptr);                  // correct

 

n  delete the wrong location

    1314 :     // allocate some memory then free the wrong location, inside the block

    1315 :

    1316 :     ptr = new char [3];

    1317 :     for(i = 1; i < 3; i++)

    1318 :     {

    1319 :         delete (ptr + i);       // incorrect cleanup

    1320 :         delete [] (ptr + i);    // incorrect cleanup

    1321 :     }

 

n  realloc the wrong location

    1367 :     ptr = (char *)malloc(10);

    1368 :     p = (char *)realloc(ptr + 1, 20);       // incorrect realloc

    1369 :     p = (char *)realloc(ptr - 1, 20);       // incorrect realloc

    1370 :     ptr = (char *)realloc(ptr, 20);         // correct realloc

 

n  delete malloc/calloc/relloc’d memory

    4708 :     char    *ptr;

    4709 :

    4710 :     ptr = (char *)malloc(1000);

    4711 :     delete ptr;             // incorrect (it will work with Visual Studio 6.0 debug, but thats relying on a side-effect caused by new wrapping free, if they change the implementation, bang codes the code!)

    4712 :     delete [] ptr;          // incorrect (it will work with Visual Studio 6.0 debug, but thats relying on a side-effect caused by new wrapping free, if they change the implementation, bang codes the code!)

    4713 :     free(ptr);              // correct

 

n  free/relloc new’d memory

    4764 :     char    *ptr;

    4765 :

    4766 :     ptr = new char [1000];

    4767 :     free(ptr);              // incorrect (it will work with Visual Studio 6.0 debug, but thats relying on a side-effect caused by new wrapping free, if they change the implementation, bang codes the code!)

    4768 :     delete [] ptr;          // correct

 

n  HeapReAlloc and HeapFree

 

n  Memory overwritten

    1502 :         char    *p;

    1503 :

    1504 :         p = new char [tmd.sizeToAllocate];

    1505 :         p[tmd.sizeToAllocate] = '/0';

 

n  Memory underwritten

    1479 :         char    *p;

    1480 :

    1481 :         p = new char [tmd.sizeToAllocate];

    1482 :         *(p - 1) = '/0';

 

n  Uninitialized on stack/heap

 

n  Heap memory overrun

 

    1923 :     // warning, the very nature of this overrun may cause this program

    1924 :     // to crash since it damages the CRT heap

    1925 :

    1926 :     char    *memory;

    1927 :

    1928 :     memory = new char[100];

    1929 :     memcpy(memory + 3, memory, 100);

 

n  Heap memory underrun

    1952 :     // warning, the very nature of this overrun may cause this program

    1953 :     // to crash since it damages the CRT heap

    1954 :

    1955 :     char    *memory;

    1956 :

    1957 :     memory = new char[100];

    1958 :     memcpy(memory - 3, memory, 100);

    1959 :

    1960 :     // I'll allow this to leak, since if I try to free it the damaged stack will

    1961 :     // be found, and if I don't try to free it, the program may not crash (for a while)

    1962 : }

 

n  Stack memory overrun

 

n  Stack memory underrun

 

n  Strcpy and wcscpy

    5081 :     char    *charBuffer = new char[20];

    5082 :     wchar_t *wcharBuffer = new wchar_t[20];  

    5085 :     char    *charBufferSource = new char[20];

    5086 :     wchar_t *wcharBufferSource = new wchar_t[20];

    5090 :     charBufferSource[0] = '/0';

    5091 :     wcharBufferSource[0] = _T('/0');

    5092:      strcpy(charBufferSource, "some test text56789");

    5093:      wcscpy(wcharBufferSource, L"some test text56789");

    5112:      // overwrite beginning of buffer

    5113:      strcpy(&charBuffer[-4], charBufferSource);

    5114:      wcscpy(&wcharBuffer[-4], wcharBufferSource);

    5115:      strncpy(&charBuffer[-4], charBufferSource, 20);

    5116:      wcsncpy(&wcharBuffer[-4], wcharBufferSource, 20);

    5117:     

    5118:      // overwrite end of buffer

    5119:      strcpy(&charBuffer[10], charBufferSource);

    5120:      wcscpy(&wcharBuffer[10], wcharBufferSource);

    5121:      strncpy(&charBuffer[10], charBufferSource, 20);

    5122:      wcsncpy(&wcharBuffer[10], wcharBufferSource, 20);

    …

    5146 :     // at this point, deleting the data may cause the program to crash (as the heap has

    5147 :     // been corrupted), if it doesn't the program will most likely crash later on

    5148 :

    5149 :     delete [] charBuffer;

    5150 :     delete [] wcharBuffer;

 

n  Forget deleting the object in an array

    1863 :     for(i = 0; i < 10; i++)

    1864 :     {

    1865 :         char    text[10];

    1866 :

    1867 :         sprintf(text, "%d", i);

    1868 :         array[i] = new CString(text);

    1869 :     }

    1870 :

    1871 :     // cleanup the array, but leak the contents

    1872 :

    1873 :     delete [] array;

 

n  Call functions on a deleted object