内存检测工具:sanitizer

1. 背景

内存泄漏是一个比较常见的问题,之前使用的是valgrind来实现内存检查的情况比较多,这里介绍一种更加便利的内存检测工具, 那就是gcc自带的sanitizer。

2. sanitizer 的用法

2.1 sanitizer的基本简介

Sanitizers 是谷歌发起的开源工具集,包括AddressSanitizer,MemorySanitizer, ThreadSanitizer, LeakSanitizer, Sanitizers项目本身是llvm项目的一部分,
gcc自带的工具, gcc从4.8版本开始支持Address和Thread Sanitizer,4.9版本开始支持Leak Sanitizer和UBSanitizer。

可以支持的内存检测:

  • Use after free
  • Heap buffer overflow
  • Stack buffer overflow
  • Global buffer overflow
  • Use after return
  • Use after scope
  • Initialization order bugs
  • Memory leaks

具体错误类型解释:

image

2.2 升级高版本的gcc和安装相关的依赖库(centos 7)

yum -y install centos-release-scl
yum -y install devtoolset-7-gcc devtoolset-7-gcc-c++ devtoolset-7-binutils
yum -y install devtoolset-7-libasan-devel.x86_64 devtoolset-7-liblsan-devel.x86_64 devtoolset-7-libtsan-devel.x86_64 devtoolset-7-libubsan-devel.x86_64
scl enable devtoolset-7 bash
echo "source /opt/rh/devtoolset-7/enable" >>/etc/profile

这里需要特别注意的是:Address Sanitizer 会替换malloc和free, 如果采用第三方的内存申请库,则无法替换,会造成功能缺失。

可以检查的内存问题包括:

1. Out-of-bounds accesses to heap, stack and globals
2. Use-after-free
3. Use-after-return (runtime flag)
4. ASAN_OPTIONS=detect_stack_use_after_return=1)
5. Use-after-scope (clang flag -fsanitize-address-use-after-scope)
6. Double-free, invalid free
7. Memory leaks (experimental)

2.3 实践测试

2.3.1 stack overflow

其中CMakeLists.txt如下:

cmake_minimum_required (VERSION 2.8)                                                                 
project (sanitizer)                                                                                  
set(CMAKE_CXX_FLAGS "-g -fsanitize=leak -fsanitize=address -fno-omit-frame-pointer")                    
add_executable(sanitizer_stack_overflow src/sanitizer_stack_overflow.cpp) 

-fsanitize=address 使能Address Sanitizer工具

-fsanitize=leak 只使能Leak Sanitizer,检测内存泄漏问题

-fno-omit-frame-pointer 检测到内存错误时打印函数调用栈

-O1 代码优化选项,可以打印更清晰的函数调用栈

其中src/sanitizer_stack_overflow.cpp如下:

#include 
#include 
#include 

int func0(void) {
    char str[4] = {0};
    strcpy(str, "1234");
    return 0;
}

int main(int argc, char *argv[]) {
    func0();
    return 0;
}

执行结果如下:

==10098==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffdcae5ca24 at pc 0x7faae0b134ca bp 0x7ffdcae5c9f0 sp 0x7ffdcae5c198
WRITE of size 5 at 0x7ffdcae5ca24 thread T0
    #0 0x7faae0b134c9  (/lib64/libasan.so.4+0x794c9)
    #1 0x400a6a in func0() /root/code/cmake_project/app/sanitizer/src/sanitizer_stack_overflow.cpp:7
    #2 0x400ad2 in main /root/code/cmake_project/app/sanitizer/src/sanitizer_stack_overflow.cpp:12
    #3 0x7faadfecf554 in __libc_start_main (/lib64/libc.so.6+0x22554)
    #4 0x4008f8  (/root/code/cmake_project/app/sanitizer/build/sanitizer_stack_overflow+0x4008f8)

Address 0x7ffdcae5ca24 is located in stack of thread T0 at offset 36 in frame
    #0 0x4009b6 in func0() /root/code/cmake_project/app/sanitizer/src/sanitizer_stack_overflow.cpp:5

  This frame has 1 object(s):
    [32, 36) 'str' <== Memory access at offset 36 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/lib64/libasan.so.4+0x794c9) 
Shadow bytes around the buggy address:

说明:

  1. 错误类型是 stack-buffer-overflow
  2. 不合法操作WRITE发生在线程WRITE of size 5 at 0x7ffdcae5ca24 thread T0
  3. 具体发生的位置:/root/code/cmake_project/app/sanitizer/src/sanitizer_stack_overflow.cpp:7
  4. 后面还有影子内存一些指示

2.3.2 heap overflow

src/sanitizer_heap_overflow.cpp 代码如下:

#include 
#include 
#include 

int func1(void) {
    char *p = (char*)malloc(sizeof(char)*4);
    char chs[] = {"12345"};
    memset(p, 0x0, 4);
    if (p != NULL) {
        memcpy(p, chs, 5);
    }
    return 0;
}

int main(int argc, char *argv[]) {
    func1();
    return 0;
}

执行结果如下:

=================================================================
==10373==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000014 at pc 0x7f8f772ba4ca bp 0x7fff5a93fc10 sp 0x7fff5a93f3b8
WRITE of size 5 at 0x602000000014 thread T0
    #0 0x7f8f772ba4c9  (/lib64/libasan.so.4+0x794c9)
    #1 0x400ac2 in func1() /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:10
    #2 0x400b2c in main /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:16
    #3 0x7f8f76676554 in __libc_start_main (/lib64/libc.so.6+0x22554)
    #4 0x4008d8  (/root/code/cmake_project/app/sanitizer/build/sanitizer_heap_overflow+0x4008d8)

0x602000000014 is located 0 bytes to the right of 4-byte region [0x602000000010,0x602000000014)
allocated by thread T0 here:
    #0 0x7f8f7731f8a0 in malloc (/lib64/libasan.so.4+0xde8a0)
    #1 0x400a0a in func1() /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:6
    #2 0x400b2c in main /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:16
    #3 0x7f8f76676554 in __libc_start_main (/lib64/libc.so.6+0x22554)

SUMMARY: AddressSanitizer: heap-buffer-overflow (/lib64/libasan.so.4+0x794c9) 
Shadow bytes around the buggy address:

说明:

  1. 错误类型:heap-buffer-overflow
  2. 错误原因:WRITE of size 5 at 0x60200000eff0 thread T0
  3. 发生位置: #2 0x400b2c in main /root/code/cmake_project/app/sanitizer/src/sanitizer_heap_overflow.cpp:16

2.3.2 use after free

#include 
#include 
#include 

void func2(void) {
    int * a = (int*)malloc(sizeof(int)*1);
    if ( a != NULL ) {
        *a = 1;
        printf("a is:%d.",*a);
        free(a);
        *a = 2;
        printf("error a is:%d.",*a);
    }
}

int main(int argc, char *argv[]) {
    func2();
    return 0;
}

执行结果如下:

=================================================================
==3838==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000000010 at pc 0x000000400a43 bp 0x7ffcdbefd570 sp 0x7ffcdbefd560
WRITE of size 4 at 0x602000000010 thread T0
    #0 0x400a42 in func2() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:11
    #1 0x400a7a in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:17
    #2 0x7ff7391d6554 in __libc_start_main (/lib64/libc.so.6+0x22554)
    #3 0x4008d8  (/root/Public/cmake_code/cmake_project/app/sanitizer/build/sanitizer_use_after_free+0x4008d8)

0x602000000010 is located 0 bytes inside of 4-byte region [0x602000000010,0x602000000014)
freed by thread T0 here:
    #0 0x7ff739e7f508 in __interceptor_free (/lib64/libasan.so.4+0xde508)
    #1 0x400a0b in func2() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:10
    #2 0x400a7a in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:17
    #3 0x7ff7391d6554 in __libc_start_main (/lib64/libc.so.6+0x22554)

previously allocated by thread T0 here:
    #0 0x7ff739e7f8a0 in malloc (/lib64/libasan.so.4+0xde8a0)
    #1 0x400998 in func2() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:6
    #2 0x400a7a in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:17
    #3 0x7ff7391d6554 in __libc_start_main (/lib64/libc.so.6+0x22554)

SUMMARY: AddressSanitizer: heap-use-after-free /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:11 in func2()
Shadow bytes around the buggy address:

说明:

  1. 错误类型:heap-use-after-free
  2. 错误原因:WRITE of size 4 at 0x602000000010 thread T0
  3. 发生位置: #0 0x400a42 in func2() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_use_after_free.cpp:11
  4. 这个挺好的,有明确的错误类型,指示位置也很准确。

2.3.3 global_buffer_overflow

src/sanitizer_global_buffer_overflow.cpp 代码如下:

#include 

int g_abc[11];

int func3(void) {
    int i = 0;
    for (i = 0; i <= 100; i++) {
        printf("value:%d\t",g_abc[i]);
        if (i%10 == 0 && i != 0) {
            printf("\n");
        }
    }

    return g_abc[12];
}

int main() {
    func3();
    return 0;
}

执行结果如下:

value:0	value:0	value:0	value:0	value:0	value:0	value:0	value:0	value:0	value:0	value:0	
=================================================================
==4137==ERROR: AddressSanitizer: global-buffer-overflow on address 0x00000060216c at pc 0x0000004009e9 bp 0x7ffc3d837020 sp 0x7ffc3d837010
READ of size 4 at 0x00000060216c thread T0
    #0 0x4009e8 in func3() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:8
    #1 0x400a86 in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:18
    #2 0x7fd1fe1a2554 in __libc_start_main (/lib64/libc.so.6+0x22554)
    #3 0x4008d8  (/root/Public/cmake_code/cmake_project/app/sanitizer/build/sanitizer_global_buffer_overflow+0x4008d8)

0x00000060216c is located 0 bytes to the right of global variable 'g_abc' defined in '/root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:3:5' (0x602140) of size 44
SUMMARY: AddressSanitizer: global-buffer-overflow /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:8 in func3()
Shadow bytes around the buggy address:

说明:

  1. 错误类型:global-buffer-overflow
  2. 错误原因: READ of size 4 at 0x00000060216c thread T0 即发生了越界读
  3. 错误位置:#0 0x4009e8 in func3() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_global_buffer_overflow.cpp:8

2.3.4 memory leaks

src/sanitizer_memory_leaks.cpp 代码如下:

#include 

char func4() {
    char *x = (char*)malloc(10 * sizeof(char*));
    return x[5];
}

int main(int argc, char *argv[]) {
    func4();
    return 0;
}

ASAN_OPTIONS=detect_leaks=1 ./sanitizer_memory_leaks

=================================================================
==5501==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 80 byte(s) in 1 object(s) allocated from:
    #0 0x7f4d1a3848a0 in malloc (/lib64/libasan.so.4+0xde8a0)
    #1 0x400848 in func4() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_memory_leaks.cpp:4
    #2 0x4008a2 in main /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_memory_leaks.cpp:9
    #3 0x7f4d196db554 in __libc_start_main (/lib64/libc.so.6+0x22554)

SUMMARY: AddressSanitizer: 80 byte(s) leaked in 1 allocation(s).
  1. 显示错误原因为:detected memory leaks
  2. 被泄漏的内存:Direct leak of 80 byte(s) in 1 object(s) allocated from:
  3. 泄漏的具体位置: #1 0x400848 in func4() /root/Public/cmake_code/cmake_project/app/sanitizer/src/sanitizer_memory_leaks.cpp:4
  4. 总结信息:AddressSanitizer: 80 byte(s) leaked in 1 allocation(s).

3. sanitizer 原理介绍

AddressSanitizer主要包括两部分:

  • 插桩(Instrumentation)
  • 动态运行库(Run-time library)。

插桩主要是针对在llvm编译器级别对访问内存的操作(store,load,alloca等),将它们进行处理。

动态运行库主要提供一些运行时的复杂的功能(比如poison/unpoison shadow memory)以及将malloc,free等系统调用函数hook住。

该算法的思路是:如果想防住Buffer Overflow漏洞,只需要在每块内存区域右端(或两端,能防overflow和underflow)加一块区域(RedZone),使RedZone的区域的影子内存(Shadow Memory)设置为不可写即可。具体的示意图如下图所示。

image

  • 内存映射

AddressSanitizer保护的主要原理是对程序中的虚拟内存提供粗粒度的影子内存(每8个字节的内存对应一个字节的影子内存),为了减少overhead,采用了直接内存映射策略,所采用的具体策略如下:Shadow=(Mem >> 3) + offset。每8个字节的内存对应一个字节的影子内存,影子内存中每个字节存取一个数字k,如果k=0,则表示该影子内存对应的8个字节的内存都能访问,如果0

  • 插桩

为了防止buffer overflow,需要将原来分配的内存两边分配额外的内存Redzone,并将这两边的内存加锁,设为不能访问状态,这样可以有效的防止buffer overflow(但不能杜绝buffer overflow)。以下是在栈中插桩的一个例子。

未插桩的代码:

image

插桩后的代码:

image

插桩后的代码:

在动态运行库中将malloc/free函数进行了替换。在malloc函数中额外的分配了Redzone区域的内存,将与Redzone区域对应的影子内存加锁,主要的内存区域对应的影子内存不加锁。
free函数将所有分配的内存区域加锁,并放到了隔离区域的队列中(保证在一定的时间内不会再被malloc函数分配),可检测Use after free类的问题。

详细了解ASan算法原理可以访问以下地址:
https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm

4. 对比 sanitizer 和 valgrind

4.1 sanitizer

  • 包括address, memory, leak等多种sanitizer检测工具

  • 在使用gcc或者clang编译时,加入额外编译选项"-fsanitize=leak"

  • memsanitizer和leaksanitizer只能够在clang中使用

  • 能够准确检测出任何memory leak或者error

  • 如果需要定位到源文件,需要指定以下环境: (否则只会定位到内存地址)

export ASAN_OPTIONS=symbolize=1

export ASAN_SYMBOLIZER_PATH=$(which llvm-symbolizer)

export MSAN_OPTIONS=symbolize=1

export MSAN_SYMBOLIZER_PATH=$(which llvm-symbolizer)

export LSAN_OPTIONS=symbolize=1

export LSAN_SYMBOLIZER_PATH=$(which llvm-symbolizer)

export ASAN_OPTIONS=‘abort_on_error=1’ ==> 过将环境变量 ASAN_OPTIONS 修改成如下形式来迫使软件崩溃

  • 优点: 定位准确, 检查全面, 性能预计降低2倍左右

  • 缺点: 需要重新编译可执行文件

4.2 valgrind

linux平台下的内存检测工具包含多种tool包

  • massif
Usage: valgrind --tool=massif ./target args
  1. 输出: massif.id.out文件, 使用ms_print massif.id.out 即可打印出结果
  2. 作用: 检测runtime时的内存消耗。
  3. 如果是debug版本的程序,可以直接定位到行。
  • memcheck
Usage: valgrind --tool=memcheck --leak-check=full ./target args
  1. 输出:内存泄漏、越界的代码位置

  2. 作用:检测内存泄漏或者内存越界。

  3. 如果是debug版本的程序,可以直接定位到行。

  4. Note:still reacheable部分可以忽略。

  5. 常见问题:
    5.1 malloc, calloc 与free不配对提前return或者goto使用时,造成possible leak

    5.2 free 多次同一内存free未初始化的内存

    5.3 如果使用了tcmalloc4 代替原始的malloc, 会使得valgrind失效

  • 优点: 可以对任何可执行文件使用, 可视化图像显示内存使用

  • 缺点: 常常会有误报, 受编译环境影响较大, 性能预计降低10倍左右

4. 参考资料

  • https://www.jianshu.com/p/9e85345e500b
  • https://www.bynav.com/cn/resource/bywork/healthy-work/70.html
  • https://juejin.im/post/6844904067538370573

你可能感兴趣的:(微服务,内存泄漏,c++)