内存问题的排查工具和方法– Clang的AddressSanitizer

How to use asan to debug native memory bugs?

AddressSanitizer (ASan)是Google开发的一款用于检查Native内存错误的工具，类似于Valgrind。其官网介绍如下：

AddressSanitizer (ASan) is a fast compiler-based tool for detecting memory bugs in native code. It is comparable to Valgrind (Memcheck tool), but, unlike it, ASan: + detects overflows on stack and global objects - does not detect uninitialized reads and memory leaks + is much faster (two-three times slowdown compared to Valgrind’s 20-100x) + has less memory overhead

经实际确认，目前在我们Android  project上可以有如下两种使用方式：

1. Enable AddressSanitizer for all apps：

这种方式修改相对简单，只需要做如下两部分修改：

In frameworks/base/cmds/app_process/Android.mk add

“LOCAL_SANITIZE:=address”

In system/core/rootdir/init.zygote(32|64).rc to add the following lines:

setenv LD_LIBRARY_PATH=/system/lib/asan:/system/libs

setenv ASAN_OPTIONS alloc_dealloc_mismatch=0:allow_user_segv_handler=true

这样修改后，就可以利用ASan来同时检测所有APP的native memory bugs, 但有利有弊，这种方式也有一定的弊端：

• System is very slow because ASan is enabled in all the apps.
• LMK are seen often.

2. Enable AddressSanitizer for specific APP or Java Process:

Google官网上有介绍，我们也可以只对某个特定的APP来enable ASan, 其使用的方式是设定wrap. property的方式，这样就可以利用property中所指定的程序来启动相应的应用程序或进程。

但是我们目前拿到的Android N AOSP这一部分存在bug，设定wrap. property后会导致系统crash， 关于这点，需要从Master branch sync如下这笔改动：

https://android-review.googlesource.com/#/c/platform/frameworks/base/+/318859

1) How to enable ASan for some APPs or Java processes：

• Runtime阶段设定Property的方式：
  setprop wrap.com.google.android.gm "asanwrapper"
  
  
• 以下几种情况通过 设定Property的方式并不合适：
  • 需要在boot阶段就来检测某个APP或Java Process的内存问题；
  • package_name/process_name中含有无效字符，例如setprop: invalid character ':'；
  • wrap.的长度大于31.

  ==>对于这种情况，我们可以直接修改code的方式来enable ASan, example如下：frameworks/base/core/java/com/android/internal/os/ZygoteConnection.java

2) 对于这种使用场景，我们需要用到asanwrapper和libclang_rt.asan-arm-android.so这两个档案（如附件）：

• libclang_rt.asan-arm-android.so可以类似于使用方式1中的修改partial build产出；

• asanwrapper的内容如下：

  #!/system/bin/sh ASAN_OPTIONS=start_deactivated=1,alloc_dealloc_mismatch=0 LD_PRELOAD=libclang_rt.asan-arm-android.so \ exec $@

有如下两种方式将两个档案放入系统中：

• 修改device.mk直接package所这两个档案：

• Runtime push的方式：

  adb push asanwrapper /system/bin/ adb push libclang_rt.asan-arm-android.so /system/lib/ adb shell sync adb shell chmod 0777 /system/bin/ asanwrapper adb shell chmod 0777 /system/lib/ libclang_rt.asan-arm-android.so adb reboot

3. 注意：对于上述两种使用的方式，在测试前，我们都需要设定SELinux为permissive mode, 有如下两种修改方式：

• Press "Enter" key then AC off->on, then console will stop at uboot then input below cmds:

  addboot androidboot.selinux=permissive saveenv reset

• 直接修改source code的方式: system/core/init/init.cpp

 

4. ASan扫描出来的memory bugs示例：

1 概述

Valgrind可以有效地监测处大多数内存问题，你肯定忍不住会想，既然c/c++的内存问题这么常见，为什么不在编译器中加入内存问题检测的功能呢？ 很可惜，GCC中还目前还不支持内存检测，可喜的是，clang支持。这里我们看看如何用clang发现内存问题

2 clang

clang 是一个C、C++、Objective-C编程语言的编译器前端。它采用了底层虚拟机作为其后端。它的目标是提供一个GNU编译器套装（GCC）的替代品， 作者是克里斯·拉特纳，在苹果公司的赞助下进行开发。

3 内存泄漏监测

AddressSanitizer是clang中的一个内存错误检测器，它可以检测到以下问题：

• Out-of-bounds accesses to heap, stack and globals
• Use-after-free
• Use-after-return (to some extent)
• Double-free, invalid free
• Memory leaks (experimental)

使用clang编译代码时用-fsanitize=address就能打开AddressSanitizer工具，为了在检测到内存错误时打印出您的程序调用栈，需要在编译时加上选项 -fno-omit-frame-pointer选项，同时为了得出更清晰的调用栈信息，请用-O1选项编译程序。

4 示例代码

下面我用clang3.4做一个示例

 1:  int main()
 2:  {
 3:      char *p = malloc(sizeof(char) * 10);
 4:      if (p == NULL) {
 5:          return 0;
 6: }  7:  8: struct elem *e = malloc(sizeof(struct elem));  9: if (e == NULL) { 10: free(p); 11: return 0; 12: } 13: 14: e->a = 10; 15: e->b = 10.10; 16: e->c = p; 17: 18: double *xx = &e->b; 19: 20: printf("%f\n", *xx); 21: 22: free(e); 23: 24: printf("%f\n", *xx); 25: 26: return 0; 27: } 

上面的代码中有两处问题，一是p未被释放，导致了内存泄漏；二是xx指向了一块被释放了的内存。我们看看怎么用clang检测这两个问题

4.1 编译它

1:  clang -O1 -g -fsanitize=address -fno-omit-frame-pointer -o core core.c

4.2 用AddressSanitizer监测进程的内存泄漏

直接运行core文件，它就会自动打印出检测到的内存错误

 1:  [cobbliu@kftest25 test]$ ./core
 2:  10.100000
 3:  =================================================================
 4:  ==11254==ERROR: AddressSanitizer: heap-use-after-free on address 0x60300000efe8 at pc 0x48211a bp 0x7fff2c776450 sp 0x7fff2c776448
 5:  READ of size 8 at 0x60300000efe8 thread T0
 6: #0 0x482119 in main /home/cobbliu/test/core.c:35  7: #1 0x36a101ecdc in __libc_start_main (/lib64/libc.so.6+0x36a101ecdc)  8: #2 0x481f3c in _start (/home/cobbliu/test/core+0x481f3c)  9: 10: 0x60300000efe8 is located 8 bytes inside of 24-byte region [0x60300000efe0,0x60300000eff8) 11: freed by thread T0 here: 12: #0 0x46bca9 in __interceptor_free /home/ads/build23_6u0_x64/workspace/t-coresystem-clang/label/build23_6u0_x64/t-coresystem-clang/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:64 13: #1 0x4820c0 in main /home/cobbliu/test/core.c:32 14: #2 0x36a101ecdc in __libc_start_main (/lib64/libc.so.6+0x36a101ecdc) 15: 16: previously allocated by thread T0 here: 17: #0 0x46be29 in malloc /home/ads/build23_6u0_x64/workspace/t-coresystem-clang/label/build23_6u0_x64/t-coresystem-clang/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:74 18: #1 0x48202a in main /home/cobbliu/test/core.c:18 19: #2 0x36a101ecdc in __libc_start_main (/lib64/libc.so.6+0x36a101ecdc) 20: 21: SUMMARY: AddressSanitizer: heap-use-after-free /home/cobbliu/test/core.c:35 main 22: Shadow bytes around the buggy address: 23: 0x0c067fff9da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 24: 0x0c067fff9db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 25: 0x0c067fff9dc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 26: 0x0c067fff9dd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 27: 0x0c067fff9de0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 28: =>0x0c067fff9df0: fa fa fa fa fa fa fa fa fa fa fa fa fd[fd]fd fa 29: 0x0c067fff9e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 30: 0x0c067fff9e10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 31: 0x0c067fff9e20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 32: 0x0c067fff9e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 33: 0x0c067fff9e40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 34: Shadow byte legend (one shadow byte represents 8 application bytes): 35: Addressable: 00 36: Partially addressable: 01 02 03 04 05 06 07 37: Heap left redzone: fa 38: Heap right redzone: fb 39: Freed heap region: fd 40: Stack left redzone: f1 41: Stack mid redzone: f2 42: Stack right redzone: f3 43: Stack partial redzone: f4 44: Stack after return: f5 45: Stack use after scope: f8 46: Global redzone: f9 47: Global init order: f6 48: Poisoned by user: f7 49: ASan internal: fe 50: ==11254==ABORTING 

可以看到，程序在提示core.c的第35行有个heap-use-after-free的错误，而且在最后还有个summary，把出错的代码位置和相应的栈信息打了出来。

5 示例代码2

上面的代码做一些小修改，我们看看它对double-free问题的检测

1:  /...
2:      struct elem *e2 = e;
3:      free(e);
4:      free(e2);
5:  /...
6: } 

按照上面相同的方法编译并运行后，提示信息如下：

 1:  [cobbliu@kftest25 test]$ ./core
 2:  10.100000
 3:  =================================================================
 4:  ==11952==ERROR: AddressSanitizer: attempting double-free on 0x60300000efe0 in thread T0:
 5:      #0 0x46bca9 in __interceptor_free /home/ads/build23_6u0_x64/workspace/t-coresystem-clang/label/build23_6u0_x64/t-coresystem-clang/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:64
 6: #1 0x4820bd in main /home/cobbliu/test/core.c:34  7: #2 0x36a101ecdc in __libc_start_main (/lib64/libc.so.6+0x36a101ecdc)  8: #3 0x481f3c in _start (/home/cobbliu/test/core+0x481f3c)  9: 10: 0x60300000efe0 is located 0 bytes inside of 24-byte region [0x60300000efe0,0x60300000eff8) 11: freed by thread T0 here: 12: #0 0x46bca9 in __interceptor_free /home/ads/build23_6u0_x64/workspace/t-coresystem-clang/label/build23_6u0_x64/t-coresystem-clang/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:64 13: #1 0x4820b0 in main /home/cobbliu/test/core.c:33 14: #2 0x36a101ecdc in __libc_start_main (/lib64/libc.so.6+0x36a101ecdc) 15: 16: previously allocated by thread T0 here: 17: #0 0x46be29 in malloc /home/ads/build23_6u0_x64/workspace/t-coresystem-clang/label/build23_6u0_x64/t-coresystem-clang/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:74 18: #1 0x482026 in main /home/cobbliu/test/core.c:18 19: #2 0x36a101ecdc in __libc_start_main (/lib64/libc.so.6+0x36a101ecdc) 20: 21: SUMMARY: AddressSanitizer: double-free /home/ads/build23_6u0_x64/workspace/t-coresystem-clang/label/build23_6u0_x64/t-coresystem-clang/llvm-3.4/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:64 __interceptor_free 22: ==11952==ABORTING 

可以看到，AddressSanitizer报错，说core.c的34行有一个double-free的错误

6 示例代码3

上面的代码做一些小修改，把释放e的代码注释掉，看看它对内存泄漏的检测

1:  /...
2:      //free(e);
3:  /...
4:  }

按照上面相同的方法编译并运行后，提示信息如下：

1:  [cobbliu@kftest25 test]$ ./core
2:  10.100000

可以看到，对内存泄漏，AddressSanitizer无法检测出来 clang中有一个工具叫LeakSanitizer，它的设计目标是用来检测内存泄漏。直到3.7版，LeakSanitizer也是在实验阶段。

7 AddressSanitizer的缺陷

• AddressSanitizer工具编译的程序的堆栈和栈占用比原生程序的大。
• AddressSanitizer不支持静态编译

 

更新：gcc4.8版本之后，有了对AddressSanitizer的支持！