通过ramdump 分析匿名页导致的TASK_UNINTERRUPTIBLE
再回收内存时出现 kernel callsite when blocked:: "page_lock_anon_vma_read+0xdc/0x138" 处于 TASK_UNINTERRUPTIBLE状态，初步分析和读写锁有关，通过ramdump 查找读写锁得持有者
在分配内存时长时间处于不可打断得状态，通过crash 工具解析ramdump 查找block 得原因
crash> ps -u |grep UN
10457 1 7 ffffffea80bb2f40 UN 6.2 8856372 646112 [email protected]
10515 673 5 ffffffeb71240000 UN 1.4 2069416 146256 azeroth-api-thrprovider 是当前正在使用得任务， 通过bt 命令查看调用栈
crash> bt 10457
PID: 10457 TASK: ffffffea80bb2f40 CPU: 7 COMMAND: "[email protected]"
#0 [ffffff801e24ace0] __switch_to at ffffffa9a2e87a98
#1 [ffffff801e24ad70] __schedule at ffffffa9a424d924
#2 [ffffff801e24ad90] schedule at ffffffa9a424dd6c
#3 [ffffff801e24ae10] rwsem_down_read_failed at ffffffa9a42510dc
#4 [ffffff801e24ae30] down_read at ffffffa9a42506f0
#5 [ffffff801e24ae60] page_lock_anon_vma_read at ffffffa9a3078d48
#6 [ffffff801e24aeb0] rmap_walk_anon at ffffffa9a307a354
#7 [ffffff801e24af40] page_referenced at ffffffa9a30790c8
#8 [ffffff801e24b090] shrink_page_list at ffffffa9a30417d0
#9 [ffffff801e24b130] shrink_inactive_list at ffffffa9a3046bf0
#10 [ffffff801e24b220] shrink_node_memcg at ffffffa9a30436d8
#11 [ffffff801e24b2b0] shrink_node at ffffffa9a3045cf0
#12 [ffffff801e24b330] do_try_to_free_pages at ffffffa9a3042df0
#13 [ffffff801e24b420] try_to_free_pages at ffffffa9a3042a0c
#14 [ffffff801e24b590] __alloc_pages_nodemask at ffffffa9a302dff4
#15 [ffffff801e24b5d0] ion_page_pool_alloc at ffffffa9a3a31278
#16 [ffffff801e24b630] alloc_largest_available at ffffffa9a3a32c54
#17 [ffffff801e24b6e0] ion_system_heap_allocate at ffffffa9a3a32484
#18 [ffffff801e24b7b0] ion_alloc_dmabuf at ffffffa9a3a2d218
#19 [ffffff801e24b7f0] ion_alloc at ffffffa9a3a2d5ec
#20 [ffffff801e24b890] cam_mem_mgr_alloc_and_map at ffffffa9a3dd43e0
#21 [ffffff801e24bbc0] cam_private_ioctl at ffffffa9a3dd2368
#22 [ffffff801e24bc60] __video_do_ioctl at ffffffa9a3888468
#23 [ffffff801e24bd70] video_usercopy at ffffffa9a3887e98
#24 [ffffff801e24bd80] video_ioctl2 at ffffffa9a38881a8
#25 [ffffff801e24bd90] v4l2_ioctl at ffffffa9a3887594
#26 [ffffff801e24be20] do_vfs_ioctl at ffffffa9a30c32b0
#27 [ffffff801e24be60] __arm64_sys_ioctl at ffffffa9a30c3704
#28 [ffffff801e24bea0] el0_svc_common at ffffffa9a2e96eb4
#29 [ffffff801e24beb0] el0_svc_handler at ffffffa9a2e96df8
#30 [ffffff801e24bff0] el0_svc at ffffffa9a2e83844
PC: 0000007c0f728c88 LR: 0000007c0f6e31a8 SP: 0000007ab8063c10
X29: 0000007ab8063d00 X28: 0000007c0a4efd38 X27: 0000007ab8067018
X26: 0000007c0a2458b0 X25: 0000000001e07000 X24: 0000000000000818
X23: 0000007ab8067018 X22: 0000000000000112 X21: 0000007ab8064068
X20: 0000007a6d4b7b28 X19: 0000007ab8067018 X18: 0000007ab7a2c000
X17: 0000007c0f6e3120 X16: 0000007c0a1a7c70 X15: 0000000000000001
X14: 0000000000000000 X13: 0000000000000410 X12: 0000007ab8063d10
X11: 0000007ab8063cc8 X10: 0000007ab8063c90 X9: 0000007ab8063cc8
X8: 000000000000001d X7: 0000000000000010 X6: 0000007c0a2458b0
X5: 0000007c0a1953c8 X4: 0000000000000068 X3: 0000000000000000
X2: 0000007ab8063d38 X1: 00000000c01856c0 X0: 000000000000000a
ORIG_X0: 000000000000000a SYSCALLNO: 1d PSTATE: a0001000
可以很明显得看到在down_read 时 获取读写锁失败导致block 在这里。 provider 任务通过ion分配内存，由于内存不足，在分配得时候直接开始回收内存，在回收内存时需要将回收得页面通过反向映射机制找到映射过这个物理页面的任务，并解除映射关系。但是由于匿名页在父任务创建子任务时惠继承，导致很多子任务使用的是同一个anon_vma ,读写锁也是一个，不同 app 的任务会相互制约，后面分析是哪个任务拿到了这个读写锁不释放。匿名页的反向映射具体参考https://www.cnblogs.com/tolimit/p/5398552.html
要找到当前读写锁的持有者就要在内核的配置文件中加入CONFIG_RWSEM_SPIN_ON_OWNER，这样owner 成员会指向当前正在持有写锁的任务。
要找到owner 成员首先就得找到anon_vma 这个结构体当前在内存中地址，需要分析page_lock_anon_vma_read 函数中是否
有关键成员压入到栈中，通过栈里得数据找到关键结构体地址，最后找到owner。
struct anon_vma *page_lock_anon_vma_read(struct page *page)
{
struct anon_vma *anon_vma = NULL;
struct anon_vma *root_anon_vma;
unsigned long anon_mapping;
rcu_read_lock();
// 如果此页被分配作为一个匿名页，那么它的mapping会指向一个anon_vma
anon_mapping = (unsigned long)READ_ONCE(page->mapping);
//判断是不是匿名页， 因为是文件页时也是mapping 指向文件页
if ((anon_mapping & PAGE_MAPPING_FLAGS) != PAGE_MAPPING_ANON)
goto out;
if (!page_mapped(page))
goto out;
........
if (!page_mapped(page)) {
rcu_read_unlock();
put_anon_vma(anon_vma);
return NULL;
}
/* we pinned the anon_vma, its safe to sleep */
rcu_read_unlock();
anon_vma_lock_read(anon_vma);//获得读者锁
if (atomic_dec_and_test(&anon_vma->refcount)) {
/*
* Oops, we held the last refcount, release the lock
* and bail -- can't simply use put_anon_vma() because
* we'll deadlock on the anon_vma_lock_write() recursion.
*/
anon_vma_unlock_read(anon_vma);
__put_anon_vma(anon_vma);
anon_vma = NULL;
}
return anon_vma;
out:
rcu_read_unlock();
return anon_vma;
}mapping保存anon_vma变量地址时，会执行如下代码：
page->mapping = (void *)&anon_vma + 1;anon_vma分配时要2字节对齐，也就是所有分配的anon_vma其最低位都为0，经过上面的操作，mapping最低位就为1了，然后通过mapping获取anon_vma地址时，进行如下操作：
struct anon_vma * anon_vma = (struct anon_vma *)(page->mapping - 1);上面函数中得代码
anon_vma = (struct anon_vma *) (anon_mapping - PAGE_MAPPING_ANON);PAGE_MAPPING_ANON 就是1，#define PAGE_MAPPING_ANON 0x1
所以如果想确定anon_vma 得地址，知道传入进来得page 地址也是可以得。
所以如果想确定anon_vma 得地址，知道传入进来得page 地址也是可以得。查找调用栈中是否有page 相关的信息压栈
首先反汇编下page_lock_anon_vma_read
crash> dis page_lock_anon_vma_read
0xffffffa9a3078c70 : str x21, [sp,#-48]!
0xffffffa9a3078c74 : stp x20, x19, [sp,#16]
0xffffffa9a3078c78 : stp x29, x30, [sp,#32]
0xffffffa9a3078c7c : add x29, sp, #0x20
0xffffffa9a3078c80 : mov x20, x0
0xffffffa9a3078c84 : bl 0xffffffa9a2f5e940
0xffffffa9a3078c88 : ldr x19, [x20,#24]
0xffffffa9a3078c8c : and x8, x19, #0x3
0xffffffa9a3078c90 : cmp x8, #0x1
0xffffffa9a3078c94 : b.ne 0xffffffa9a3078ccc
0xffffffa9a3078c98 : mov x0, x20
0xffffffa9a3078c9c : bl 0xffffffa9a304ef88
0xffffffa9a3078ca0 : tbz w0, #0, 0xffffffa9a3078ccc
0xffffffa9a3078ca4 : ldr x8, [x19,#-1]!
0xffffffa9a3078ca8 : add x21, x8, #0x8
0xffffffa9a3078cac : mov x0, x21
0xffffffa9a3078cb0 : bl 0xffffffa9a2f3f438
0xffffffa9a3078cb4 : cbz w0, 0xffffffa9a3078ce8
0xffffffa9a3078cb8 : mov x0, x20
0xffffffa9a3078cbc : bl 0xffffffa9a304ef88
0xffffffa9a3078cc0 : tbnz w0, #0, 0xffffffa9a3078cd0
0xffffffa9a3078cc4 : mov x0, x21
0xffffffa9a3078cc8 : bl 0xffffffa9a2f3f4d8
0xffffffa9a3078ccc : mov x19, xzr
0xffffffa9a3078cd0 : bl 0xffffffa9a2f5e958
0xffffffa9a3078cd4 : mov x0, x19
0xffffffa9a3078cd8 : ldp x29, x30, [sp,#32]
0xffffffa9a3078cdc : ldp x20, x19, [sp,#16]
0xffffffa9a3078ce0 : ldr x21, [sp],#48
0xffffffa9a3078ce4 : ret
0xffffffa9a3078ce8 : ldr w8, [x19,#56]
0xffffffa9a3078cec : cbz w8, 0xffffffa9a3078ccc
0xffffffa9a3078cf0 : add w10, w8, #0x1
0xffffffa9a3078cf4 : add x11, x19, #0x38
0xffffffa9a3078cf8 : sxtw x9, w8
0xffffffa9a3078cfc : sxtw x10, w10
0xffffffa9a3078d00 : prfm pstl1strm, [x11]
0xffffffa9a3078d04 : ldxr w13, [x11]
0xffffffa9a3078d08 : eor w12, w13, w9
0xffffffa9a3078d0c : cbnz w12, 0xffffffa9a3078d1c
0xffffffa9a3078d10 : stlxr w12, w10, [x11]
0xffffffa9a3078d14 : cbnz w12, 0xffffffa9a3078d04
0xffffffa9a3078d18 : dmb ish
0xffffffa9a3078d1c : cmp w8, w13
0xffffffa9a3078d20 : mov w8, w13
0xffffffa9a3078d24 : b.ne 0xffffffa9a3078cec
0xffffffa9a3078d28 : mov x0, x20
0xffffffa9a3078d2c : bl 0xffffffa9a304ef88
0xffffffa9a3078d30 : mov w20, w0
0xffffffa9a3078d34 : bl 0xffffffa9a2f5e958
0xffffffa9a3078d38 : tbz w20, #0, 0xffffffa9a3078d78
0xffffffa9a3078d3c : mov x20, x19
0xffffffa9a3078d40 : ldr x8, [x20],#56
0xffffffa9a3078d44 : add x0, x8, #0x8
0xffffffa9a3078d48 : bl 0xffffffa9a42506a0
0xffffffa9a3078d4c : prfm pstl1strm, [x20]
0xffffffa9a3078d50 : ldxr w8, [x20]
0xffffffa9a3078d54 : sub w8, w8, #0x1
0xffffffa9a3078d58 : stlxr w9, w8, [x20]
0xffffffa9a3078d5c : cbnz w9, 0xffffffa9a3078d50
0xffffffa9a3078d60 : dmb ish
0xffffffa9a3078d64 : cbnz w8, 0xffffffa9a3078cd4
0xffffffa9a3078d68 : ldr x8, [x19]
0xffffffa9a3078d6c : add x0, x8, #0x8
0xffffffa9a3078d70 : bl 0xffffffa9a2f3f4d8
0xffffffa9a3078d74 : b 0xffffffa9a3078d98
0xffffffa9a3078d78 : add x8, x19, #0x38
0xffffffa9a3078d7c : prfm pstl1strm, [x8]
0xffffffa9a3078d80 : ldxr w9, [x8]
0xffffffa9a3078d84 : sub w9, w9, #0x1
0xffffffa9a3078d88 : stlxr w10, w9, [x8]
0xffffffa9a3078d8c : cbnz w10, 0xffffffa9a3078d80
0xffffffa9a3078d90 : dmb ish
0xffffffa9a3078d94 : cbnz w9, 0xffffffa9a3078da0
0xffffffa9a3078d98 : mov x0, x19
0xffffffa9a3078d9c : bl 0xffffffa9a3078da8
0xffffffa9a3078da0 : mov x19, xzr
0xffffffa9a3078da4 : b 0xffffffa9a3078cd4 page_lock_anon_vma_read 只有一个参数page， 根据arm 体系结构，前6 个寄存器都是通过 寄存器传值，所以x0寄存器存放着page 参数的值
在page_lock_anon_vma_read 反汇编中没有发现x0寄存器有压栈操作。这样就没办法从page_lock_anon_vma_read 的栈帧中查找到page 的地址信息。
再往上查看rmap_walk_anon 函数，如果rmap_walk_anon在调用page_lock_anon_vma_read 之前将page 值压入到栈中，那也可以获得到page 信息。或者某个寄存器含有page 的值，在page_lock_anon_vma_read
执行时候将这个寄存器压栈了，我们看到page_lock_anon_vma_read函数除了x29,x30这两个栈帧寄存器，返回地址寄存器被压栈了，还将x20, x19 进行了压栈。如果x20,x19 正好保存了page 的地址我们也就得到了page
的压栈信息。
查看rmap_walk_anon函数内容，看是否能找到page 相关的信息。rmap_walk_anon没有直接调用page_lock_anon_vma_read，而是通过page_anon_vma->page_lock_anon_vma_read
static void rmap_walk_anon(struct page *page, struct rmap_walk_control *rwc,
bool locked)
{
struct anon_vma *anon_vma;
pgoff_t pgoff_start, pgoff_end;
struct anon_vma_chain *avc;
if (rwc->target_vma) {
unsigned long address = vma_address(page, rwc->target_vma);
rwc->rmap_one(page, rwc->target_vma, address, rwc->arg);
return;
}
if (locked) {
anon_vma = page_anon_vma(page); //这个里把获得到的page，传递给了page_anon_vma
/* anon_vma disappear under us? */
VM_BUG_ON_PAGE(!anon_vma, page);
} else {
anon_vma = rmap_walk_anon_lock(page, rwc);
}
省略一部分代码................
}page_anon_vma 的page 参数是通过x0 寄存器传入，我们反汇编看看x0相关的寄存器是否在调用page_anon_vma之前有压栈
反汇编page_anon_vma
crash> dis rmap_walk_anon
0xffffffa9a307a2c8 : stp x26, x25, [sp,#-80]!
0xffffffa9a307a2cc : stp x24, x23, [sp,#16]
0xffffffa9a307a2d0 : stp x22, x21, [sp,#32]
0xffffffa9a307a2d4 : stp x20, x19, [sp,#48]
0xffffffa9a307a2d8 : stp x29, x30, [sp,#64]
0xffffffa9a307a2dc : add x29, sp, #0x40
0xffffffa9a307a2e0 : mov x19, x1
0xffffffa9a307a2e4 : ldr x1, [x1,#8]
0xffffffa9a307a2e8 : mov x20, x0
0xffffffa9a307a2ec : cbz x1, 0xffffffa9a307a334
0xffffffa9a307a2f0 : ldr x8, [x20,#32]
0xffffffa9a307a2f4 : mov x0, x20
0xffffffa9a307a2f8 : ldr x9, [x1,#152]
0xffffffa9a307a2fc : ldr x10, [x1]
0xffffffa9a307a300 : ldr x3, [x19]
0xffffffa9a307a304 : sub x8, x8, x9
0xffffffa9a307a308 : ldr x9, [x19,#16]
0xffffffa9a307a30c : add x8, x10, x8, lsl #12
0xffffffa9a307a310 : cmp x8, x10
0xffffffa9a307a314 : csel x2, x8, x10, hi
0xffffffa9a307a318 : blr x9
0xffffffa9a307a31c : ldp x29, x30, [sp,#64]
0xffffffa9a307a320 : ldp x20, x19, [sp,#48]
0xffffffa9a307a324 : ldp x22, x21, [sp,#32]
0xffffffa9a307a328 : ldp x24, x23, [sp,#16]
0xffffffa9a307a32c : ldp x26, x25, [sp],#80
0xffffffa9a307a330 : ret
0xffffffa9a307a334 : mov w21, w2
0xffffffa9a307a338 : tbz w2, #0, 0xffffffa9a307a348
0xffffffa9a307a33c : mov x0, x20
0xffffffa9a307a340 : bl 0xffffffa9a304f018 ///anon_vma = page_anon_vma(page);
0xffffffa9a307a344 : b 0xffffffa9a307a358
0xffffffa9a307a348 : ldr x8, [x19,#32]
0xffffffa9a307a34c : cbz x8, 0xffffffa9a307a364
0xffffffa9a307a350 : mov x0, x20
0xffffffa9a307a354 : blr x8
0xffffffa9a307a358 : mov x22, x0
0xffffffa9a307a35c : cbnz x0, 0xffffffa9a307a380
0xffffffa9a307a360 : b 0xffffffa9a307a31c
0xffffffa9a307a364 : mov x0, x20
0xffffffa9a307a368 : bl 0xffffffa9a304f018
0xffffffa9a307a36c : cbz x0, 0xffffffa9a307a31c
0xffffffa9a307a370 : ldr x8, [x0]
0xffffffa9a307a374 : mov x22, x0
0xffffffa9a307a378 : add x0, x8, #0x8
0xffffffa9a307a37c : bl 0xffffffa9a42506a0
0xffffffa9a307a380 : ldr x23, [x20,#32]
0xffffffa9a307a384 : add x0, x22, #0x48
0xffffffa9a307a388 : mov x1, x23
0xffffffa9a307a38c : mov x2, x23
0xffffffa9a307a390 : bl 0xffffffa9a3064e18
0xffffffa9a307a394 : cbz x0, 0xffffffa9a307a41c
0xffffffa9a307a398 : mov x24, x0
0xffffffa9a307a39c : ldr x25, [x24]
0xffffffa9a307a3a0 : ldr x8, [x20,#32]
0xffffffa9a307a3a4 : ldr x9, [x25,#152]
0xffffffa9a307a3a8 : ldr x10, [x25]
0xffffffa9a307a3ac : sub x8, x8, x9
0xffffffa9a307a3b0 : add x9, x10, x8, lsl #12
0xffffffa9a307a3b4 : ldr x8, [x19,#40]
0xffffffa9a307a3b8 : cmp x9, x10
0xffffffa9a307a3bc : csel x26, x9, x10, hi
0xffffffa9a307a3c0 : cbz x8, 0xffffffa9a307a3d4
0xffffffa9a307a3c4 : ldr x1, [x19]
0xffffffa9a307a3c8 : mov x0, x25
0xffffffa9a307a3cc : blr x8
0xffffffa9a307a3d0 : tbnz w0, #0, 0xffffffa9a307a404
0xffffffa9a307a3d4 : ldr x8, [x19,#16]
0xffffffa9a307a3d8 : mov x0, x20
0xffffffa9a307a3dc : ldr x3, [x19]
0xffffffa9a307a3e0 : mov x1, x25
0xffffffa9a307a3e4 : mov x2, x26
0xffffffa9a307a3e8 : blr x8
0xffffffa9a307a3ec : tbz w0, #0, 0xffffffa9a307a41c
0xffffffa9a307a3f0 : ldr x8, [x19,#24]
0xffffffa9a307a3f4 : cbz x8, 0xffffffa9a307a404
0xffffffa9a307a3f8 : mov x0, x20
0xffffffa9a307a3fc : blr x8
0xffffffa9a307a400 : cbnz w0, 0xffffffa9a307a41c
0xffffffa9a307a404 : mov x0, x24
0xffffffa9a307a408 : mov x1, x23
0xffffffa9a307a40c : mov x2, x23
0xffffffa9a307a410 : bl 0xffffffa9a3064ea8
0xffffffa9a307a414 : mov x24, x0
0xffffffa9a307a418 : cbnz x0, 0xffffffa9a307a39c
0xffffffa9a307a41c : tbnz w21, #0, 0xffffffa9a307a31c
0xffffffa9a307a420 : ldr x8, [x22]
0xffffffa9a307a424 : add x0, x8, #0x8
0xffffffa9a307a428 : bl 0xffffffa9a2f3f4d8
0xffffffa9a307a42c : b 0xffffffa9a307a31c
我们要查找那个函数跳转指令是跳转到page_anon_vma。 有个简单方法是执行sym 命令
crash> sym 0xffffffa9a304f018
ffffffa9a304f018 (T) page_anon_vma /home/mi/miui/J1/kernel/msm-4.19/include/linux/compiler.h: 193确定了跳转page_anon_vma 的代码是那一句，在往前查找x0寄存器。 从代码中看到x0 的值是来自x20,也就是x20 就是存放着x20的page 的值
联系前面的page_lock_anon_vma_read 函数，进去后就立马将x20压栈，我们只要从page_lock_anon_vma_read栈中读取出来x20 压入的数据就得到page
0xffffffa9a3078c74 : stp x20, x19, [sp,#16] 上面这句压栈指令，是将x20 压入到sp +16 的位置 这里的#16 是10进制的，也就是sp + 0x10 就是x20寄存器的值，现在要计算出page_lock_anon_vma_read 的sp 值。
要计算sp 还是要从调用栈入手
#4 [ffffff801e24ae30] down_read at ffffffa9a42506f0
down_read 前面的ffffff801e24ae30 是 page_lock_anon_vma_read 的fp。 将fp +0x10 就是page_lock_anon_vma_read 的sp
备注： 为什么 sp 是fp +x10，从的反汇编能够告诉我们为什么是0x10
crash> dis down_read
0xffffffa9a42506a0
0xffffffa9a42506a4
刚进函数sp = sp - 32; fp(x29) = sp + 16; fp(x29) = sp - 16; 转换成16 进制就是 fp = sp - 0x10;
经过以上信息最后计算page_lock_anon_vma_read 的sp = ffffff801e24ae30 + 0x10 = ffffff801e24ae40;
page_lock_anon_vma_read 的栈中x20的值 = sp + 16 = sp +0x10 = ffffff801e24ae40 + 0x10 = ffffff801e24ae50;
即 page = ffffff801e24ae50;
读此地址的值，就是page 页面的位置
crash> rd ffffff801e24ae50 4
ffffff801e24ae50: ffffffbfaf1bbb80 ffffff801e24aed0 ..........$.....
ffffff801e24ae60: ffffff801e24aeb0 ffffffa9a307a358 ..$.....X.......ffffff801e24ae50 地址的值是ffffffbfaf1bbb80，查看page 得相关信息
crash> struct page ffffffbfaf1bbb80
struct page {
flags = 525389,
{
{
lru = {
next = 0xdead000000000100,
prev = 0xdead000000000200
},
mapping = 0xffffffec1bcc38a1, //如果时匿名页最后一位会是1，这里是1
//也再次验证查找得是对得
index = 62,//再vma 中对应得页 index
private = 265679
},
{
{
slab_list = {
next = 0xdead000000000100,
prev = 0xdead000000000200
},
{
next = 0xdead000000000100,
pages = 512,
pobjects = -559087616
}
},
slab_cache = 0xffffffec1bcc38a1,
freelist = 0x3e,
{
s_mem = 0x40dcf,
counters = 265679,
{
inuse = 3535,
objects = 4,
frozen = 0
}
}
},
{
compound_head = 16045481047390945536,
compound_dtor = 0 '\000',
compound_order = 2 '\002',
compound_mapcount = {
counter = -559087616
}
},
{
_compound_pad_1 = 16045481047390945536,
_compound_pad_2 = 16045481047390945792,
deferred_list = {
next = 0xffffffec1bcc38a1,
prev = 0x3e
}
},
{
_pt_pad_1 = 16045481047390945536,
pmd_huge_pte = 0xdead000000000200,
_pt_pad_2 = 18446743988276574369,
{
pt_mm = 0x3e,
pt_frag_refcount = {
counter = 62
}
},
ptl = {
{
rlock = {
raw_lock = {
{
val = {
counter = 265679
},
{
locked = 207 '\317',
pending = 13 '\r'
},
{
locked_pending = 3535,
tail = 4
}
}
}
}
}
}
},
{
pgmap = 0xdead000000000100,
hmm_data = 16045481047390945792,
_zd_pad_1 = 18446743988276574369
},
callback_head = {
next = 0xdead000000000100,
func = 0xdead000000000200
}
},
{
_mapcount = {
counter = 14
},
page_type = 14,
active = 14,
units = 14
},
_refcount = {
counter = 17
},
mem_cgroup = 0xffffffeabfcb1000
}mapping = 0xffffffec1bcc38a1 的最后一位是1,证明找到的page 地址应该是对的。这里需要主要如果计算的page 地址出错最后读取struct page 也是能读出信息，但是信息可能是错误的。
要通过读出的信息去判断上面反推的对不对。
index = 62 存放着是 page 对应的虚拟地址在vma 中的偏移。 例如vma 的虚拟地址其实地址是a， 假设page 对应的虚拟地址是b。 那么b = a + 4096 * 62; 4096 是假设一个页面大小是4k
anon_vma = mapping - 1 = 0xffffffec1bcc38a1 -1 =0xffffffec1bcc38a0;
读取anon_vma 的信息
crash> struct anon_vma 0xffffffec1bcc38a0
struct anon_vma {
root = 0xffffffec1bcc38a0,
rwsem = {
count = {
counter = -8589934591
},
wait_list = {
next = 0xffffff80095b3928,
prev = 0xffffff801e24adc8
},
wait_lock = {
raw_lock = {
{
val = {
counter = 0
},
{
locked = 0 '\000',
pending = 0 '\000'
},
{
locked_pending = 0,
tail = 0
}
}
}
},
osq = {
tail = {
counter = 0
}
},
owner = 0xffffffeb44564ec0,
m_count = 0
},
refcount = {
counter = 57
},
degree = 51,
parent = 0xffffffec1bcc38a0,
rb_root = {
rb_root = {
rb_node = 0xffffffea87bbed60
},
rb_leftmost = 0xffffffec1d7a71a0
}
}anon_vma 相关的信息已经获得。
回到之前函数page_lock_anon_vma_read 查看 anon_vma_lock_read 获得 读写锁
static inline void anon_vma_lock_read(struct anon_vma *anon_vma)
{
down_read(&anon_vma->root->rwsem);
}可以看到是先指向root 的读写锁。 anon_vma 有个root 指针，指向当前anon_vma 的父anon_vma 。如果一个进程是被父任务创建，继承父的anon_vma 会指向父任务的anon_vma。
这里没有获取的anon_vma 的root 是指向自己，说明这个匿名页没有父anon_vma。 这个读写锁也就是自己的读写锁。
owner = 0xffffffeb44564ec0 指向写着的task
crash> struct task_struct.comm,pid,tpid 0xffffffeb44564ec0
comm = "ThreadPool_thre"
pid = 28624
struct: invalid data structure reference: task_struct.tpid
crash> struct task_struct.comm,pid,tgid 0xffffffeb44564ec0
comm = "ThreadPool_thre"
pid = 28624
tgid = 28350
crash>可以看到此任务是有品app创建的。 他的优先级是139 优先级非常低。 查看28624 的调用栈查看他在干嘛，为啥长时间持有写锁
crash> bt 28624
PID: 28624 TASK: ffffffeb44564ec0 CPU: 0 COMMAND: "ThreadPool_thre"
#0 [ffffff8027e1b780] __switch_to at ffffffa9a2e87a98
#1 [ffffff8027e1b810] __schedule at ffffffa9a424d924
#2 [ffffff8027e1b830] preempt_schedule_irq at ffffffa9a424df10
#3 [ffffff8027e1b970] el1_preempt at ffffffa9a2e82c6c
#4 [ffffff8027e1b9b0] anon_vma_interval_tree_remove at ffffffa9a3064ba0
#5 [ffffff8027e1ba10] unlink_anon_vmas at ffffffa9a3078918
#6 [ffffff8027e1ba60] free_pgtables at ffffffa9a3068e78
#7 [ffffff8027e1bb20] exit_mmap at ffffffa9a3075610
#8 [ffffff8027e1bb40] mmput at ffffffa9a2eb48e0
#9 [ffffff8027e1bbb0] do_exit at ffffffa9a2ebcb34
#10 [ffffff8027e1bbe0] do_group_exit at ffffffa9a2ebd288
#11 [ffffff8027e1bd00] get_signal at ffffffa9a2eca5b4
#12 [ffffff8027e1beb0] do_notify_resume at ffffffa9a2e8ccb0
#13 [ffffff8027e1bff0] work_pending at ffffffa9a2e8372c
PC: e947c898 LR: e944bac5 SP: a701cb48 PSTATE: 208b0010
X12: a701cb58 X11: ffffffff X10: 00000000 X9: adc99390
X8: adc993dc X7: 0000015a X6: ea2ae00c X5: 00000008
X4: 00000000 X3: ffffffff X2: 00000010 X1: a701cba8
X0: fffffffffffffffc
crash>可以看到进程是再被杀后退出，然后解除映射，这个时候持有了锁。unlink_anon_vmas 会编译任务的全部vma 然后解除全部的映射。
unlink_anon_vmas 会获得写锁，在这我们看到当前任务 有el1_preempt 函数，这意味着任务被抢占了。获得写锁后被切出去，一直没有回cpu运行。
这也和前面看到的有限机是139 非常低项符合。为了验证此问题，在查看cpu的运行等待队列，看看28624 这个任务是否一直在等待被调度上去。
crash> runq
CPU 0 RUNQUEUE: ffffffec3794cfc0
CURRENT: PID: 22277 TASK: ffffffebec926e40 COMMAND: "HeapTaskDaemon"
RT PRIO_ARRAY: ffffffec3794d1c0
[no tasks queued]
CFS RB_ROOT: ffffffec3794d078
[120] PID: 10484 TASK: ffffffea80bb3f00 COMMAND: "omi.bsp.gps.nps"
[139] PID: 8319 TASK: ffffffead05f1f80 COMMAND: "rameworkStarted"
[139] PID: 4218 TASK: ffffffea88561f80 COMMAND: "utdid:0"
[120] PID: 10455 TASK: ffffffeb35083f00 COMMAND: "Res_ResDownload"
[139] PID: 31022 TASK: ffffffeb78c5bf00 COMMAND: "ThreadPool_thre"
[130] PID: 10026 TASK: ffffffeacf8bde80 COMMAND: "AsyncTask #2"
[139] PID: 28624 TASK: ffffffeb44564ec0 COMMAND: "ThreadPool_thre" //在cpu等待队列中 28624 任务在cpu0 上等待被调度上去运行。
这里涉及到父子进程之前匿名页的 继承关系。 他们两个很可能都是由同一个父任务创建的。要判断出当前回收的page 是有品父任务的还是有品自己的。
crash> ps -p 28624
PID: 0 TASK: ffffffa9a561bdc0 CPU: 0 COMMAND: "swapper/0"
PID: 1 TASK: ffffffea46881f80 CPU: 4 COMMAND: "init"
PID: 673 TASK: ffffffec1ebbcec0 CPU: 4 COMMAND: "main"
PID: 28624 TASK: ffffffeb44564ec0 CPU: 0 COMMAND: "ThreadPool_thre"可以看到 ThreadPool_thre 是由 main 创建的。main 就是Zygote 任务。 android 的应用都是他创建的。
anon_vma 中有个红黑树，父任务创建的子任务都是拷贝父任务的vma 时都是创建一个avc 结构体，挂载到这个红黑树中，可以通过这个红黑树找到 anon_vma 对应的vma 的task 创建了多少子任务。
anon_vma 中也有
refcount = {
counter = 57
},
记录红黑树中有多少成员。前面获得anon_vma 的值时有红黑树的root 节点地址
rb_root = {
rb_root = {
rb_node = 0xffffffea87bbed60
},
rb_leftmost = 0xffffffec1d7a71a0
}crash> tree -t rbtree -o rb_root_cached.rb_root -N 0xffffffea87bbed60
ffffffea87bbed60
ffffffeb726d25e0
ffffffead4ac1720
ffffffeb7238c160
ffffffebbdc4f5e0
ffffffec1d7a71a0
ffffffec0a8beda0
ffffffea99f3f520
ffffffeb20871160
ffffffeacbc5d760
ffffffeb7c20c160
ffffffea65bc2d60
ffffffea87f82e20
ffffffea89cb6020
ffffffeb7ff08ce0
ffffffeb7a05ade0
ffffffeb728497e0
ffffffeb7afb8060
ffffffeb960e8e20
ffffffea80314620
ffffffebb535a320
ffffffead61162a0
ffffffebc6b96b60
ffffffeb8061ae60
ffffffeb75d93520
ffffffeb724a2b20
ffffffeb71cf5760
ffffffeaa04c7720
ffffffeae17eb660
ffffffeb759bae20
ffffffeb712c83a0
ffffffec1bfb20a0
ffffffeadb28c260
ffffffebb5f387a0
ffffffeb33713da0
ffffffead36e93e0
ffffffeaebadfa20
ffffffeb6dedb620
ffffffeac67373e0
ffffffeb43d7f6e0
ffffffea76aafa60
ffffffeac94f9020
ffffffebd7ae7460
ffffffeae524f0a0
ffffffea6defab20
ffffffead5948be0
ffffffea8fbe7fe0
ffffffea5a54cce0
ffffffebb3f00e60
ffffffeb71e1e1a0
ffffffeb75ed1ae0
ffffffeb75512460
ffffffea7152af60
ffffffebf081d1a0
ffffffead68e82a0
ffffffebaa1487a0
crash>查出来的个数和counter=57 相对应。查出来的的地址是anon_vma_chain对应的地址，结构体如下：
crash> whatis -o anon_vma_chain
struct anon_vma_chain {
[0] struct vm_area_struct *vma;
[8] struct anon_vma *anon_vma;
[16] struct list_head same_vma;
[32] struct rb_node rb;
[56] unsigned long rb_subtree_last;
}
SIZE: 64vma 指向了对应的vma 结构。 vma 又有指针指向了mm_struct , mm_struct有指针指向了task ，通过反推最后能找到是那个任务的anon_vma_chain。
vma 地址 = ffffffebaa1487A0 -0x20 = ffffffebaa148780
crash> struct anon_vma_chain ffffffebaa148780
struct anon_vma_chain {
vma = 0xffffffeb68cbc3c0,
anon_vma = 0xffffffec1bcc38a0,
same_vma = {
next = 0xffffffeb68cbc438,
prev = 0xffffffebaa1483d0
},
rb = {
__rb_parent_color = 18446743982819934880,
rb_right = 0x0,
rb_left = 0x0
},
rb_subtree_last = 77
}
crash> struct -x vm_area_struct 0xffffffeb68cbc3c0
struct vm_area_struct {
vm_start = 0x6f7c4000,
vm_end = 0x6f812000,
vm_next = 0xffffffeb68cbc900,
vm_prev = 0xffffffeb68cbc480,
vm_rb = {
__rb_parent_color = 0xffffffeb68cbc921,
rb_right = 0x0,
rb_left = 0x0
},
rb_subtree_gap = 0x0,
vm_mm = 0xffffffea803ee200,
vm_page_prot = {
pgprot = 0x60000000000fd3
},
vm_flags = 0x100073,
{
shared = {
rb = {
__rb_parent_color = 0xffffffec27db1a18,
rb_right = 0x0,
rb_left = 0x0
},
rb_subtree_last = 0x4d
},
anon_name = 0xffffffec27db1a18 "\231\230\377m\353\377\377\377\030\304\313h\353\377\377\377"
},
anon_vma_chain = {
next = 0xffffffebaa1483d0,
prev = 0xffffffebaa148790
},
anon_vma = 0xffffffeb67ac99c0,
vm_ops = 0xffffffa9a442d840,
vm_pgoff = 0x0,
vm_file = 0xffffffec1d4ce800,
vm_private_data = 0x0,
swap_readahead_info = {
counter = 0x6f7fb004
},
vm_userfaultfd_ctx = {},
vm_sequence = {
sequence = 0x0
},
vm_ref_count = {
counter = 0x1
}
}
crash> struct -x mm_struct 0xffffffea803ee200
struct mm_struct {
{
mmap = 0xffffffeb68cbc540,
mm_rb = {
rb_node = 0xffffffebadcc0ce0
},
vmacache_seqnum = 0x189,
mm_rb_lock = {
raw_lock = {
{
cnts = {
counter = 0x0
},
{
wlocked = 0x0,
__lstate = "\000\000"
}
},
wait_lock = {
{
val = {
counter = 0x0
},
{
locked = 0x0,
pending = 0x0
},
{
locked_pending = 0x0,
tail = 0x0
}
}
}
}
},
get_unmapped_area = 0xffffffa9a30742f8,
mmap_base = 0xeb2b1000,
mmap_legacy_base = 0x0,
task_size = 0xfffff000,
highest_vm_end = 0xffff1000,
pgd = 0xffffffeaa0ebd000,
mm_users = {
counter = 0x37
},
mm_count = {
counter = 0x2
},
pgtables_bytes = {
counter = 0x139000
},
map_count = 0x94a,
page_table_lock = {
{
rlock = {
raw_lock = {
{
val = {
counter = 0x0
},
{
locked = 0x0,
pending = 0x0
},
{
locked_pending = 0x0,
tail = 0x0
}
}
}
}
}
},
mmap_sem = {
count = {
counter = 0x0
},
wait_list = {
next = 0xffffffea803ee270,
prev = 0xffffffea803ee270
},
wait_lock = {
raw_lock = {
{
val = {
counter = 0x0
},
{
locked = 0x0,
pending = 0x0
},
{
locked_pending = 0x0,
tail = 0x0
}
}
}
},
osq = {
tail = {
counter = 0x0
}
},
owner = 0x1,
m_count = 0x0
},
mmlist = {
next = 0xffffffea803e8418,
prev = 0xffffffec27e07418
},
hiwater_rss = 0x657b,
hiwater_vm = 0x7e27b,
total_vm = 0x7e4ea,
locked_vm = 0x0,
pinned_vm = 0x0,
data_vm = 0x48aed,
exec_vm = 0x99ec,
stack_vm = 0x800,
def_flags = 0x0,
arg_lock = {
{
rlock = {
raw_lock = {
{
val = {
counter = 0x0
},
{
locked = 0x0,
pending = 0x0
},
{
locked_pending = 0x0,
tail = 0x0
}
}
}
}
}
},
start_code = 0x4ffb000,
end_code = 0x4fffd70,
start_data = 0x5000000,
end_data = 0x50002f4,
start_brk = 0x57f6000,
brk = 0x57f6000,
start_stack = 0xffc550f0,
arg_start = 0xffc556a5,
arg_end = 0xffc55712,
env_start = 0xffc55712,
env_end = 0xffc55fde,
saved_auxv = {0x37b0d600000010, 0x100000000006, 0x6400000011, 0x4ffb03400000003, 0x2000000004, 0xa00000005, 0xeb1e700000000007, 0x8, 0x4ffd00100000009, 0xb, 0xc, 0xd, 0xe, 0x100000017, 0xffc551ec00000019, 0x1f0000001a, 0xffc55fde0000001f, 0xffc551fc0000000f, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},
rss_stat = {
count = {{
counter = 0x4fcc
}, {
counter = 0x1547
}, {
counter = 0x11b4
}, {
counter = 0x49
}, {
counter = 0x0
}}
},
binfmt = 0xffffffa9a5659e08,
context = {
id = {
counter = 0x12324
},
vdso = 0xffff0000,
flags = 0x1
},
flags = 0x8d,
core_state = 0x0,
membarrier_state = {
counter = 0x32
},
ioctx_lock = {
{
rlock = {
raw_lock = {
{
val = {
counter = 0x0
},
{
locked = 0x0,
pending = 0x0
},
{
locked_pending = 0x0,
tail = 0x0
}
}
}
}
}
},
ioctx_table = 0x0,
owner = 0xffffffeb73878000,
user_ns = 0xffffffa9a562a2e8,
exe_file = 0xffffffec1e6ede00,
tlb_flush_pending = {
counter = 0x0
},
uprobes_state = {
xol_area = 0x0
},
async_put_work = {
data = {
counter = 0x0
},
entry = {
next = 0x0,
prev = 0x0
},
func = 0x0
}
},
cpu_bitmap = 0xffffffea803ee570
}
crash>
crash>
crash> struct task_struct.comm,pid,tgid,prio 0xffffffeb73878000
comm = ".smile.gifmaker"
pid = 10353
tgid = 10353
prio = 120 ".smile.gifmaker" 应该是快手的
crash> ps -p 10353
PID: 0 TASK: ffffffa9a561bdc0 CPU: 0 COMMAND: "swapper/0"
PID: 1 TASK: ffffffea46881f80 CPU: 4 COMMAND: "init"
PID: 673 TASK: ffffffec1ebbcec0 CPU: 4 COMMAND: "main"
PID: 10353 TASK: ffffffeb73878000 CPU: 5 COMMAND: ".smile.gifmaker"也是zygote 创建的。
多个这样查找发现这些任务都是zygote 创建的， 并且这个page 对应的anon_vma的 root 指针指向自己，也就是没有父的anon_vma，说明他就是zygote 创建的匿名页。
因为子会复制父任务的vma ，所以各个子任务中的到的vma 都是相同的
struct vm_area_struct {
vm_start = 0x6f7c4000,
vm_end = 0x6f812000,
vm_next = 0xffffffeb68cbc900,
vm_prev = 0xffffffeb68cbc480,
vm_rb = {
__rb_parent_color = 0xffffffeb68cbc921,
rb_right = 0x0,
rb_left = 0x0
},crash> vm -R 0x6f7c4000
PID: 673 TASK: ffffffec1ebbcec0 CPU: 4 COMMAND: "main"
MM PGD RSS TOTAL_VM
ffffffec27e07380 ffffffec1dd86000 36968k 1798172k
VMA START END FLAGS FILE
ffffffec27db19c0 6f7c4000 6f812000 100073 /system/framework/arm/boot-bouncycastle.art
VIRTUAL PHYSICAL
6f7c4000 252c52000FLAGS 是100073 表明是可读写。 由于在crash 写是没有办法看proc 下得smaps，在实际手机中查看下main 进程得smaps
70eca000-70f24000 rw-p 00000000 fc:00 2245 /system/framework/arm64/boot-bouncycastle.art
Size: 360 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Rss: 360 kB
Pss: 4 kB
Shared_Clean: 0 kB
Shared_Dirty: 360 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 360 kB
Anonymous: 360 kB
LazyFree: 0 kB
AnonHugePages: 0 kB
ShmemPmdMapped: 0 kB
Shared_Hugetlb: 0 kB
Private_Hugetlb: 0 kB
Swap: 0 kB
SwapPss: 0 kB
Locked: 0 kB
VmFlags: rd wr mr mw me lo ac可以看到存在匿名页，在回收这些页面时候，就会收到有品这类得app 获得读写锁导致卡顿