使用pstack查看进程栈跟踪信息

pstack说明

• 在程序运行过程中，如果发生阻塞或者work状态，可以使用该命令显示进程的栈跟踪。
• 该命令必须由进程的属主或root运行。
• 使用该命令来确定进程挂起的位置。
• 该命令允许使用的唯一选项是要检查的进程的 PID，即运行格式为：pstack pid。

pstack应用

安装pstree工具

pstree以树结构显示进程，centos7上默认没有安装psmisc包。

# 在 Mac OS 上安装
brew install pstree

# 在 Fedora/Red Hat/CentOS 安装
yum -y install psmisc

# 在 Ubuntu/Debian 上安装
apt-get install psmisc

pstree使用

• 查看某个服务的线程信息

pstree -p work | grep redis

work为工作用户，-p为显示进程识别码，如下图所示：redis共启动了3个子线程，加上主线程共4个线程。

pstree -p redis | grep redis
redis-server(1190)-+-{redis-server}(1191)
                   |-{redis-server}(1192)
                   `-{redis-server}(1193)

• 查看某个进程的所有线程

[root@jolay ~]# ps -Lf 1190
UID        PID  PPID   LWP  C NLWP STIME TTY      STAT   TIME CMD
redis     1190     1  1190  1    4 May20 ?        Ssl  174:44 /home/q/redis/redis10010/bin/redis-server *:10010
redis     1190     1  1191  0    4 May20 ?        Ssl    0:00 /home/q/redis/redis10010/bin/redis-server *:10010
redis     1190     1  1192  0    4 May20 ?        Ssl    0:00 /home/q/redis/redis10010/bin/redis-server *:10010
redis     1190     1  1193  0    4 May20 ?        Ssl    0:00 /home/q/redis/redis10010/bin/redis-server *:10010

pstack使用

• 查看某个进程的pstack信息

[root@jolay ~]# pstack 1190
Thread 4 (Thread 0x7f836dfff700 (LWP 1191)):
#0  0x00007f836ed85965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000485b26 in bioProcessBackgroundJobs (arg=0x0) at bio.c:176
#2  0x00007f836ed81dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f836eaaaead in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7f836d7fe700 (LWP 1192)):
#0  0x00007f836ed85965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000485b26 in bioProcessBackgroundJobs (arg=0x1) at bio.c:176
#2  0x00007f836ed81dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f836eaaaead in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f836cffd700 (LWP 1193)):
#0  0x00007f836ed85965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000485b26 in bioProcessBackgroundJobs (arg=0x2) at bio.c:176
#2  0x00007f836ed81dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f836eaaaead in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f836f8b6f80 (LWP 1190)):
#0  0x00007f836eaab483 in epoll_wait () from /lib64/libc.so.6
#1  0x000000000042908e in aeApiPoll (tvp=, eventLoop=0x7f836e44c0a0) at ae_epoll.c:112
#2  aeProcessEvents (eventLoop=0x7f836e44c0a0, flags=11) at ae.c:411
#3  0x000000000042952b in aeMain (eventLoop=0x7f836e44c0a0) at ae.c:501
#4  0x000000000043184f in main (argc=, argv=) at server.c:4603

• 格式化查看pstack信息

1. 生成脚本

(
cat < ./pstack.py
chmod +x pstack.py

2. 执行脚本

[root@jolay ~]# ./pstack.py 1190
Count:  1
#0  0x00007f836ed85965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000485b26 in bioProcessBackgroundJobs (arg=0x1) at bio.c:176
#2  0x00007f836ed81dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f836eaaaead in clone () from /lib64/libc.so.6
-----------------------------------------------
Count:  1
#0  0x00007f836eaab483 in epoll_wait () from /lib64/libc.so.6
#1  0x000000000042908e in aeApiPoll (tvp=, eventLoop=0x7f836e44c0a0) at ae_epoll.c:112
#2  aeProcessEvents (eventLoop=0x7f836e44c0a0, flags=11) at ae.c:411
#3  0x000000000042952b in aeMain (eventLoop=0x7f836e44c0a0) at ae.c:501
#4  0x000000000043184f in main (argc=, argv=) at server.c:4603
-----------------------------------------------
Count:  1
#0  0x00007f836ed85965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000485b26 in bioProcessBackgroundJobs (arg=0x2) at bio.c:176
#2  0x00007f836ed81dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f836eaaaead in clone () from /lib64/libc.so.6
-----------------------------------------------
Count:  1
#0  0x00007f836ed85965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000485b26 in bioProcessBackgroundJobs (arg=0x0) at bio.c:176
#2  0x00007f836ed81dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f836eaaaead in clone () from /lib64/libc.so.6
-----------------------------------------------