CPU占用率高问题定位

1、用top命令找到CPU利用率最高的进程
root@XXX:/# top
top - 09:36:26 up 22 min,  2 users,  load average: 6.02, 5.10, 3.58
Tasks: 43 total, 1 running, 42 sleeping, 0 stopped, 0 zombie
Cpu(s):  1.0%us, 12.6%sy,  0.0%ni, 86.4%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    514336k total,    53156k used,   461180k free,     6776k buffers
Swap:        0k total,        0k used,        0k free,    24072k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                               
1145 root       20  0  539m 6008 2940 S 65.9  1.2   2:40.77 test.out                                                              
6 root 15 -5 0 0 0 S 0.3 0.0 0:00.13 khelper
1327 root      20   0  3352 1372 1136 S  0.3  0.3   0:02.01 run                                                                   
12465 root      20   0  8432 2300 1932 S  0.3  0.4   0:00.07 sshd                                                                  
    1 root      20   0  2436  704  616 S  0.0  0.1   0:00.73 init                                                                  
    2 root      15  -5     0    0    0 S  0.0  0.0   0:00.00 kthreadd

2、找到对应进程占用CPU时间最长的线程
root@XXX:/#ps -efL | grep test.out | grep -v grep
UID        PID  PPID   LWP  C NLWP STIME TTY          TIME CMD
root      1145     1  1145  0   62 09:14 ?        00:00:03 /home/ligt/bin/test.out
root      1145     1  1148  0   62 09:14 ?        00:00:01 /home/ligt/bin/test.out
root      1145     1  1149  0   62 09:14 ?        00:00:03 /home/ligt/bin/test.out
root      1145     1  1151  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1152  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1153  0   62 09:14 ?        00:00:06 /home/ligt/bin/test.out
root      1145     1  1154  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1155  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1157  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1158  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1159  0   62 09:14 ?        00:00:06 /home/ligt/bin/test.out
root      1145     1  1160  0   62 09:14 ?        00:00:01 /home/ligt/bin/test.out
root      1145     1  1161  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1162 10   62 09:14 ?        00:02:41 /home/ligt/bin/test.out
root      1145     1  1163  0   62 09:14 ?        00:00:01 /home/ligt/bin/test.out
root      1145     1  1164  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1165  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1166  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1167  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1169  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1170  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1171  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1172  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1173  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1174  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1175  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1176  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1177  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1178  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1179  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1180  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1181  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1182  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1183  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1186  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1187  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1188  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1189  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1192  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1193  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1194  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1195  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1196  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1197  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1198  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1199  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1200  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1201  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1202  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1203  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1204  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1205  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1206  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1207  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
root      1145     1  1208  0   62 09:14 ?        00:00:00 /home/ligt/bin/test.out
   

3、   attach到对应的线程上
root@XXX:/#gdb attach 1162
GNU gdb (Wind River Linux Sourcery G++ 4.3a-274) 6.8.50.20080821-cvs
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "powerpc-wrs-linux-gnu".
For bug reporting instructions, please see:
<[email protected]>...
attach: No such file or directory.
Attaching to process 1162

warning: process 1162 is a cloned process
Reading symbols from /common/home/ligt/bin/test.out...done.
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /usr/lib/libmscm_auth.so...done.
Loaded symbols for /usr/lib/libmscm_auth.so
Reading symbols from /usr/lib/libmscm_ldap.so...done.
Loaded symbols for /usr/lib/libmscm_ldap.so
Reading symbols from /usr/lib/libmscm_cfgparse.so...done.
Loaded symbols for /usr/lib/libmscm_cfgparse.so
Reading symbols from /usr/lib/libldap-2.4.so.2...done.
Loaded symbols for /usr/lib/libldap-2.4.so.2
Reading symbols from /usr/lib/liblber-2.4.so.2...done.
Loaded symbols for /usr/lib/liblber-2.4.so.2
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libstdc++.so.6...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/libm.so.6...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libgcc_s.so.1...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libpthread.so.0...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/ld.so.1...done.
Loaded symbols for /lib/ld.so.1
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
0x0ff1dbf0 in select () from /lib/libc.so.6
(gdb) bt
#0  0x0ff1dbf0 in select () from /lib/libc.so.6
#1  0x101396cc in OSAL_TaskDelay (ulMillSecs=<value optimized out>)
    at /home/ligt/IPMI/src/vos/ipmi_osal_task.c:498
#2  0x101307a8 in VOS_TaskDelay (ulMillSecs=<value optimized out>) at /home/ligt/IPMI/src/vos/ipmi_v_task.c:3512
#3  0x10089ed4 in CIPMB_Processor::OnSend (this=0x103cc0a8) at /home/ligt/IPMI/src/ipmi/ipmbproc.cpp:3422
#4  0x10132db8 in VOS_TaskEntry (pulArg=0x1000d) at /home/ligt/IPMI/src/vos/ipmi_v_task.c:3459
#5  0x0fdf6d8c in ?? () from /lib/libpthread.so.0
#6  0x0ff25a90 in clone () from /lib/libc.so.6
(gdb) q
The program is running.  Quit anyway (and detach it)? (y or n) y
Detaching from program: /home/ligt/bin/test.out, process 1162        

  

通过以上三步分析，基本可以定位哪些代码造成CPU占用率高的问题，希望对遇到同样问题的朋友有所帮助。