ANR of app caused by native mediaserver
本文记述了由mediaserver引起的其它应用ANR的分析方法. 因为android的pthread_mutex使用BITS_NORMAL类型futex实现, 不能从futex本身和内核对象知道当前哪个进程拥有锁, 所以针对mediaserver这种native应用, 只能从线程组内各线程的调用栈来分析锁拥有线程链.
For android JellyBean release.
From Android system log
WARN [ 3078.755804] (564:631) BroadcastQueue Timeout of broadcast BroadcastRecord{438e0100 com.android.server.WifiManager.action.START_SCAN} -receiver=android.app.LoadedApk$ReceiverDispatcher$InnerReceiver@429a7950, started 60002ms ago
PROBABLE CAUSE OF PROBLEM:
Timeout
Receiver: android.app.LoadedApk$ReceiverDispatcher$InnerReceiver
.... Generating Dalvik backtraces. This might take some time ....
Receiver: might be pid 630
Pid 630 (ServerThread) could be interesting to investigated
***** Dalvik stack for pid 630 *****
#0 android.media.AudioService.handleDeviceConnection (AudioService.java:3523)
#1 android.media.AudioService.access$7100 (AudioService.java:110)
#2 android.media.AudioService$AudioServiceBroadcastReceiver.onReceive (AudioService.java:3725)
#3 android.app.LoadedApk$ReceiverDispatcher$Args.run (LoadedApk.java:765)
#4 android.os.Handler.handleCallback (Handler.java:615)
#5 android.os.Handler.dispatchMessage (Handler.java:94)
#6 android.os.Looper.loop (Looper.java:256)
#7 com.android.server.ServerThread.run (SystemServer.java:278)
-- Break frame --
============================================================================
The log says:
Receiver pid 630 has not resposed for the broadcast.
Pid 630 needs investigation.
Dump process 630 and use gdb to analyze it.
Thread ServerThread information and java call stack list as follows.
thread list len = 101
(mutexes: tll=0 tsl=0 tscl=0 ghl=0)
* 1 "android.server.ServerThread" prio=5 tid=12 MONITOR
| group="N/A" sCount=0 dsCount=0 obj=0x42755328 self=0x5e0fc008
| sysTid=630 nice=0 sched=0/0 cgrp= handle=0x5e0fc458
- waiting to lock <0x42bfe3a8> (a Ljava/util/HashMap;) held by tid=49 (AudioService)
#0 android.media.AudioService.handleDeviceConnection()
#1 android.media.AudioService.access$7100()
#2 android.media.AudioService$AudioServiceBroadcastReceiver.onReceive()
#3 android.app.LoadedApk$ReceiverDispatcher$Args.run()
#4 android.os.Handler.handleCallback()
#5 android.os.Handler.dispatchMessage()
#6 android.os.Looper.loop()
#7 com.android.server.ServerThread.run()
#8 --- break frame ---
ServerThread is waiting for a lock held by tid=49 (AudioService), thread 61.
Investigate source code of thread 61 to find out the synchronized lock.
handleDeviceConnection() @ AudioService.java
3521 private boolean handleDeviceConnection(boolean connected, int device, String params) {
3522 synchronized (mConnectedDevices) { ***************
3523 boolean isConnected = (mConnectedDevices.containsKey(device) &&
3524 (params.isEmpty() || mConnectedDevices.get(device).equals(params)));
3525
3526 if (isConnected && !connected) {
3527 AudioSystem.setDeviceConnectionState(device,
3528 AudioSystem.DEVICE_STATE_UNAVAILABLE,
3529 mConnectedDevices.get(device));
3530 mConnectedDevices.remove(device);
3531 return true;
3532 } else if (!isConnected && connected) {
3533 AudioSystem.setDeviceConnectionState(device,
3534 AudioSystem.DEVICE_STATE_AVAILABLE,
3535 params);
3536 mConnectedDevices.put(new Integer(device), params);
3537 return true;
3538 }
3539 }
3540 return false;
3541 }
Thread AudioService's java call stack lists as follows.
[Switching to thread 61 (LWP 805)]
#0 __ioctl () at bionic/libc/arch-arm/syscalls/__ioctl.S:10
10 bionic/libc/arch-arm/syscalls/__ioctl.S: No such file or directory.
#0 android.media.AudioSystem.setDeviceConnectionState()
#1 android.media.AudioService.makeA2dpDeviceUnavailableNow()
#2 android.media.AudioService.onSetA2dpConnectionState()
#3 android.media.AudioService.access$6500()
#4 android.media.AudioService$AudioHandler.handleMessage()
#5 android.os.Handler.dispatchMessage()
#6 android.os.Looper.loop()
#7 android.media.AudioService$AudioSystemThread.run()
#8 --- break frame ---
It can be found that the thread is in AudioSystem::setDeviceConnectionState() method, holding the lock of mConnectedDevices and waiting binder communication respose.
The related source code,
onSetA2dpConnectionState() @ AudioService.java
3462 private void onSetA2dpConnectionState(BluetoothDevice btDevice, int state)
3463 {
3464 if (btDevice == null) {
3465 return;
3466 }
3467 String address = btDevice.getAddress();
3468 if (!BluetoothAdapter.checkBluetoothAddress(address)) {
3469 address = "";
3470 }
3471 synchronized (mConnectedDevices) { ************
3472 boolean isConnected =
3473 (mConnectedDevices.containsKey(AudioSystem.DEVICE_OUT_BLUETOOTH_A2DP) &&
3474 mConnectedDevices.get(AudioSystem.DEVICE_OUT_BLUETOOTH_A2DP).equals(address));
3475
3476 if (isConnected && state != BluetoothProfile.STATE_CONNECTED) {
3477 if (btDevice.isBluetoothDock()) {
3478 if (state == BluetoothProfile.STATE_DISCONNECTED) {
3479 // introduction of a delay for transient disconnections of docks when
3480 // power is rapidly turned off/on, this message will be canceled if
3481 // we reconnect the dock under a preset delay
3482 makeA2dpDeviceUnavailableLater(address);
3483 // the next time isConnected is evaluated, it will be false for the dock
3484 }
3485 } else {
3486 makeA2dpDeviceUnavailableNow(address);
3487 }
3488 synchronized (mCurAudioRoutes) {
3489 if (mCurAudioRoutes.mBluetoothName != null) {
3490 mCurAudioRoutes.mBluetoothName = null;
3491 sendMsg(mAudioHandler, MSG_REPORT_NEW_ROUTES,
3492 SENDMSG_NOOP, 0, 0, null, 0);
3493 }
3494 }
3495 } else if (!isConnected && state == BluetoothProfile.STATE_CONNECTED) {
3496 if (btDevice.isBluetoothDock()) {
3497 // this could be a reconnection after a transient disconnection
3498 cancelA2dpDeviceTimeout();
3499 mDockAddress = address;
3500 } else {
3501 // this could be a connection of another A2DP device before the timeout of
3502 // a dock: cancel the dock timeout, and make the dock unavailable now
3503 if(hasScheduledA2dpDockTimeout()) {
3504 cancelA2dpDeviceTimeout();
3505 makeA2dpDeviceUnavailableNow(mDockAddress);
3506 }
3507 }
3508 makeA2dpDeviceAvailable(address);
3509 synchronized (mCurAudioRoutes) {
3510 String name = btDevice.getAliasName();
3511 if (!TextUtils.equals(mCurAudioRoutes.mBluetoothName, name)) {
3512 mCurAudioRoutes.mBluetoothName = name;
3513 sendMsg(mAudioHandler, MSG_REPORT_NEW_ROUTES,
3514 SENDMSG_NOOP, 0, 0, null, 0);
3515 }
3516 }
3517 }
3518 }
3519 }
makeA2dpDeviceUnavailableNow() @ AudioService.java
3431 // must be called synchronized on mConnectedDevices
3432 private void makeA2dpDeviceUnavailableNow(String address) {
3433 AudioSystem.setDeviceConnectionState(AudioSystem.DEVICE_OUT_BLUETOOTH_A2DP, ********
3434 AudioSystem.DEVICE_STATE_UNAVAILABLE,
3435 address);
3436 mConnectedDevices.remove(AudioSystem.DEVICE_OUT_BLUETOOTH_A2DP);
3437 }
Call the following cpp layer via native method.
setDeviceConnectionState() @ AudioSystem.cpp
574status_t AudioSystem::setDeviceConnectionState(audio_devices_t device,
575 audio_policy_dev_state_t state,
576 const char *device_address)
577{
578 const sp<IAudioPolicyService>& aps = AudioSystem::get_audio_policy_service();
579 const char *address = "";
580
581 if (aps == 0) return PERMISSION_DENIED;
582
583 if (device_address != NULL) {
584 address = device_address;
585 }
586
587 return aps->setDeviceConnectionState(device, state, address); **********
588}
It is waiting on blocking binder io,
Examine thread AudioService's native call stack. .
#0 __ioctl () at bionic/libc/arch-arm/syscalls/__ioctl.S:10
#1 0x4028b030 in ioctl (fd=<optimized out>, request=-1072143871) at bionic/libc/bionic/ioctl.c:41
#2 0x403f6c08 in android::IPCThreadState::talkWithDriver (this=0x5cfe6a68, doReceive=<optimized out>) at frameworks/native/libs/binder/IPCThreadState.cpp:817
#3 0x403f7134 in android::IPCThreadState::waitForResponse (this=0x5cfe6a68, reply=0x5f3fac08, acquireResult=0x0) at frameworks/native/libs/binder/IPCThreadState.cpp:679
#4 0x403f734e in android::IPCThreadState::transact (this=0x5cfe6a68, handle=12, code=1, data=..., reply=0x5f3fac08, flags=16)
at frameworks/native/libs/binder/IPCThreadState.cpp:570
#5 0x403f4aa6 in android::BpBinder::transact (this=0x5e00d958, code=1, data=..., reply=0x5f3fac08, flags=0) at frameworks/native/libs/binder/BpBinder.cpp:165
#6 0x4118c040 in android::BpAudioPolicyService::setDeviceConnectionState (this=<optimized out>, device=AUDIO_DEVICE_OUT_BLUETOOTH_A2DP,
state=AUDIO_POLICY_DEVICE_STATE_UNAVAILABLE, device_address=0x5cfdefc0 "00:18:13:E0:D1:E3") at frameworks/av/media/libmedia/IAudioPolicyService.cpp:87
#7 0x41180bca in android::AudioSystem::setDeviceConnectionState (device=AUDIO_DEVICE_OUT_BLUETOOTH_A2DP, state=AUDIO_POLICY_DEVICE_STATE_UNAVAILABLE,
device_address=<optimized out>) at frameworks/av/media/libmedia/AudioSystem.cpp:577
#8 0x404c67d2 in android_media_AudioSystem_setDeviceConnectionState (env=<optimized out>, thiz=<optimized out>, device=128, state=0, device_address=0x26500005)
at frameworks/base/core/jni/android_media_AudioSystem.cpp:154
#9 0x40e0e3f4 in dvmPlatformInvoke () at dalvik/vm/arch/arm/CallEABI.S:258
#10 0x40e3dbb2 in dvmCallJNIMethod (args=0x40f3de38, pResult=0x5ca26060, method=0x574408d8, self=0x5ca26050) at dalvik/vm/Jni.cpp:1184
#11 0x40e178a4 in dalvik_mterp () at dalvik/vm/mterp/out/InterpAsm-armv7-a-neon.S:16311
#12 0x40e1c40c in dvmInterpret (self=0x5ca26050, method=<optimized out>, pResult=0x5f3faeb0) at dalvik/vm/interp/Interp.cpp:1964
#13 0x40e50588 in dvmCallMethodV (self=0x5ca26050, method=0x57734dd0, obj=<optimized out>, fromJni=<optimized out>, pResult=0x5f3faeb0, args=...)
at dalvik/vm/interp/Stack.cpp:526
#14 0x40e505b2 in dvmCallMethod (self=<optimized out>, method=<optimized out>, obj=<optimized out>, pResult=0x5f3faeb0) at dalvik/vm/interp/Stack.cpp:429
#15 0x40e4516e in interpThreadStart (arg=0x5ca26050) at dalvik/vm/Thread.cpp:1543
#16 0x40275eb4 in __thread_entry (func=0x40e450cd <interpThreadStart(void*)>, arg=0x5ca26050, tls=0x5f3faf00) at bionic/libc/bionic/pthread.c:218
#17 0x4027560c in pthread_create (thread_out=0x5ca264a0, attr=0x5eefad70, start_routine=0x40e450cd <interpThreadStart(void*)>, arg=0x5ca26050)
at bionic/libc/bionic/pthread.c:357
#18 0x00000000 in ?? ()
Switch to AudioService(pid 805) to examine the kernel stack to check binder communication, especially to get target binder thread.
State of thread 805
PID: 805
COMMAND: "AudioService"
TASK: dcf70000 [THREAD_INFO: dcf4a000]
CPU: 0
STATE: TASK_INTERRUPTIBLE
AudioSerive is schedualed out for waiting binder reply.
Kernel stack for pid 805
#0 [INLINE] context_switch
kernel/kernel/sched/core.c:2086
next (task_struct *) = 0xD3557600
prev (task_struct *) = 0xDCF70000
rq (rq *) = 0xC244A5C0
block
mm (mm_struct *) = 0xEE3D8380
oldmm (mm_struct *) = 0xEE3D8380
#1 0xC06EFC94 in __schedule+0x06A8(+1704)
kernel/kernel/sched/core.c:3233
prev (task_struct *) = 0xDCF70000
next (task_struct *) = 0xD3557600
switch_count (long unsigned int *) = <optimized out>
rq (rq *) = 0xC244A5C0
cpu (int) = <optimized out>
#2 0xC04F9814 in binder_thread_read+0x034C(+844)
kernel/drivers/staging/android/binder.c:2281
proc (binder_proc *) = 0xEDB55C00
thread (binder_thread *) = 0xD43FEB00
buffer (? *) = 0xEE360F20
size (int) = <optimized out>
consumed (long int *) = 0xDCF4BEF8
non_block (int) = 0
ptr (? *) = 0x40CD25E4
end (? *) = 0xDCF4BE7C
ret (int) = <unknown>
wait_for_proc_work (int) = 1087186400
block
__ret (int) = <unknown>
block
__wait (__wait_queue) = {
flags = 0,
private = 0xDCF70000,
func = 0xC00978DC,
task_list = {
next = 0xD43FEB30,
prev = 0xD43FEB30}}
#3 0xC04FA28C in binder_ioctl+0x0220(+544)
kernel/drivers/staging/android/binder.c:2695
filp (file *) = 0xEDB94C00
cmd (unsigned int) = 0xC0186201
arg (long unsigned int) = 0x5F3FAAB0
ret (int) = <optimized out>
proc (binder_proc *) = 0xEDB55C00
thread (binder_thread *) = 0xD43FEB00
size (unsigned int) = <unknown>
ubuf (? *) = 0x5F3FAAB0
block
bwr (binder_write_read) = {
write_size = 0,
write_consumed = 0,
write_buffer = 0x5DB88828,
read_size = 256,
read_consumed = 0,
read_buffer = 0x40CD25E0}
#4 0xC013CDF8 in do_vfs_ioctl+0x04D8(+1240)
kernel/fs/ioctl.c:43
filp (file *) = 0xEDB94C00
fd (unsigned int) = <optimized out>
cmd (unsigned int) = <optimized out>
arg (long unsigned int) = 0x5F3FAAB0
error (int) = <unknown>
argp (int *) = 0x5F3FAAB0
inode (inode *) = 0xEE2A8D10
#5 0xC013CEA0 in sys_ioctl+0x0034(+52)
kernel/fs/ioctl.c:618
fd (unsigned int) = 9
cmd (unsigned int) = 0xC0186201
arg (long unsigned int) = <optimized out>
filp (file *) = 0xEDB94C00
error (int) = <optimized out>
fput_needed (int) = -1072443744
#6 System call exception frame.
Registers from userland:
r00=0x00000009 r01=0xC0186201 r02=0x5F3FAAB0 r03=0x5F3FAAAC
r04=0x5CFE6A98 r05=0x5CFE6A68 r06=0x5CFE6AC8 r07=0x00000036
r08=0x00000001 r09=0x00007206 r10=0x00007211 r11=0x402B3A8C
r12=0x40402F24 r13=0x5F3FAA90 r14=0x4028B031 r15=0x4026FBD4
Use the binder_transaction to get the target PID
(crash)> p ((struct binder_thread*)0xD43FEB00)->transaction_stack
$9 = (struct binder_transaction *) 0xe1051f40
The target (proc, thread) is (0xee24b800, 0xed88c400). The whole struct binder_transaction is at address 0xe1051f40
struct binder_transaction {
debug_id = 387572,
work = {
entry = {
next = 0x100100,
prev = 0x200200
},
type = BINDER_WORK_TRANSACTION
},
from = 0xd43feb00,
from_parent = 0x0,
to_proc = 0xee24b800,
to_thread = 0xed88c400,
to_parent = 0x0,
need_reply = 1,
buffer = 0xf49001fc,
code = 1,
flags = 16,
priority = 0,
saved_priority = 0,
sender_euid = 1000
}
(crash)> p ((struct binder_proc*)0xee24b800)->pid
$10 = 214
(crash)> p ((struct binder_thread*)0xed88c400)->pid
$11 = 214
The binder target thread is pid 214, which is mediaserver.
Info of thread 214
PID PPID CPU TASK ST %MEM VSZ RSS COMM
214 1 0 eeefb600 IN 1.5 61956 12772 mediaserver
The thread group of pid 214.
PID: 214 TASK: eeefb600 CPU: 0 COMMAND: "mediaserver"
PID: 553 TASK: ee347180 CPU: 0 COMMAND: "AudioCommand"
PID: 554 TASK: ee347600 CPU: 0 COMMAND: "ApmCommand"
PID: 555 TASK: ee344000 CPU: 0 COMMAND: "mediaserver"
PID: 556 TASK: ee345200 CPU: 0 COMMAND: "FastMixer"
PID: 638 TASK: e9bdf600 CPU: 0 COMMAND: "AudioOut_2"
PID: 639 TASK: e9bf1f80 CPU: 0 COMMAND: "Binder_1"
PID: 1253 TASK: ce50fa80 CPU: 0 COMMAND: "Binder_2"
PID: 1611 TASK: e6508d80 CPU: 0 COMMAND: "FastMixer"
PID: 5972 TASK: e6509680 CPU: 0 COMMAND: "Binder_3"
PID: 6015 TASK: cc319f80 CPU: 0 COMMAND: "Binder_4"
PID: 6016 TASK: cc318480 CPU: 0 COMMAND: "Binder_5"
PID: 6289 TASK: e9be1200 CPU: 0 COMMAND: "Binder_6"
PID: 6290 TASK: ca553600 CPU: 0 COMMAND: "Binder_7"
PID: 6539 TASK: ca981b00 CPU: 0 COMMAND: "Binder_8"
PID: 6540 TASK: ca553a80 CPU: 0 COMMAND: "Binder_9"
PID: 6696 TASK: c2d18900 CPU: 0 COMMAND: "Binder_A"
PID: 6697 TASK: dcc03600 CPU: 0 COMMAND: "Binder_B"
PID: 6707 TASK: dcc03180 CPU: 0 COMMAND: "Binder_C"
State of Thread 214,
PID: 214
COMMAND: "mediaserver"
TASK: eeefb600 [THREAD_INFO: edf28000]
CPU: 0
STATE: TASK_INTERRUPTIBLE
The following mediaserver call stack both in user space and kernel space show that it is waiting to lock mutex.
Userland stack for pid 214
#0 0x4030FCDC in __futex_syscall3 /system/lib/libc.so
bionic/libc/arch-arm/bionic/futex_arm.S:59
size of frame = 8
reg[7] = 0x00000002 from 0xBEA6599C
reg[4] = 0x402A04E4 from 0xBEA65998
#1 [INLINE] __bionic_swap
bionic/libc/bionic/pthread.c:1050
ptr (int *) = <optimized out>
new_value (int) = <optimized out>
block
prev (int) = <optimized out>
status (int) = <optimized out>
#2 [INLINE] _normal_lock
bionic/libc/private/bionic_atomic_arm.h:191
shared (int) = 0
mutex (pthread_mutex_t *) = 0x402A04E4
block
unlocked (int) = 0
locked_uncontended (int) = <optimized out>
block
locked_contended (int) = 2
#3 0x40314210 in pthread_mutex_lock_impl+0x0050(+80) /system/lib/libc.so
bionic/libc/bionic/pthread.c:1190
mutex (pthread_mutex_t *) = 0x402A04E4
mvalue (int) = <optimized out>
mtype (int) = <optimized out>
tid (int) = <optimized out>
new_lock_type (int) = <optimized out>
shared (int) = 0
#4 0x402EE4C4 in Autolock+0x000C(+12) /system/lib/libaudioflinger.so
frameworks/native/include/utils/Mutex.h:112
this (Autolock *) = 0xBEA659C4
mutex (?) = <optimized out>
#5 0x402EE726 in android::AudioPolicyService::setDeviceConnectionState+0x0038(+56) /system/lib/libaudioflinger.so
frameworks/av/services/audioflinger/AudioPolicyService.cpp:161
this (AudioPolicyService *) = 0x402A04D0
device (audio_devices_t) = AUDIO_DEVICE_OUT_BLUETOOTH_A2DP (128)
state (audio_policy_dev_state_t) = AUDIO_POLICY_DEVICE_STATE_UNAVAILABLE (0)
device_address (char *) = 0x40FA1278 "00:18:13:E0:D1:E3"
block
_l (Autolock) = {
mLock = }
#6 0x406D31CA in android::BnAudioPolicyService::onTransact+0x018E(+398) /system/lib/libmedia.so
frameworks/av/media/libmedia/IAudioPolicyService.cpp:480
this (? *) = 0x402A04D0
code (unsigned int) = <optimized out>
data (?) = <optimized out>
reply (? *) = 0xBEA65B4C
flags (unsigned int) = 0x10
block
output (int) = <optimized out>
stream (unsigned int) = <optimized out>
session (int) = <optimized out>
#7 0x404883CA in android::BBinder::transact+0x003E(+62) /system/lib/libbinder.so
frameworks/native/libs/binder/Binder.cpp:108
this (? *) = 0x402A04D4
code (unsigned int) = 1
data (?) =
reply (? *) = 0xBEA65B4C
flags (unsigned int) = 0x10
block
err (int) = <unknown>
#8 0x4048AFC6 in android::IPCThreadState::executeCommand+0x020A(+522) /system/lib/libbinder.so
frameworks/native/libs/binder/IPCThreadState.cpp:1044
this (IPCThreadState *) = 0x40586298
cmd (int) = <optimized out>
block
obj (BBinder *) = <optimized out>
refs (weakref_type *) = <optimized out>
result (int) = 0
block
tr (?) = {
target = {
handle = 0x402A0548,
ptr = 0x402A0548},
cookie = 0x402A04D4,
code = 1,
flags = 0x10,
sender_pid = 564,
sender_euid = 0x3E8,
data_size = 0x68,
offsets_size = 0,
data = {
ptr = {
buffer = 0x40FA1224,
offsets = 0x40FA128C},
buf = {Ox24, Ox12, OxFA, Ox40, Ox8C, Ox12, OxFA, Ox40}}}
buffer (?) = {
mError = 0,
mData = 0x40FA1224,
mDataSize = 0x68,
mDataCapacity = 0x68,
mDataPos = 0x68,
mObjects = 0x40FA128C,
mObjectsSize = 0,
mObjectsCapacity = 0,
mNextObjectHint = 0,
mFdsKnown = 1,
mHasFds = 0,
mAllowFds = 1,
mOwner = 0x4048AAB1,
mOwnerCookie = 0x40586298}
origPid (int) = 214
origUid (unsigned int) = 0x3F5
curPrio (int) = <optimized out>
reply (?) = {
mError = 0,
mData = 0x0,
mDataSize = 0,
mDataCapacity = 0,
mDataPos = 0,
mObjects = 0x0,
mObjectsSize = 0,
mObjectsCapacity = 0,
mNextObjectHint = 0,
mFdsKnown = 1,
mHasFds = 0,
mAllowFds = 1,
mOwner = 0x0,
mOwnerCookie = 0x40352A8C}
block
b (?) = {
m_ptr = 0x402A04D4}
error (int) = <optimized out>
#9 0x4048B418 in android::IPCThreadState::joinThreadPool+0x00BC(+188) /system/lib/libbinder.so
frameworks/native/libs/binder/IPCThreadState.cpp:478
this (IPCThreadState *) = 0x40586298
isMain (bool) = <optimized out>
block
result (int) = 0
block
cmd (int) = <optimized out>
block
IN (unsigned int) = <optimized out>
Kernel stack for pid 214
#0 [INLINE] context_switch
kernel/kernel/sched/core.c:2086
next (task_struct *) = 0xEDD2C000
prev (task_struct *) = 0xEEEFB600
rq (rq *) = 0xC244A5C0
block
mm (mm_struct *) = 0xED81C700
oldmm (mm_struct *) = 0xED81CFC0
#1 0xC06EFC94 in __schedule+0x06A8(+1704)
kernel/kernel/sched/core.c:3233
prev (task_struct *) = 0xEEEFB600
next (task_struct *) = 0xEDD2C000
switch_count (long unsigned int *) = <optimized out>
rq (rq *) = 0xC244A5C0
cpu (int) = <optimized out>
#2 [INLINE] current_thread_info
kernel/kernel/futex.c:1788
block
sp (long unsigned int) = 0xEDF29D50
#3 [INLINE] get_current
kernel/arch/arm/include/asm/thread_info.h:97
#4 0xC00BA008 in futex_wait_queue_me+0x00DC(+220)
kernel/kernel/futex.c:1789
hb (futex_hash_bucket *) = <optimized out>
q (futex_q *) = 0xEDF29E28 ***************
timeout (hrtimer_sleeper *) = 0x0
#5 [INLINE] unqueue_me
kernel/kernel/futex.c:1904
q (futex_q *) = <unknown>
block
lock_ptr (spinlock *) = <optimized out>
ret (int) = <unknown>
#6 0xC00BAB68 in futex_wait+0x00F0(+240)
kernel/kernel/futex.c:1904
uaddr (unsigned int *) = 0x402A04E4
flags (unsigned int) = 0
val (unsigned int) = 2
abs_time (ktime *) = 0x0
bitset (unsigned int) = 0xFFFFFFFF
timeout (hrtimer_sleeper) = {
timer = {
node = {
node = {
rb_parent_color = 0xEDF28000,
rb_right = 0x00010000,
rb_left = 0xEDF28000},
expires = {
tv64 = 3992092672}},
_softexpires = {
tv64 = -1280623711687114752},
function = 0x0,
base = 0xC008B6D8,
state = 0,
start_pid = -298169728,
start_site = 0xEE3AB680,
start_comm = {'\0', '\0', '\0', '\0', '\260', '\237', '\362', '\355', '\10', '\0', '\0', '\0', '\354', '\236', '\362', '\355'}},
task = 0x40FA10C8}
to (hrtimer_sleeper *) = 0x0
restart (restart_block *) = <optimized out>
hb (futex_hash_bucket *) = 0xEE24B800 **************
q (futex_q) = { **************
list = {
prio = 100,
prio_list = {
next = 0xEDF29E2C,
prev = 0xEDF29E2C},
node_list = {
next = 0xC0CE44A8,
prev = 0xC2CE3E34}},
task = 0xEEEFB600,
lock_ptr = 0xC0CE44A4,
key = {
shared = {
pgoff = 0x402A0000,
inode = 0xED81CFC0,
offset = 1252},
private = {
address = 0x402A0000,
mm = 0xED81CFC0,
offset = 1252},
both = {
word = 0x402A0000,
ptr = 0xED81CFC0,
offset = 1252}},
pi_state = 0x0,
rt_waiter = 0x0,
requeue_pi_key = 0x0,
bitset = 0xFFFFFFFF}
ret (int) = <unknown>
#7 0xC00BC018 in do_futex+0x00C4(+196)
kernel/kernel/futex.c:2649
uaddr (unsigned int *) = 0x402A04E4
op (int) = <optimized out>
val (unsigned int) = 2
timeout (ktime *) = <optimized out>
uaddr2 (unsigned int *) = 0x402A04E4
val2 (unsigned int) = 0
val3 (unsigned int) = 0x402A04E4
cmd (int) = <optimized out>
flags (unsigned int) = 0
#8 0xC00BCA1C in sys_futex+0x0140(+320)
kernel/kernel/futex.c:2707
uaddr (unsigned int *) = 0x402A04E4
op (int) = 128
val (unsigned int) = 2
utime (timespec *) = <optimized out>
uaddr2 (unsigned int *) = 0x402A04E4
val3 (unsigned int) = 0
ts (timespec) = {
tv_sec = 0,
tv_nsec = 1}
t (ktime) = {
tv64 = -1300836601647013784}
tp (ktime *) = <optimized out>
val2 (unsigned int) = 0
cmd (int) = 0
#9 System call exception frame.
Registers from userland:
r00=0x402A04E4 r01=0x00000080 r02=0x00000002 r03=0x00000000
r04=0x402A04E4 r05=0x00000000 r06=0x00000002 r07=0x000000F0
r08=0x00000080 r09=0x000000D6 r10=0x40352A8C r11=0x00000000
r12=0x00000000 r13=0xBEA65998 r14=0x40314210 r15=0x4030FCDC
There are other threads in the mediaserver that is waiting for this lock.
Using the following command to find which threads are waiting on one lock.
(crash)> locks -b | grep 214
0xC0CE44A4: 3944 4076 4674 825 1162 1253 6697 214
No many, main and Binder_2 and Binder_B are waiting on it.
The processes waiting on the hashed futex bucket are
(crash)> list -o 12 -s futex_q.task -H 0xC0CE44A8
cb019e28
task = 0xcaa7f600
ed233e28
task = 0xea7a0000
cc037e28
task = 0xcc319b00
ecf35e28
task = 0xedfa1200
cdf03e28
task = 0xd1ed5f80
caa81e28
task = 0xce50fa80
c2ce3e28
task = 0xdcc03600
edf29e28
task = 0xeeefb600
For futex bits_normal, no simple way to know which thread is holding the lock/mutex, we have to check bt of each thread to look up which thread is in execution of AudioPolicyService::method()!!!
(crash)> ps -g 214
PID: 214 TASK: eeefb600 CPU: 0 COMMAND: "mediaserver"
PID: 553 TASK: ee347180 CPU: 0 COMMAND: "AudioCommand"
PID: 554 TASK: ee347600 CPU: 0 COMMAND: "ApmCommand"
PID: 555 TASK: ee344000 CPU: 0 COMMAND: "mediaserver"
PID: 556 TASK: ee345200 CPU: 0 COMMAND: "FastMixer"
PID: 638 TASK: e9bdf600 CPU: 0 COMMAND: "AudioOut_2"
PID: 639 TASK: e9bf1f80 CPU: 0 COMMAND: "Binder_1"
PID: 1253 TASK: ce50fa80 CPU: 0 COMMAND: "Binder_2"
PID: 1611 TASK: e6508d80 CPU: 0 COMMAND: "FastMixer"
PID: 5972 TASK: e6509680 CPU: 0 COMMAND: "Binder_3"
PID: 6015 TASK: cc319f80 CPU: 0 COMMAND: "Binder_4"
PID: 6016 TASK: cc318480 CPU: 0 COMMAND: "Binder_5"
PID: 6289 TASK: e9be1200 CPU: 0 COMMAND: "Binder_6"
PID: 6290 TASK: ca553600 CPU: 0 COMMAND: "Binder_7"
PID: 6539 TASK: ca981b00 CPU: 0 COMMAND: "Binder_8"
PID: 6540 TASK: ca553a80 CPU: 0 COMMAND: "Binder_9"
PID: 6696 TASK: c2d18900 CPU: 0 COMMAND: "Binder_A"
PID: 6697 TASK: dcc03600 CPU: 0 COMMAND: "Binder_B"
PID: 6707 TASK: dcc03180 CPU: 0 COMMAND: "Binder_C"
(gdb) info threads
Id Target Id Frame
19 LWP 6707 __ioctl () at bionic/libc/arch-arm/syscalls/__ioctl.S:10
18 LWP 6697 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
17 LWP 6696 __ioctl () at bionic/libc/arch-arm/syscalls/__ioctl.S:10
16 LWP 6540 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
15 LWP 6539 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
14 LWP 6290 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
13 LWP 6289 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
12 LWP 6016 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
11 LWP 6015 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
10 LWP 5972 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
9 LWP 1611 0xffff0520 in ?? ()
8 LWP 1253 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
7 LWP 639 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
6 LWP 638 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
5 LWP 556 nanosleep () at bionic/libc/arch-arm/syscalls/nanosleep.S:10
4 LWP 555 read () at bionic/libc/arch-arm/syscalls/read.S:10
3 LWP 554 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
2 LWP 553 0xffff0520 in ?? ()
* 1 LWP 214 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
For lock trace, the object interested:
ThreadBase obj 0x41851008;
PlaybackThread obj 0x41851008; Another obj is not concerned
MixerThread obj 0x41851008;
(gdb) bt
#0 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
#1 0x40314210 in _normal_lock (shared=0, mutex=0x402a04e4) at bionic/libc/bionic/pthread.c:1069
#2 pthread_mutex_lock_impl (mutex=0x402a04e4) at bionic/libc/bionic/pthread.c:1191
#3 0x402ee4c4 in lock (this=<optimized out>) at frameworks/native/include/utils/Mutex.h:112
#4 android::Mutex::Autolock::Autolock (this=0xbea659c4, mutex=...) at frameworks/native/include/utils/Mutex.h:65
#5 0x402ee726 in android::AudioPolicyService::setDeviceConnectionState (this=0x402a04d0, device=AUDIO_DEVICE_OUT_BLUETOOTH_A2DP,
state=AUDIO_POLICY_DEVICE_STATE_UNAVAILABLE, device_address=0x40fa1278 "00:18:13:E0:D1:E3") at frameworks/av/services/audioflinger/AudioPolicyService.cpp:161
#6 0x406d31ca in android::BnAudioPolicyService::onTransact (this=0x402a04d0, code=<optimized out>, data=..., reply=0xbea65b4c, flags=16)
at frameworks/av/media/libmedia/IAudioPolicyService.cpp:480
#7 0x404883ca in android::BBinder::transact (this=0x402a04d4, code=1, data=..., reply=0xbea65b4c, flags=16) at frameworks/native/libs/binder/Binder.cpp:108
#8 0x4048afc6 in android::IPCThreadState::executeCommand (this=0x40586298, cmd=<optimized out>) at frameworks/native/libs/binder/IPCThreadState.cpp:1044
#9 0x4048b418 in android::IPCThreadState::joinThreadPool (this=0x40586298, isMain=<optimized out>) at frameworks/native/libs/binder/IPCThreadState.cpp:478
#10 0x400e6d2a in main (argc=<optimized out>, argv=<optimized out>) at frameworks/av/media/mediaserver/main_mediaserver.cpp:73
Wait on mutex=0x402a04e4.
The lock is held by Thread 639 taken in function AudioPolicyManager::setPhoneState.
And thread 639 is waiting for the parametersCommand to be completed.
(gdb) thread 7
[Switching to thread 7 (LWP 639)]
#0 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
59 in bionic/libc/arch-arm/bionic/futex_arm.S
(gdb) bt
#0 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
#1 0x40314ef8 in __pthread_cond_timedwait_relative (cond=0x419df358, mutex=0x402a06ac, reltime=0x0) at bionic/libc/bionic/pthread.c:1714
#2 0x40314f54 in __pthread_cond_timedwait (cond=0x419df358, mutex=0x402a06ac, abstime=<optimized out>, clock=<optimized out>) at bionic/libc/bionic/pthread.c:1737
#3 0x402ee8de in android::Condition::wait (this=<optimized out>, mutex=...) at frameworks/native/include/utils/Condition.h:93
#4 0x402ef44a in android::AudioPolicyService::AudioCommandThread::parametersCommand (this=0x402a0688, ioHandle=<optimized out>, keyValuePairs=0x402a8b40 "routing=1",
delayMs=0) at frameworks/av/services/audioflinger/AudioPolicyService.cpp:913
#5 0x402ededc in android::aps_set_parameters (service=<optimized out>, io_handle=<optimized out>, kv_pairs=<optimized out>, delay_ms=<optimized out>)
at frameworks/av/services/audioflinger/AudioPolicyService.cpp:1660
#6 0x40642be4 in android_audio_legacy::AudioPolicyCompatClient::setParameters (this=<optimized out>, ioHandle=<optimized out>, keyValuePairs=..., delayMs=<optimized out>)
at hardware/libhardware_legacy/audio/AudioPolicyCompatClient.cpp:122
#7 0x4063c1d2 in android_audio_legacy::AudioPolicyManager::setOutputDevice (this=0x402a0008, output=2, device=AUDIO_DEVICE_OUT_EARPIECE, force=<optimized out>, delayMs=0)
at hardware/qcom/audio/alsa_sound/AudioPolicyManagerALSA.cpp:928
#8 0x4063c634 in setPhoneState (state=0, this=0x402a0008) at hardware/qcom/audio/alsa_sound/AudioPolicyManagerALSA.cpp:350
#9 android_audio_legacy::AudioPolicyManager::setPhoneState (this=0x402a0008, state=0) at hardware/qcom/audio/alsa_sound/AudioPolicyManagerALSA.cpp:274
#10 0x4063b3d4 in android_audio_legacy::ap_set_phone_state (pol=<optimized out>, state=<optimized out>) at hardware/qcom/audio/alsa_sound/audio_policy_hal.cpp:94
#11 0x402ee6de in android::AudioPolicyService::setPhoneState (this=0x402a04d0, state=AUDIO_MODE_NORMAL) at frameworks/av/services/audioflinger/AudioPolicyService.cpp:195
#12 0x406d311a in android::BnAudioPolicyService::onTransact (this=0x402a04d0, code=<optimized out>, data=..., reply=0x41fbbe04, flags=16)
at frameworks/av/media/libmedia/IAudioPolicyService.cpp:434
#13 0x404883ca in android::BBinder::transact (this=0x402a04d4, code=3, data=..., reply=0x41fbbe04, flags=16) at frameworks/native/libs/binder/Binder.cpp:108
#14 0x4048afc6 in android::IPCThreadState::executeCommand (this=0x41ac06d0, cmd=<optimized out>) at frameworks/native/libs/binder/IPCThreadState.cpp:1044
#15 0x4048b418 in android::IPCThreadState::joinThreadPool (this=0x41ac06d0, isMain=<optimized out>) at frameworks/native/libs/binder/IPCThreadState.cpp:478
#16 0x4048f1c4 in android::PoolThread::threadLoop (this=0x40586590) at frameworks/native/libs/binder/ProcessState.cpp:67
#17 0x40558f48 in android::Thread::_threadLoop (user=0x40586590) at frameworks/native/libs/utils/Threads.cpp:793
#18 0x40558aae in thread_data_t::trampoline (t=<optimized out>) at frameworks/native/libs/utils/Threads.cpp:132
#19 0x40314eb4 in __thread_entry (func=0x40558a15 <thread_data_t::trampoline(thread_data_t const*)>, arg=0x41ac0668, tls=0x41fbbf00) at bionic/libc/bionic/pthread.c:218
#20 0x4031460c in pthread_create (thread_out=0x41ac0690, attr=0xbea65b38, start_routine=0x40558a15 <thread_data_t::trampoline(thread_data_t const*)>, arg=0x41ac0668)
at bionic/libc/bionic/pthread.c:357
#21 0x00000000 in ?? ()
Wait on mutex=0x402a06ac held by 554.
(gdb) thread 3
[Switching to thread 3 (LWP 554)]
#0 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
59 bionic/libc/arch-arm/bionic/futex_arm.S: No such file or directory.
(gdb) bt
#0 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
#1 0x40314210 in _normal_lock (shared=0, mutex=0x4185102c) at bionic/libc/bionic/pthread.c:1069
#2 pthread_mutex_lock_impl (mutex=0x4185102c) at bionic/libc/bionic/pthread.c:1191
#3 0x40314f04 in pthread_mutex_lock (mutex=0x4185102c) at bionic/libc/bionic/pthread.c:1260
#4 __pthread_cond_timedwait_relative (cond=<optimized out>, mutex=0x4185102c, reltime=0x4178cda8) at bionic/libc/bionic/pthread.c:1715
#5 0x402dea5c in android::Condition::waitRelative (this=0x41851058, mutex=..., reltime=<optimized out>) at frameworks/native/include/utils/Condition.h:100
#6 0x402deac2 in android::AudioFlinger::ThreadBase::setParameters (this=0x41851008, keyValuePairs=...) at frameworks/av/services/audioflinger/AudioFlinger.cpp:1529
#7 0x402ed808 in android::AudioFlinger::setParameters (this=0x405865d0, ioHandle=<optimized out>, keyValuePairs=...)
at frameworks/av/services/audioflinger/AudioFlinger.cpp:1221
#8 0x406c8268 in android::AudioSystem::setParameters (ioHandle=2, keyValuePairs=...) at frameworks/av/media/libmedia/AudioSystem.cpp:182
#9 0x402ee9ec in android::AudioPolicyService::AudioCommandThread::threadLoop (this=0x402a0688) at frameworks/av/services/audioflinger/AudioPolicyService.cpp:740
#10 0x40558f48 in android::Thread::_threadLoop (user=0x402a0688) at frameworks/native/libs/utils/Threads.cpp:793
#11 0x40558aae in thread_data_t::trampoline (t=<optimized out>) at frameworks/native/libs/utils/Threads.cpp:132
#12 0x40314eb4 in __thread_entry (func=0x40558a15 <thread_data_t::trampoline(thread_data_t const*)>, arg=0x402a0720, tls=0x4178cf00) at bionic/libc/bionic/pthread.c:218
#13 0x4031460c in pthread_create (thread_out=0x402a0748, attr=0xbea65a88, start_routine=0x40558a15 <thread_data_t::trampoline(thread_data_t const*)>, arg=0x402a0720)
at bionic/libc/bionic/pthread.c:357
#14 0x00000000 in ?? ()
wait on mutex=0x4185102c held by pid 638
(gdb) bt
#0 __futex_syscall3 () at bionic/libc/arch-arm/bionic/futex_arm.S:59
#1 0x40314ef8 in __pthread_cond_timedwait_relative (cond=0x40669724, mutex=0x4034b408 <gThreadListLock>, reltime=0x0) at bionic/libc/bionic/pthread.c:1714
#2 0x40314f54 in __pthread_cond_timedwait (cond=0x40669724, mutex=0x4034b408 <gThreadListLock>, abstime=<optimized out>, clock=<optimized out>)
at bionic/libc/bionic/pthread.c:1737
#3 0x40314fec in pthread_join (thid=1080465152, ret_val=0x0) at bionic/libc/bionic/pthread.c:662
#4 0x41fc5394 in adev_close_output_stream_locked (dev=0x40393c30, stream=0x40669e80) at external/bluetooth/bluez/audio/android_audio_hw.c:763
#5 0x41fc53f6 in adev_close_output_stream_locked (stream=0x40669e80, dev=0x40393c30) at external/bluetooth/bluez/audio/android_audio_hw.c:749
#6 adev_close_output_stream (dev=0x40393c30, stream=0x40669e80) at external/bluetooth/bluez/audio/android_audio_hw.c:782
#7 0x415c1284 in android_audio_legacy::AudioHardwareALSA::closeA2dpOutput (this=0x400e5008) at hardware/qcom/audio/alsa_sound/AudioHardwareALSA.cpp:2256
#8 0x415c1ce0 in android_audio_legacy::AudioHardwareALSA::stopA2dpPlayback_l (this=0x400e5008, activeUsecase=1) at hardware/qcom/audio/alsa_sound/AudioHardwareALSA.cpp:2191
#9 0x415c2384 in android_audio_legacy::AudioHardwareALSA::doRouting (this=0x400e5008, device=1) at hardware/qcom/audio/alsa_sound/AudioHardwareALSA.cpp:924
#10 0x415c6550 in android_audio_legacy::ALSAStreamOps::setParameters (this=0x41abc3fc, keyValuePairs=...) at hardware/qcom/audio/alsa_sound/ALSAStreamOps.cpp:208
#11 0x415c7590 in android_audio_legacy::out_set_parameters (stream=<optimized out>, kvpairs=<optimized out>) at hardware/qcom/audio/alsa_sound/audio_hw_hal.cpp:125
#12 0x402ea62e in android::AudioFlinger::MixerThread::checkForNewParameters_l (this=0x41851008) at frameworks/av/services/audioflinger/AudioFlinger.cpp:3879
#13 0x402e7b14 in android::AudioFlinger::PlaybackThread::threadLoop (this=0x41851008) at frameworks/av/services/audioflinger/AudioFlinger.cpp:2907
#14 0x40558f48 in android::Thread::_threadLoop (user=0x41851008) at frameworks/native/libs/utils/Threads.cpp:793
#15 0x40558aae in thread_data_t::trampoline (t=<optimized out>) at frameworks/native/libs/utils/Threads.cpp:132
#16 0x40314eb4 in __thread_entry (func=0x40558a15 <thread_data_t::trampoline(thread_data_t const*)>, arg=0x41abf4d8, tls=0x41eabf00) at bionic/libc/bionic/pthread.c:218
#17 0x4031460c in pthread_create (thread_out=0x41abf500, attr=0xbea65960, start_routine=0x40558a15 <thread_data_t::trampoline(thread_data_t const*)>, arg=0x41abf4d8)
at bionic/libc/bionic/pthread.c:357
#18 0x00000000 in ?? ()
(gdb) up 3
#3 0x40314fec in pthread_join (thid=1080465152, ret_val=0x0) at bionic/libc/bionic/pthread.c:662
662 bionic/libc/bionic/pthread.c: No such file or directory.
(gdb) info args
thid = 1080465152
ret_val = 0x0
Thread 638 is in thread_join for a child thread. Try to find the child thread out.
From the info and args of frame #3, the child thread pid can be gott.
#3 0x40314FEC in pthread_join+0x0070(+112) /system/lib/libc.so
bionic/libc/bionic/pthread.c:662
thid (long int) = 1080465152
ret_val (? * *) = 0x0
thread (pthread_internal_t *) = 0x40669700 (thid) *************
count (int) = <optimized out>
631int pthread_join(pthread_t thid, void ** ret_val)
632{
633 pthread_internal_t* thread = (pthread_internal_t*)thid; *************
634 int count;
635
636 // check that the thread still exists and is not detached
637 pthread_mutex_lock(&gThreadListLock);
638
639 for (thread = gThreadList; thread != NULL; thread = thread->next)
640 if (thread == (pthread_internal_t*)thid)
641 goto FoundIt;
642
643 pthread_mutex_unlock(&gThreadListLock);
644 return ESRCH;
645
646FoundIt:
647 if (thread->attr.flags & PTHREAD_ATTR_FLAG_DETACHED) {
648 pthread_mutex_unlock(&gThreadListLock);
649 return EINVAL;
650 }
651
652 /* wait for thread death when needed
653 *
654 * if the 'join_count' is negative, this is a 'zombie' thread that
655 * is already dead and without stack/TLS
656 *
657 * otherwise, we need to increment 'join-count' and wait to be signaled
658 */
659 count = thread->join_count;
660 if (count >= 0) {
661 thread->join_count += 1;
662 pthread_cond_wait( &thread->join_cond, &gThreadListLock );
663 count = --thread->join_count;
664 }
665 if (ret_val)
666 *ret_val = thread->return_value;
667
668 /* remove thread descriptor when we're the last joiner or when the
669 * thread was already a zombie.
670 */
671 if (count <= 0) {
672 _pthread_internal_remove_locked(thread);
673 _pthread_internal_free(thread);
674 }
675 pthread_mutex_unlock(&gThreadListLock);
676 return 0;
677}
35typedef struct pthread_internal_t
36{
37 struct pthread_internal_t* next;
38 struct pthread_internal_t** pref;
39 pthread_attr_t attr; (24 )
40 pid_t kernel_id;
41 pthread_cond_t join_cond;
42 int join_count;
43 void* return_value;
44 int intern;
45 __pthread_cleanup_t* cleanup_stack;
46 void** tls; /* thread-local storage area */
47} pthread_internal_t;
The pthread_internal_t object shape is here.
0x40669700: 0x410a5e60 0x402a8d30 0x00000000 0x43179000
0x40669710: 0x00100000 0x00001000 0x00000000 0x00000000
0x40669720: 0x0000064b(**) 0x00000000
0x0000064b = 1611d. It is thread 1611, but 1611 maybe has something wrong?
Examine user stack and kernel stack of thread 1611.
User stack for thread 1611.
(gdb) bt
#0 0xffff0520 in ?? () ----- only deduced with the current pc, remove it from user stack.
#1 0x4031ca32 in usleep (usec=<optimized out>) at bionic/libc/unistd/usleep.c:46
#2 0x000003e8 in ?? () from system/bin/linker
Kernel stack for pid 1611
#0 [INLINE] context_switch
kernel/kernel/sched/core.c:2086
next (task_struct *) = 0xEDD2C000
prev (task_struct *) = 0xE6508D80
rq (rq *) = 0xC244A5C0
block
mm (mm_struct *) = 0xED81C700
oldmm (mm_struct *) = 0xED81CFC0
#1 0xC06EFC94 in __schedule+0x06A8(+1704)
kernel/kernel/sched/core.c:3233
prev (task_struct *) = 0xE6508D80
next (task_struct *) = 0xEDD2C000
switch_count (long unsigned int *) = <optimized out>
rq (rq *) = 0xC244A5C0
cpu (int) = <optimized out>
#2 0xC06EEF84 in do_nanosleep+0x0078(+120)
kernel/kernel/hrtimer.c:1504
t (hrtimer_sleeper *) = 0xE49AFF40
mode (hrtimer_mode) = <optimized out>
#3 0xC06EF01C in hrtimer_nanosleep_restart+0x0040(+64)
kernel/kernel/hrtimer.c:1542
restart (restart_block *) = 0xE49AE2C8
t (hrtimer_sleeper) = {
timer = {
node = {
node = {
rb_parent_color = 0xED831F29,
rb_right = 0xC8707AC8,
rb_left = 0x0},
expires = {
tv64 = 4325397121115}},
_softexpires = {
tv64 = 4325397121115},
function = 0xC009AA48,
base = 0xC2447970,
state = 1,
start_pid = 1611,
start_site = 0xC009B634,
start_comm = {'F', 'a', 's', 't', 'M', 'i', 'x', 'e', 'r', '\0', '\0', '\0', '\0', '\0', '\0', '\0'}},
task = 0xE6508D80}
rmtp (timespec *) = <optimized out>
ret (int) = <unknown>
#4 0xC008BB88 in sys_restart_syscall+0x001C(+28) ******************??????????????????
kernel/kernel/signal.c:2502
restart (restart_block *) = <unknown>
#5 System call exception frame.
Registers from userland:
r00=0xFFFFFFFC r01=0x43278E30 r02=0x00000695 r03=0x000F4240
r04=0x000003E8 r05=0x00000000 r06=0x00000200 r07=0x000000A2
r08=0x00000001 r09=0x00000A00 r10=0x00000008 r11=0x40004808
r12=0x41FCBF98 r13=0x43278E1C r14=0x4031CA33 r15=0xFFFF0520
The kernel stack indicates it still in usleep() which is implemented via system call nanosleep().
NOTE:
The nanosleep uses sys_restart_syscall.
State of Thread 1611,
PID: 1611
COMMAND: "FastMixer"
TASK: e6508d80 [THREAD_INFO: e49ae000]
CPU: 0
STATE: TASK_INTERRUPTIBLE
Task state is TASK_INTERRUPTIBLE, which indicates that the task is not exited.
So, thread 1611 is still running for so long when it is commanded to stop?
Whether or not, thread_join() has not returned. The thread is still waiting the forked thread.
The application logic should be double checked.