Android 死锁问题分析记录

问题描述：
binder thread 和 ActivityManager之间发生死锁
Traces:

"ActivityManager" prio=5 tid=12 Blocked
  ...
  at ActivityManagerService.updateCpuStatsNow(ActivityManagerService.java:3107)
  - waiting to lock <0x0037ca7c> (aandroid.util.SparseArray) held by thread 135
  - locked <0x0dfe4378> (a com.android.internal.os.ProcessCpuTracker)
  - locked <0x064e7a29> (a com.android.internal.os.BatteryStatsImpl)
  at AppErrors.appNotResponding(AppErrors.java:983)
  at ActivityManagerService$17.run(ActivityManagerService.java:13416)
  at android.os.Handler.handleCallback(Handler.java:793)
  at android.os.Handler.dispatchMessage(Handler.java:98)
  at android.os.Looper.loop(Looper.java:173)
  at android.os.HandlerThread.run(HandlerThread.java:65)
  at com.android.server.ServiceThread.run(ServiceThread.java:46)

"Binder:1328_15" prio=5 tid=135 Blocked
  ...
  at BatteryStatsService.noteEvent(BatteryStatsService.java:422)
  - waiting to lock <0x064e7a29> (acom.android.internal.os.BatteryStatsImpl) held by thread 12
  at ActivityManagerService.updateProcessForegroundLocked(ActivityManagerService.java:23108)
  at ActivityManagerService.importanceTokenDied(ActivityManagerService.java:8177)
  - locked <0x0ef100c6> (a ActivityManagerService)
  - locked <0x0037ca7c> (a android.util.SparseArray)
  at ActivityManagerService$13.binderDied(ActivityManagerService.java:8209)
  at android.os.BinderProxy.sendDeathNotice(Binder.java:842)

对于ActivityManager，查看trace 发现 “ waiting to lock <0x0037ca7c> (aandroid.util.SparseArray) held by thread 135 ”， 找到 thread 135：

"Binder:1328_15" prio=5 tid=135 Blocked

发现这个Binder线程也 waiting to lock <0x064e7a29> 并且 held by thread 12
而 thread 12正是“ “ActivityManager” prio=5 tid=12”。

由此确认为死锁问题

 - waiting to lock <0x064e7a29> (acom.android.internal.os.BatteryStatsImpl) held by thread 12
  at ActivityManagerService.updateProcessForegroundLocked(ActivityManagerService.java:23108)
  at ActivityManagerService.importanceTokenDied(ActivityManagerService.java:8177)

解决：
services/core/java/com/android/server/am/ActivityManagerService.java

    void importanceTokenDied(ImportanceToken token) {
    +   ProcessRecord pr = null;
        synchronized (ActivityManagerService.this) {
            synchronized (mPidsSelfLocked) {
                ImportanceToken cur
                    = mImportantProcesses.get(token.pid);
                if (cur != token) {
                    return;
                }
                mImportantProcesses.remove(token.pid);
           -    ProcessRecord pr = mPidsSelfLocked.get(token.pid);
           +    pr = mPidsSelfLocked.get(token.pid);
                if (pr == null) {
                    return;
                }
                pr.forcingToImportant = null;
        -       updateProcessForegroundLocked(pr, false, false);
            }
        +   updateProcessForegroundLocked(pr, false, false);
            updateOomAdjLocked();
        }

另外一个例子：
手机U启动后mount externel srotage 可能引起dead lock

trace stack:

"Binder_6" prio=5 tid=57 Blocked
  | group="main" sCount=1 dsCount=0 obj=0x12fa7fa0 self=0x7f9674d000
  | sysTid=3218 nice=0 cgrp=default sched=0/0 handle=0x7f941a3440
  | state=S schedstat=( 450091692 353243785 1757 ) utm=30 stm=15 core=8 HZ=100
  | stack=0x7f940a7000-0x7f940a9000 stackSize=1013KB
  | held mutexes=
  at com.android.server.MountService.getVolumeList(MountService.java:3014)
  - waiting to lock <0x064315bf> (a java.lang.Object) held by thread 14                                        B
  at android.os.storage.StorageManager.getVolumeList(StorageManager.java:918)
  at android.os.storage.StorageManager.getStorageVolume(StorageManager.java:853)
  at android.os.Environment.isExternalStorageEmulated(Environment.java:742)
  at android.os.Environment.isExternalStorageEmulated(Environment.java:730)
  at com.android.server.pm.PackageManagerService.isExternalMediaAvailable(PackageManagerService.java:10378)
  at com.android.server.pm.PackageManagerService.nextPackageToClean(PackageManagerService.java:10385)
  - locked <0x05b654c7> (a android.util.ArrayMap)                                                              A
  at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:1636)
  at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:2937)
  at android.os.Binder.execTransact(Binder.java:458)

"android.fg" prio=5 tid=14 Blocked
  | group="main" sCount=1 dsCount=0 obj=0x12da1f90 self=0x7fa9ad8800
  | sysTid=1792 nice=0 cgrp=default sched=0/0 handle=0x7f9867f440
  | state=S schedstat=( 79482537 29476078 684 ) utm=4 stm=4 core=5 HZ=100
  | stack=0x7f9857d000-0x7f9857f000 stackSize=1037KB
  | held mutexes=
  at com.android.server.am.ActivityManagerService.broadcastIntent(ActivityManagerService.java:19159)
  - waiting to lock <0x074b3319> (a com.android.server.am.ActivityManagerService) held by thread 98           C
  at android.app.ContextImpl.sendBroadcastAsUser(ContextImpl.java:942)
  at com.android.server.MountService.onVolumeStateChangedLocked(MountService.java:1424)
  at com.android.server.MountService.onEventLocked(MountService.java:1134)
  at com.android.server.MountService.onEvent(MountService.java:1039)
  - locked <0x064315bf> (a java.lang.Object)                                                                  B
  at com.android.server.NativeDaemonConnector.handleMessage(NativeDaemonConnector.java:135)
  at android.os.Handler.dispatchMessage(Handler.java:107)
  at android.os.Looper.loop(Looper.java:207)
  at android.os.HandlerThread.run(HandlerThread.java:61)
  at com.android.server.ServiceThread.run(ServiceThread.java:46)

"Binder_F" prio=5 tid=98 Blocked
  | group="main" sCount=1 dsCount=0 obj=0x13f740a0 self=0x7f96ab6400
  | sysTid=3483 nice=0 cgrp=default sched=0/0 handle=0x7f8dbb7440
  | state=S schedstat=( 472520780 311910624 1572 ) utm=35 stm=12 core=6 HZ=100
  | stack=0x7f8dabb000-0x7f8dabd000 stackSize=1013KB
  | held mutexes=
  at com.android.server.pm.PackageManagerService.queryContentProviders(PackageManagerService.java:5974)
  - waiting to lock <0x05b654c7> (a android.util.ArrayMap) held by thread 57                                   A
  at com.android.server.am.ActivityManagerService.generateApplicationProvidersLocked(ActivityManagerService.java:10786)
  at com.android.server.am.ActivityManagerService.attachApplicationLocked(ActivityManagerService.java:7405)
  at com.android.server.am.ActivityManagerService.attachApplication(ActivityManagerService.java:7577)
  - locked <0x074b3319> (a com.android.server.am.ActivityManagerService)                                       C
  at android.app.ActivityManagerNative.onTransact(ActivityManagerNative.java:513)
  at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2764)
  at android.os.Binder.execTransact(Binder.java:458)

services/core/java/com/android/server/pm/PackageManagerService.java修改

    @Override
    public PackageCleanItem nextPackageToClean(PackageCleanItem lastPackage) {
        if (getInstantAppPackageName(Binder.getCallingUid()) != null) {
            return null;
        }
        // writer
    -    synchronized (mPackages) {
            if (!isExternalMediaAvailable()) {
                // If the external storage is no longer mounted at this point,
                // the caller may not have been able to delete all of this
                // packages files and can not delete any more.  Bail.
                return null;
            }
    +    synchronized (mPackages) {
            final ArrayList pkgs = mSettings.mPackagesToBeCleaned;
            if (lastPackage != null) {
                pkgs.remove(lastPackage);
            }
            if (pkgs.size() > 0) {
                return pkgs.get(0);
            }
        }
        return null;