Error while unloading appdomain. (Exception from HRESULT: 0x80131015)

消息中间件以不同AppDomain方式隔离加载消息处理器。其中的AppDomainWatcher类型会监控每个插件目录的文件是否变化，当发生变化时则卸载这个AppDomain，并在下次访问时新建。开发进度下，往往会频繁更新这些插件。极为偶然的情况下会出现：Attempted to access an unloaded AppDomain.   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName) 。WinDbg附加等待错误发生，之后捕捉到一个second chance。大致内容如下：

 1 ModLoad: 000007fe`e9290000 000007fe`e937a000   C:\Windows\Microsoft.NET\Framework64\v4.0.30319\diasymreader.dll
 2 System.Transactions Critical: 0 : <TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Critical"><TraceIdentifier>http://msdn.microsoft.com/TraceCodes/System/ActivityTracing/2004/07/Reliability/Exception/Unhandled</TraceIdentifier><Description>Unhandled exception</Description><AppDomain>xxxxxx.exe</AppDomain><Exception><ExceptionType>System.CannotUnloadAppDomainException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType><Message>Error while unloading appdomain. (Exception from HRESULT: 0x80131015)</Message><StackTrace>   at System.AppDomain.Unload(AppDomain domain)
 3    at xxxxxx.Messaging.Configuration.AppDomainPool.AppDomainWatcher.fileSystemWatcher_Changed(Object sender, FileSystemEventArgs e)
 4    at System.IO.FileSystemWatcher.CompletionStatusChanged(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* overlappedPointer)
 5    at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)</StackTrace><ExceptionString>System.CannotUnloadAppDomainException: Error while unloading appdomain. (Exception from HRESULT: 0x80131015)
 6    at System.AppDomain.Unload(AppDomain domain)
 7    at xxxxxx.Messaging.Configuration.AppDomainPool.AppDomainWatcher.fileSystemWatcher_Changed(Object sender, FileSystemEventArgs e)
 8    at System.IO.FileSystemWatcher.CompletionStatusChanged(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* overlappedPointer)
 9    at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)</ExceptionString></Exception></TraceRecord>
10 (119c.1358): CLR exception - code e0434352 (!!! second chance !!!)
11 KERNELBASE!RaiseException+0x39:
12 000007fe`fdaea49d 4881c4c8000000  add     rsp,0C8h

AppDomainWatcher产生异常的源代码部分：

 1 /// <summary>
 2 /// 文件系统变化
 3 /// </summary>
 4 /// <param name="sender">发送方</param>
 5 /// <param name="e">事件参数</param>
 6 private void fileSystemWatcher_Changed(Object sender, FileSystemEventArgs e)
 7 {
 8 　　m_fileSystemWatcher.EnableRaisingEvents = false;
 9    m_fileSystemWatcher.Changed -= new FileSystemEventHandler(fileSystemWatcher_Changed);
10    m_fileSystemWatcher.Dispose();
11 
12    lock (this.SyncRoot)
13    {
14    　　AppDomain.Unload(this.AppDomain);
15    }
16 
17    OnDomainUnload(EventArgs.Empty);
18 }

在卸载一个AppDomain时抛出了System.CannotUnloadAppDomainException，导致卸载该AppDomain失败。查阅MSDN关于AppDomain.Unload函数Remarks：

The dedicated thread attempts to unload the domain, and all threads in the domain are aborted. If a thread does not abort, for example because it is executing unmanaged code, or because it is executing a finally block, then after a period of time a CannotUnloadAppDomainException is thrown in the thread that originally called Unload. If the thread that could not be aborted eventually ends, the target domain is not unloaded

用~* k、~* e !clrstack查看所有线程的call stack，发现有一处停留在获取数据库连接的等待中，这就是AppDomain卸载失败的原因。再看一段MSDN关于AppDomain卸载的补充：

By the way, when a thread calls AppDomain.Unload, the CLR waits 10 seconds (by default) for the threads in the unloading AppDomain to leave it. If after 10 seconds, the thread that called AppDomain.Unload doesn’t return, it will throw a CannotUnloadAppDomainException, and the AppDomain may or may not be unloaded in the future.

Note If a thread calling AppDomain.Unload is in the AppDomain being unloaded, the CLR creates another thread, and this new thread attempts to unload the AppDomain. The first thread will forcibly throw the ThreadAbortException and unwind. The new thread will waitfor the AppDomain to unload, and then the new thread terminates. If the AppDomain fails to unload, the new thread will process a CannotUnloadAppDomainException, but since you did not write the code that this new thread executes, you can’t catch this exception.

默认情况下CLR会等待10秒由所有线程退出，之后调用卸载线程没有返回则抛出CannotUnloadAppDomainException。对于在被卸载AppDomain中的线程尝试卸载，CLR会新建一个线程处理。