SocketAsyncEventArgs的释放问题

起因是发现一个同事编写的程序运行两个月左右，占用了服务器20G左右的内存。用WinDbg查看发现存在大量的Async Pinned Handles，而它们的gcroot都来自于SocketAsyncEventArgs。下面是场景的简易模拟代码（为了说明问题添加了手动GC）：

for (var i = 0; i < 1000; ++i)
{
    var endPoint = new IPEndPoint(IPAddress.Parse(host), port);
    var socket = new Socket(endPoint.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
    socket.ReceiveBufferSize = bufferSize;
    socket.SendBufferSize = bufferSize;

    var iocp = new SocketAsyncEventArgs();
    iocp.Completed += new EventHandler<SocketAsyncEventArgs>(OnIoSocketCompleted);
    iocp.SetBuffer(new Byte[bufferSize], 0, bufferSize);
    iocp.AcceptSocket = socket;

    try
    {
        socket.Connect(endPoint);
        Console.WriteLine(i);
    }
    catch (SocketException ex)
    {
        Console.WriteLine(ex.Message);
    }

    socket.Close();
    //iocp.Dispose();
}

GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();

SocketAsyncEventArgs的SetBuffer函数内部会pin住buffer数据（查阅SetBufferInternal实现），它的析构函数会调用FreeOverlapped函数释放资源。而问题就是在于FreeOverlapped函数的实现和传递参数。来看一下Dispose、~SocketAsyncEventArgs的实现：

public void Dispose()
{
    this.m_DisposeCalled = true;
    if (Interlocked.CompareExchange(ref this.m_Operating, 2, 0) != 0)
    {
        return;
    }
    this.FreeOverlapped(false);
    GC.SuppressFinalize(this);
}

~SocketAsyncEventArgs()
{
    this.FreeOverlapped(true);
}

两者都会调用FreeOverlapped函数释放，但是一个传递了false、一个传递了true参数。再来看FreeOverlapped实现：

private void FreeOverlapped(bool checkForShutdown)
{
    if (!checkForShutdown || !NclUtilities.HasShutdownStarted)
    {
        if (this.m_PtrNativeOverlapped != null && !this.m_PtrNativeOverlapped.IsInvalid)
        {
            this.m_PtrNativeOverlapped.Dispose();
            this.m_PtrNativeOverlapped = null;
            this.m_Overlapped = null;
            this.m_PinState = SocketAsyncEventArgs.PinState.None;
            this.m_PinnedAcceptBuffer = null;
            this.m_PinnedSingleBuffer = null;
            this.m_PinnedSingleBufferOffset = 0;
            this.m_PinnedSingleBufferCount = 0;
        }
        if (this.m_SocketAddressGCHandle.IsAllocated)
        {
            this.m_SocketAddressGCHandle.Free();
        }
        if (this.m_WSAMessageBufferGCHandle.IsAllocated)
        {
            this.m_WSAMessageBufferGCHandle.Free();
        }
        if (this.m_WSARecvMsgWSABufferArrayGCHandle.IsAllocated)
        {
            this.m_WSARecvMsgWSABufferArrayGCHandle.Free();
        }
        if (this.m_ControlBufferGCHandle.IsAllocated)
        {
            this.m_ControlBufferGCHandle.Free();
        }
    }
}

用WinDbg查看gchandles统计：

Statistics:
              MT    Count    TotalSize Class Name
000007fb1bdb6ae8        1           24 System.Object
000007fb1bdb6b80        1           48 System.SharedStatics
000007fb1bdb7f58        1           64 System.Security.PermissionSet
000007fb1bdb6a10        1          160 System.ExecutionEngineException
000007fb1bdb6998        1          160 System.StackOverflowException
000007fb1bdb6920        1          160 System.OutOfMemoryException
000007fb1bdb6738        1          160 System.Exception
000007fb1bdb7b90        2          192 System.Threading.Thread
000007fb1bdb6c40        1          216 System.AppDomain
000007fb1bdb6a88        2          320 System.Threading.ThreadAbortException
000007fb1ae19770        6          336 System.Net.Logging+NclTraceSource
000007fb1bdbf958        3          480 System.RuntimeType+RuntimeTypeCache
000007fb1ae19900        6          480 System.Diagnostics.SourceSwitch
000007fb1bd64458        6        34520 System.Object[]
000007fb1b6f5e40     1000       112000 System.Threading.OverlappedData
Total 1033 objects

Handles:
    Strong Handles:       12
    Pinned Handles:       5
    Async Pinned Handles: 1000
    Weak Long Handles:    3
    Weak Short Handles:   13

确实存在1000个Async Pinned Handles，也就是说无法通过SocketAsyncEventArgs的析构函数释放SafeNativeOverlapped相关的资源。将示例代码的"iocp.Dispose();"注释去除，并重新执行再次查看gchandles：

Statistics:
              MT    Count    TotalSize Class Name
000007fb1bdb6ae8        1           24 System.Object
000007fb1bdb6b80        1           48 System.SharedStatics
000007fb1bdb7f58        1           64 System.Security.PermissionSet
000007fb1bdb6a10        1          160 System.ExecutionEngineException
000007fb1bdb6998        1          160 System.StackOverflowException
000007fb1bdb6920        1          160 System.OutOfMemoryException
000007fb1bdb6738        1          160 System.Exception
000007fb1bdb7b90        2          192 System.Threading.Thread
000007fb1bdb6c40        1          216 System.AppDomain
000007fb1bdb6a88        2          320 System.Threading.ThreadAbortException
000007fb1ae19770        6          336 System.Net.Logging+NclTraceSource
000007fb1bdbf958        3          480 System.RuntimeType+RuntimeTypeCache
000007fb1ae19900        6          480 System.Diagnostics.SourceSwitch
000007fb1bd64458        6        34520 System.Object[]
Total 33 objects

Handles:
    Strong Handles:       12
    Pinned Handles:       5
    Weak Long Handles:    3
    Weak Short Handles:   13

1000个Async Pinned Handles已不存在，但SocketAsyncEventArgs的析构函数从实现来看应该也可以完成释放，为什么失败了？

用!bpmd命令添加NclUtilities.HasShutdownStarted、 m_PtrNativeOverlapped.Dispose()的断点。

!bpmd System.dll System.Net.NclUtilities.get_HasShutdownStarted
!bpmd mscorlib.dll System.Runtime.InteropServices.SafeHandle.Dispose

 以上函数无法命中，由于.NET Framework ngen的关系。实际可以看到SocketAsyncEventArgs的Finalize函数内联了FreeOverlapped。

0:002> !clrstack -a
OS Thread Id: 0x46c (2)
        Child SP               IP Call Site
000000002656f940 000007ff138485e9 System.Net.Sockets.SocketAsyncEventArgs.FreeOverlapped(Boolean)
    PARAMETERS:
        this = <no data>
        checkForShutdown = <no data>

000000002656f980 000007ff1385222d System.Net.Sockets.SocketAsyncEventArgs.Finalize()
    PARAMETERS:
        this (0x000000002656f9c0) = 0x000000000e140438

000000002656fd38 000007ff72c1c446 [DebuggerU2MCatchHandlerFrame: 000000002656fd38] 

逐一单步执行可以发现NclUtilities.HasShutdownStarted也被内联，直接调用外部函数clr!SystemNative::HasShutdownStarted。

0:002> t
000007ff`138485ec 85c0            test    eax,eax
0:002> t
000007ff`138485ee 7438            je      000007ff`13848628              [br=0]
0:002> t
000007ff`138485f0 e8c798485f      call    clr!SystemNative::HasShutdownStarted (000007ff`72cd1ebc)

那问题就在于HasShutdownStarted的返回值了，如果为false则肯定会释放资源。

0:002> t
clr!SystemNative::HasShutdownStarted:
000007ff`72cd1ebc 4883ec28        sub     rsp,28h
0:002> t
clr!SystemNative::HasShutdownStarted+0x4:
000007ff`72cd1ec0 f60501767a0004  test    byte ptr [clr!g_fEEShutDown (000007ff`734794c8)],4 ds:000007ff`734794c8=07
0:002> t
clr!SystemNative::HasShutdownStarted+0xb:
000007ff`72cd1ec7 751a            jne     clr!SystemNative::HasShutdownStarted+0x27 (000007ff`72cd1ee3) [br=1]
0:002> t
clr!SystemNative::HasShutdownStarted+0x27:
000007ff`72cd1ee3 b901000000      mov     ecx,1
0:002> t
clr!SystemNative::HasShutdownStarted+0x2c:
000007ff`72cd1ee8 ebf2            jmp     clr!SystemNative::HasShutdownStarted+0x20 (000007ff`72cd1edc)
0:002> t
clr!SystemNative::HasShutdownStarted+0x20:
000007ff`72cd1edc 8bc1            mov     eax,ecx
0:002> t
clr!SystemNative::HasShutdownStarted+0x22:
000007ff`72cd1ede 4883c428        add     rsp,28h
0:002> t
clr!SystemNative::HasShutdownStarted+0x26:
000007ff`72cd1ee2 c3              ret
0:002> t
000007ff`138485f5 0fb6c8          movzx   ecx,al
0:002> t
000007ff`138485f8 85c9            test    ecx,ecx

查看ecx的值发现竟然是1。

 

0:002> r ecx
ecx=1
0:002> r al
al=1