近日,客户说在某检索画面检索大量数据后,再执行CSV下载,出现服务器异常,显示OutOfMemoryException。
先用任务管理器/性能查看了下CSV下载页面的内存使用情况,并在检索及CSV下载完成后各dump了一次。
执行前:
检索及画面表示后:
CSV下载后:
从图上看,在检索及页面表示时一口气吃了280M可用内存。dump的文件大小,检索:370M,CSV下载:400M。
用windbg分析第一个dump文件(检索)
(部分略)
dumpheap -stat
total 8,597,930 objectsTotal 8,597,930 objects, Total size: 303,663,236
可以看到,一共有23,170条TableRow,509,740个TableCell,每行数据22列。 TableCell、LiteralControl这两项就占用了86M,
为什么会有686,345个String、686,345个Object[]呢,一步步分析。
从上面可以看到Dataset只有80bytes,这只是它结构本身的大小,不包括其下DataRow等的实际大小。
用!objsize查看它的实际大小
!dumpheap -mt 0x040ca234
Using our cache to search the heap.
Address MT Size Gen
0x1632d88 0x40ca234 80 2 System.Data.DataSet
Statistics:
MT Count TotalSize Class Name
0x040ca234 1 80 System.Data.DataSet
!objsize 0x1632d88
sizeof(0x1632d88) = 24,741,124 (0x1798504) bytes (System.Data.DataSet)
总共24M。
首先引起我注意的是System.Web.UI.StateBag,MSDN上是这么说的:管理 ASP.NET 服务器控件(包括页)的视图状态。
ViewState机制由浅入深3 中的表述:StateBag中 保存Key/Value对,Key是String类型,Value是Object类型。但是在StateBag内部保存Value不是Object类型, 而是将Object类型转换为StateItem类型然后保存,从StateBag中取出的时候再将StateItem类型转换为Object类型,也就 是说StateBag中的Key/Value对实际上是String/StateItem类型。转换过程是在StateBag内部实现客户感觉不到。
MT Count TotalSize Class Name
0x040c6c90 509,761 8,156,176 System.Web.UI.StateBag
!dumpheap -mt 0x040c6c90 -random ,然后用!objsize查看了几个
Address MT Size Gen
0x161eaa4 0x40c6c90 16 2 System.Web.UI.StateBag 168
0x161ec14 0x40c6c90 16 2 System.Web.UI.StateBag 160
0x1630070 0x40c6c90 16 2 System.Web.UI.StateBag 72
0x162a764 0x40c6c90 16 2 System.Web.UI.StateBag 36
0x161f888 0x40c6c90 16 2 System.Web.UI.StateBag 256
0x161f77c 0x40c6c90 16 2 System.Web.UI.StateBag 252
最后一列是该对象的实际大小。
!do 0x161f888
Name: System.Web.UI.StateBag
MethodTable 0x040c6c90
EEClass 0x040f1bc0
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020001ba (c:"windows"assembly"gac"system.web"1.0.5000.0__b03f5f7f11d50a3a"system.web.dll)
FieldDesc*: 0x040c6904
MT Field Offset Type Attr Value Name
0x040c6c90 0x4000d86 0x4 CLASS instance 0x0161f898 bag
0x040c6c90 0x4000d87 0x8 System.Boolean instance 1 marked
0x040c6c90 0x4000d88 0x9 System.Boolean instance 0 ignoreCase
bag是个什么东东呢
!do 0x0161f898
Name: System.Collections.Specialized.HybridDictionary
MethodTable 0x03c8c450
EEClass 0x03ca2e8c
Size 20(0x14) bytes
GC Generation: 2
mdToken: 0x02000172 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8c254
MT Field Offset Type Attr Value Name
0x03c8c450 0x4000a77 0x4 CLASS instance 0x0161f8bc list
0x03c8c450 0x4000a78 0x8 CLASS instance 0x00000000 hashtable
0x03c8c450 0x4000a79 0xc System.Boolean instance 0 caseInsensitive
0x03c8c450 0x4000a75 0 CLASS shared static hashCodeProvider
>> Domain:Value 0x00155980:NotInit 0x00193830:0x0162d6a4 <<
0x03c8c450 0x4000a76 0x4 CLASS shared static comparer
>> Domain:Value 0x00155980:NotInit 0x00193830:0x0155d574 <<
原来bag是ystem.Collections.Specialized.HybridDictionary,MSDN:在集合较小时,使用 ListDictionary 来实现 IDictionary,
然后当集合变大时,切换到Hashtable。(集合较小是指key/value键值对的个数较少)
很明显,list就是上面提到的ListDictionary,hashtable嘛就是Hashtable了。继续查看list成员。
!do 0x0161f8bc
Name: System.Collections.Specialized.ListDictionary
MethodTable 0x03c8ca90
EEClass 0x03ca32a4
Size 24(0x18) bytes
GC Generation: 2
mdToken: 0x02000174 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8c8fc
MT Field Offset Type Attr Value Name
0x03c8ca90 0x4000a7a 0x4 CLASS instance 0x0161f8d4 head
0x03c8ca90 0x4000a7b 0xc System.Int32 instance 2 version
0x03c8ca90 0x4000a7c 0x10 System.Int32 instance 2 count
0x03c8ca90 0x4000a7d 0x8 CLASS instance 0x00000000 comparer
head是个什么东东呢?
!do 0x0161f8d4
Name: System.Collections.Specialized.ListDictionary/DictionaryNode
MethodTable 0x03c8cc24
EEClass 0x03ca3394
Size 20(0x14) bytes
GC Generation: 2
mdToken: 0x02000178 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8cbd4
MT Field Offset Type Attr Value Name
0x03c8cc24 0x4000a89 0x4 CLASS instance 0x0155d610 key
0x03c8cc24 0x4000a8a 0x8 CLASS instance 0x0161f8ac value
0x03c8cc24 0x4000a8b 0xc CLASS instance 0x0161f8f8 next
再看一下next
!do 0x0161f8f8
Name: System.Collections.Specialized.ListDictionary/DictionaryNode
MethodTable 0x03c8cc24
EEClass 0x03ca3394
Size 20(0x14) bytes
GC Generation: 2
mdToken: 0x02000178 (c:"windows"assembly"gac"system"1.0.5000.0__b77a5c561934e089"system.dll)
FieldDesc*: 0x03c8cbd4
MT Field Offset Type Attr Value Name
0x03c8cc24 0x4000a89 0x4 CLASS instance 0x0155d930 key
0x03c8cc24 0x4000a8a 0x8 CLASS instance 0x0161f8e8 value
0x03c8cc24 0x4000a8b 0xc CLASS instance 0x00000000 next
从head、next可以看出是个链表结构。
继续查看head的key、value
!do 0x0155d610
String: CssClass
!do 0x0161f8ac
Name: System.Web.UI.StateItem
MethodTable 0x040c6ff4
EEClass 0x040f1d14
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020001bb (c:"windows"assembly"gac"system.web"1.0.5000.0__b03f5f7f11d50a3a"system.web.dll)
FieldDesc*: 0x040c6f70
MT Field Offset Type Attr Value Name
0x040c6ff4 0x4000d89 0x4 CLASS instance 0x0155d8f8 value
0x040c6ff4 0x4000d8a 0x8 System.Boolean instance 0 isDirty
!do 0x0155d8f8
String: button
原来是CssClass=button,key是string对象,value是StateItem对象,这与前面ViewState机制由浅入深3 的表述吻合。
StateBag-->bag(HybridDictionary)-->list(ListDictionary)-->head(ListDictionary/DictionaryNode)-->key(string)
到这里可以知道为什么托管堆上会有那么多StateBag、HybridDictionary、ListDictionary、Dictionary/DictionaryNode、StateItem、string对象了。
至于其他的,等下回了。
-----------------------------------------
2009/4/1
-----------------------------------------
近日在调查一个高内存+高cpu的处理,昨日终于找到症结所在。
先前展示了ListDictionary类的内部结构,key/value键值对以单向链表形式存放。那HashTable内部结构又是什么样的呢,带着这个疑问我继续了探查。
!dumpheap -mt 0x79bab9b4
Loaded Son of Strike data table version 5 from "C:"WINDOWS"Microsoft.NET"Framework"v1.1.4322"mscorwks.dll"
Loading the heap objects into our cache.
Address MT Size Gen
0x014c191c 0x79bab9b4 52 2 System.Collections.Hashtable
0x014c22a0 0x79bab9b4 52 2 System.Collections.Hashtable
0x014c2390 0x79bab9b4 52 2 System.Collections.Hashtable
0x014c2564 0x79bab9b4 52 2 System.Collections.Hashtable
0x014c4124 0x79bab9b4 52 2 System.Collections.Hashtable
0x014c64a0 0x79bab9b4 52 2 System.Collections.Hashtable
0x014ca4e8 0x79bab9b4 52 2 System.Collections.Hashtable
0x014cb3d8 0x79bab9b4 52 2 System.Collections.Hashtable
0x014ccfcc 0x79bab9b4 52 2 System.Collections.Hashtable
0x014cd500 0x79bab9b4 52 2 System.Collections.Hashtable
0x014cdbe8 0x79bab9b4 52 2 System.Collections.Hashtable
0x014ce0d8 0x79bab9b4 52 2 System.Collections.Hashtable
从里面随意挑了个Hashtable对象
!do 0x014c191c
Name: System.Collections.Hashtable
MethodTable 0x79bab9b4
EEClass 0x79babaf4
Size 52(0x34) bytes
GC Generation: 2
mdToken: 0x02000118 (c:"windows"microsoft.net"framework"v1.1.4322"mscorlib.dll)
FieldDesc*: 0x79babb58
MT Field Offset Type Attr Value Name
0x79bab9b4 0x400039e 0x4 CLASS instance 0x014c1b90 buckets
0x79bab9b4 0x400039f 0x1c System.Int32 instance 2 count
0x79bab9b4 0x40003a0 0x20 System.Int32 instance 0 occupancy
0x79bab9b4 0x40003a1 0x24 System.Int32 instance 16 loadsize
0x79bab9b4 0x40003a2 0x28 System.Single instance 0.720000 loadFactor
0x79bab9b4 0x40003a3 0x2c System.Int32 instance 2 version
0x79bab9b4 0x40003a4 0x8 CLASS instance 0x00000000 keys
0x79bab9b4 0x40003a5 0xc CLASS instance 0x00000000 values
0x79bab9b4 0x40003a6 0x10 CLASS instance 0x00000000 _hcp
0x79bab9b4 0x40003a7 0x14 CLASS instance 0x00000000 _comparer
0x79bab9b4 0x40003a8 0x18 CLASS instance 0x00000000 m_siInfo
0x79bab9b4 0x400039d 0 CLASS shared static primes
>> Domain:Value 0x00155980:0x014c1950 0x00193830:0x014cdc1c <<
count字段说明只有2对key/value键值对,buckets则是接下来要探查的
!do 0x014c1b90
Name: System.Collections.Hashtable/bucket[]
MethodTable 0x011d2970
EEClass 0x011d28ec
Size 288(0x120) bytes
GC Generation: 2
Array: Rank 1, Type VALUETYPE
Element Type: System.Collections.Hashtable/bucket
Content: 23 items
很明显,buckets 是个System.Collections.Hashtable/bucket 数组,对象本身占用288个字节。
*这里有点要注意,content字段说明该数组有23个元素,这与前面count字段的2 看上去有出入,怎么解释呢。
bucket数组占用了23个元素的内存空间,但实际上只用到2个。
dd 0x014c1b90
014c1b90 011d2970 00000017 00000000 00000000
014c1ba0 00000000 00000000 00000000 00000000
014c1bb0 00000000 00000000 00000000 00000000
014c1bc0 00000000 00000000 00000000 00000000
014c1bd0 00000000 00000000 00000000 00000000
014c1be0 00000000 00000000 00000000 00000000
014c1bf0 00000000 00000000 00000000 00000000
014c1c00 00000000 00000000 00000000 00000000
看到这边我傻眼了,按照前面所说应该有2个key/value键值对,怎么这里一个也没有呢...
且慢,前面说到该数组占用288个字节,而这里只显示了128个字节,后面还有呢...
整个是这样子的:
014c1b90 011d2970 00000017 00000000 00000000
014c1ba0 00000000 00000000 00000000 00000000
014c1bb0 00000000 00000000 00000000 00000000
014c1bc0 00000000 00000000 00000000 00000000
014c1bd0 00000000 00000000 00000000 00000000
014c1be0 00000000 00000000 00000000 00000000
014c1bf0 00000000 00000000 00000000 00000000
014c1c00 00000000 00000000 00000000 00000000
014c1c10 00000000 00000000 00000000 00000000
014c1c20 00000000 00000000 00000000 00000000
014c1c30 00000000 014c9c38 014ca1d8 035e2a40
014c1c40 00000000 00000000 00000000 00000000
014c1c50 00000000 00000000 00000000 00000000
014c1c60 00000000 00000000 00000000 00000000
014c1c70 00000000 00000000 00000000 00000000
014c1c80 00000000 00000000 00000000 00000000
014c1c90 00000000 01563530 01563574 040f65bc
014c1ca0 00000000 00000000 00000000 80000000
再次看得一头雾水,网上看了篇回复才明白。bucket对象占用12字节,也就是这里的3列。第一列是key,第二列是指向Value对象的指针,第三列没说(囧)...
探查了下第一对
!do 014c9c38
Name: System.RuntimeType
MethodTable 0x79ba98b8
EEClass 0x79ba9e90
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020000c4 (c:"windows"microsoft.net"framework"v1.1.4322"mscorlib.dll)
FieldDesc*: 0x79ba9ef4
MT Field Offset Type Attr Value Name
0x79ba6184 0x400025d 0x4 CLASS instance 0x00000000 m_cachedData
0x79ba86d0 0x400025f 0 CLASS shared static FilterAttribute
>> Domain:Value 0x00155980:0x014c223c 0x00193830:0x014cfb0c <<
0x79ba86d0 0x4000260 0x4 CLASS shared static FilterName
>> Domain:Value 0x00155980:0x014c2258 0x00193830:0x014cfb28 <<
0x79ba86d0 0x4000261 0x8 CLASS shared static FilterNameIgnoreCase
>> Domain:Value 0x00155980:0x014c2274 0x00193830:0x014cfb44 <<
0x79ba86d0 0x4000262 0xc CLASS shared static Missing
>> Domain:Value 0x00155980:0x014c21e4 0x00193830:0x014cfae4 <<
0x79ba86d0 0x4000263 0x24 System.Char shared static Delimiter
>> Domain:Value 0x00155980:0x2e 0x00193830:0x2e <<
0x79ba86d0 0x4000264 0x10 CLASS shared static EmptyTypes
>> Domain:Value 0x00155980:0x014c21f0 0x00193830:0x014cfaf0 <<
0x79ba86d0 0x4000265 0x14 CLASS shared static defaultBinder
>> Domain:Value 0x00155980:0x014c9dac 0x00193830:0x0150d118 <<
0x79ba86d0 0x4000266 0x18 CLASS shared static valueType
>> Domain:Value 0x00155980:0x014c2200 0x00193830:0x014c2200 <<
0x79ba86d0 0x4000267 0x1c CLASS shared static enumType
>> Domain:Value 0x00155980:0x014c2210 0x00193830:0x014c2210 <<
0x79ba86d0 0x4000268 0x20 CLASS shared static objectType
>> Domain:Value 0x00155980:0x014c2220 0x00193830:0x014c2220 <<
0x79ba98b8 0x4000273 0x8 System.Int32 instance 56502848 _pData
0x79ba98b8 0x4000274 0 CLASS shared static valueType
>> Domain:Value 0x00155980:0x014c2200 0x00193830:0x014c2200 <<
0x79ba98b8 0x4000275 0x4 CLASS shared static s_ForwardCallBinder
>> Domain:Value 0x00155980:0x00000000 0x00193830:0x00000000 <<
!do 014ca1d8
Name: System.Security.PermissionToken
MethodTable 0x79bae5f0
EEClass 0x79bae648
Size 16(0x10) bytes
GC Generation: 2
mdToken: 0x020003cc (c:"windows"microsoft.net"framework"v1.1.4322"mscorlib.dll)
FieldDesc*: 0x79bae6ac
MT Field Offset Type Attr Value Name
0x79bae5f0 0x400111e 0x4 System.Int32 instance 9 m_index
0x79bae5f0 0x400111f 0x8 System.Boolean instance 1 m_isUnrestricted
0x79bae5f0 0x400111d 0 CLASS shared static s_theTokenFactory
>> Domain:Value 0x00155980:0x014c18bc 0x00193830:0x014c18bc <<
0x79bae5f0 0x4001120 0x4 CLASS shared static s_reflectPerm
>> Domain:Value 0x00155980:0x00000000 0x00193830:0x00000000 <<
对Hashtable 内部结构的探查就到这里吧。这里贴篇.Net类库中实现的HashTable,有兴趣的朋友可以继续深入
参考: