C#+无unsafe的非托管大数组(large unmanaged array in c# without 'unsafe' keyword)

C#+无unsafe的非托管大数组(large unmanaged array in c# without 'unsafe' keyword)

+BIT祝威+悄悄在此留下版了个权的信息说：

C#申请一个大数组(Use a large array in C#)

在C#里，有时候我需要能够申请一个很大的数组、使用之、然后立即释放其占用的内存。

Sometimes I need to allocate a large array, use it and then release its memory space immediately.

由于在C#里提供的 int[] array = new int[1000000]; 这样的数组，其内存释放很难由程序员完全控制，在申请一个大数组后，程序可能会变得很慢。

If I use something like  int[] array = new int[1000000]; , it will be difficult to release its memory space by programmer and the app probably runs slower and slower.

特别是在C#+OpenGL编程中，我在使用VAO/VBO时十分需要设计一个非托管的数组，比如在glBufferData时我希望可以使用下面的glBufferData：

Specially in C#+OpenGL routines when I'm using VAO/VBO, I need an unmanaged array for glBufferData:

 1         /// <summary>
 2         /// 设置当前VBO的数据。
 3         /// </summary>
 4         /// <param name="target"></param>
 5         /// <param name="data"></param>
 6         /// <param name="usage"></param>
 7         public static void glBufferData(uint target, UnmanagedArrayBase data, uint usage)
 8         {
 9             GetDelegateFor<glBufferData>()((uint)target,
10                 data.ByteLength, // 使用非托管数组
11                 data.Header, // 使用非托管数组
12                 (uint)usage);
13         }
14         // ...
15         // glBufferData的声明
16         private delegate void glBufferData(uint target, int size, IntPtr data, uint usage);

而在指定VBO的数据时，可能是float、vec3等等类型：

And the content in VBO can be float, vec3 and any other structs.

 1         /// <summary>
 2         /// 金字塔的posotion array.
 3         /// </summary>
 4         static vec3[] positions = new vec3[]
 5         {
 6             new vec3(0.0f, 1.0f, 0.0f),
 7             new vec3(-1.0f, -1.0f, 1.0f),
 8             // ...
 9             new vec3(-1.0f, -1.0f, 1.0f),   
10         };            
11 //  Create a vertex buffer for the vertex data.
12             {
13                 uint[] ids = new uint[1];
14                 GL.GenBuffers(1, ids);
15                 GL.BindBuffer(GL.GL_ARRAY_BUFFER, ids[0]);
16                 // 使用vec3作为泛型的非托管数组的参数
17                 UnmanagedArray<vec3> positionArray = new UnmanagedArray<vec3>(positions.Length);
18                 for (int i = 0; i < positions.Length; i++)
19                 {
20                     // 使用this[i]这样的索引方式来读写非托管数组的元素
21                     positionArray[i] = positions[i];
22                 }
23                 GL.BufferData(BufferDataTarget.ArrayBuffer, positionArray, BufferDataUsage.StaticDraw);
24                 GL.VertexAttribPointer(positionLocation, 3, GL.GL_FLOAT, false, 0, IntPtr.Zero);
25                 GL.EnableVertexAttribArray(positionLocation);
26             }

 

+BIT祝威+悄悄在此留下版了个权的信息说：

UnmanagedArray<T>

所以我设计了这样一个非托管的数组类型：无unsafe，可接收任何struct类型作为泛型参数，可随时释放内存。

So I designed this UnmangedArray<T> : no 'unsafe' keyword, takes any struct as generic parameter, can be released anytime you want.

  1     /// <summary>
  2     /// 元素类型为sbyte, byte, char, short, ushort, int, uint, long, ulong, float, double, decimal, bool或其它struct的非托管数组。
  3     /// <para>不能使用enum类型作为T。</para>
  4     /// </summary>
  5     /// <typeparam name="T">sbyte, byte, char, short, ushort, int, uint, long, ulong, float, double, decimal, bool或其它struct, 不能使用enum类型作为T。</typeparam>
  6     public class UnmanagedArray<T> : UnmanagedArrayBase where T : struct
  7     {
  8 
  9         /// <summary>
 10         ///元素类型为sbyte, byte, char, short, ushort, int, uint, long, ulong, float, double, decimal, bool或其它struct的非托管数组。 
 11         /// </summary>
 12         /// <param name="count"></param>
 13         [MethodImpl(MethodImplOptions.Synchronized)]
 14         public UnmanagedArray(int count)
 15             : base(count, Marshal.SizeOf(typeof(T)))
 16         {
 17         }
 18      
 19         /// <summary>
 20         /// 获取或设置索引为<paramref name="index"/>的元素。
 21         /// </summary>
 22         /// <param name="index"></param>
 23         /// <returns></returns>
 24         public T this[int index]
 25         {
 26             get
 27             {
 28                 if (index < 0 || index >= this.Count)
 29                     throw new IndexOutOfRangeException("index of UnmanagedArray is out of range");
 30 
 31                 var pItem = this.Header + (index * elementSize);
 32                 //var obj = Marshal.PtrToStructure(pItem, typeof(T));
 33                 //T result = (T)obj;
 34                 T result = Marshal.PtrToStructure<T>(pItem);// works in .net 4.5.1
 35                 return result;
 36             }
 37             set
 38             {
 39                 if (index < 0 || index >= this.Count)
 40                     throw new IndexOutOfRangeException("index of UnmanagedArray is out of range");
 41                 
 42                 var pItem = this.Header + (index * elementSize);
 43                 //Marshal.StructureToPtr(value, pItem, true);
 44                 Marshal.StructureToPtr<T>(value, pItem, true);// works in .net 4.5.1
 45             }
 46         }
 47 
 48         /// <summary>
 49         /// 按索引顺序依次获取各个元素。
 50         /// </summary>
 51         /// <returns></returns>
 52         public IEnumerable<T> GetElements()
 53         {
 54             if (!this.disposed)
 55             {
 56                 for (int i = 0; i < this.Count; i++)
 57                 {
 58                     yield return this[i];
 59                 }
 60             }
 61         }
 62     }
 63 
 64     /// <summary>
 65     /// 非托管数组的基类。
 66     /// </summary>
 67     public abstract class UnmanagedArrayBase : IDisposable
 68     {
 69 
 70         /// <summary>
 71         /// 数组指针。
 72         /// </summary>
 73         public IntPtr Header { get; private set; }
 74 
 75         /// <summary>
 76         /// 元素数目。
 77         /// </summary>
 78         public int Count { get; private set; }
 79 
 80         /// <summary>
 81         /// 单个元素的字节数。
 82         /// </summary>
 83         protected int elementSize;
 84 
 85         /// <summary>
 86         /// 申请到的字节数。（元素数目 * 单个元素的字节数）。
 87         /// </summary>
 88         public int ByteLength
 89         {
 90             get { return this.Count * this.elementSize; }
 91         }
 92 
 93            
 94         /// <summary>
 95         /// 非托管数组。
 96         /// </summary>
 97         /// <param name="elementCount">元素数目。</param>
 98         /// <param name="elementSize">单个元素的字节数。</param>
 99         [MethodImpl(MethodImplOptions.Synchronized)]
100         protected UnmanagedArrayBase(int elementCount, int elementSize)
101         {
102             this.Count = elementCount;
103             this.elementSize = elementSize;
104 
105             int memSize = elementCount * elementSize;
106             this.Header = Marshal.AllocHGlobal(memSize);
107 
108             allocatedArrays.Add(this);
109         }
110 
111         private static readonly List<IDisposable> allocatedArrays = new List<IDisposable>();
112 
113         /// <summary>
114         /// 立即释放所有<see cref="UnmanagedArray"/>。
115         /// </summary>
116         [MethodImpl(MethodImplOptions.Synchronized)]
117         public static void FreeAll()
118         {
119             foreach (var item in allocatedArrays)
120             {
121                 item.Dispose();
122             }
123             allocatedArrays.Clear();
124         }
125 
126         ~UnmanagedArrayBase()
127         {
128             Dispose();
129         }
130 
131         #region IDisposable Members
132 
133         /// <summary>
134         /// Internal variable which checks if Dispose has already been called
135         /// </summary>
136         protected Boolean disposed;
137 
138         /// <summary>
139         /// Releases unmanaged and - optionally - managed resources
140         /// </summary>
141         /// <param name="disposing"><c>true</c> to release both managed and unmanaged resources; <c>false</c> to release only unmanaged resources.</param>
142         protected void Dispose(Boolean disposing)
143         {
144             if (disposed)
145             {
146                 return;
147             }
148 
149             if (disposing)
150             {
151                 //Managed cleanup code here, while managed refs still valid
152             }
153             //Unmanaged cleanup code here
154             IntPtr ptr = this.Header;
155 
156             if (ptr != IntPtr.Zero)
157             {
158                 this.Count = 0;
159                 this.Header = IntPtr.Zero;
160                 Marshal.FreeHGlobal(ptr);
161             }
162 
163             disposed = true;
164         }
165 
166         /// <summary>
167         /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
168         /// </summary>
169         public void Dispose()
170         {
171             this.Dispose(true);
172             GC.SuppressFinalize(this);
173         }
174 
175         #endregion
176                 
177     }

UnmanagedArray

 

+BIT祝威+悄悄在此留下版了个权的信息说：

如何使用(How to use)

UnmanagedArray<T>使用方式十分简单，就像一个普通的数组一样：

Using UnamangedAray<T> is just like a normal array(int[], vec3[], etc.):

 1         internal static void TypicalScene()
 2         {
 3             const int count = 100;
 4 
 5             // 测试float类型
 6             var floatArray = new UnmanagedArray<float>(count);
 7             for (int i = 0; i < count; i++)
 8             {
 9                 floatArray[i] = i;
10             }
11             for (int i = 0; i < count; i++)
12             {
13                 var item = floatArray[i];
14                 if (item != i)
15                 { throw new Exception(); }
16             }
17 
18             // 测试int类型
19             var intArray = new UnmanagedArray<int>(count);
20             for (int i = 0; i < count; i++)
21             {
22                 intArray[i] = i;
23             }
24             for (int i = 0; i < count; i++)
25             {
26                 var item = intArray[i];
27                 if (item != i)
28                 { throw new Exception(); }
29             }
30 
31             // 测试bool类型
32             var boolArray = new UnmanagedArray<bool>(count);
33             for (int i = 0; i < count; i++)
34             {
35                 boolArray[i] = i % 2 == 0;
36             }
37             for (int i = 0; i < count; i++)
38             {
39                 var item = boolArray[i];
40                 if (item != (i % 2 == 0))
41                 { throw new Exception(); }
42             }
43 
44             // 测试vec3类型
45             var vec3Array = new UnmanagedArray<vec3>(count);
46             for (int i = 0; i < count; i++)
47             {
48                 vec3Array[i] = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
49             }
50             for (int i = 0; i < count; i++)
51             {
52                 var item = vec3Array[i];
53                 var old = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
54                 if (item.x != old.x || item.y != old.y || item.z != old.z)
55                 { throw new Exception(); }
56             }
57 
58             // 测试foreach
59             foreach (var item in vec3Array.GetElements())
60             {
61                 Console.WriteLine(item);
62             }
63 
64             // 释放此数组占用的内存，这之后就不能再使用vec3Array了。
65             vec3Array.Dispose();
66 
67             // 立即释放所有非托管数组占用的内存，这之后就不能再使用上面申请的数组了。
68             UnmanagedArrayBase.FreeAll();
69         }

快速读写UnmanagedArray<T>

UnmanagedArrayHelper

由于很多时候需要申请和使用很大的UnmanagedArray<T>，直接使用this[index]索引方式速度会偏慢，所以我添加了几个辅助方法，专门解决快速读写UnmanagedArray<T>的问题。

 1     public static class UnmanagedArrayHelper
 2     {
 3         ///// <summary>
 4         ///// 错误    1    无法获取托管类型(“T”)的地址和大小，或无法声明指向它的指针
 5         ///// </summary>
 6         ///// <typeparam name="T"></typeparam>
 7         ///// <param name="array"></param>
 8         ///// <returns></returns>
 9         //public static unsafe T* FirstElement<T>(this UnmanagedArray<T> array) where T : struct
10         //{
11         //    var header = (void*)array.Header;
12         //    return (T*)header;
13         //}
14 
15         /// <summary>
16         /// 获取非托管数组的第一个元素的地址。
17         /// </summary>
18         /// <param name="array"></param>
19         /// <returns></returns>
20         public static unsafe void* FirstElement(this UnmanagedArrayBase array)
21         {
22             var header = (void*)array.Header;
23 
24             return header;
25         }
26 
27         public static unsafe void* LastElement(this UnmanagedArrayBase array)
28         {
29             var last = (void*)(array.Header + (array.ByteLength - array.ByteLength / array.Length));
30 
31             return last;
32         }
33 
34         /// <summary>
35         /// 获取非托管数组的最后一个元素的地址再向后一个单位的地址。
36         /// </summary>
37         /// <param name="array"></param>
38         /// <returns></returns>
39         public static unsafe void* TailAddress(this UnmanagedArrayBase array)
40         {
41             var tail = (void*)(array.Header + array.ByteLength);
42 
43             return tail;
44         }
45     }

如何使用

这个类型实现了3个扩展方法，可以获取UnmanagedArray<T>的第一个元素的位置、最后一个元素的位置、最后一个元素+1的位置。用这种unsafe的方法可以实现C语言一样的读写速度。

下面是一个例子。用unsafe的方式读写UnmanagedArray<T>，速度比this[index]方式快10到70倍。

 1         public static void TypicalScene()
 2         {
 3             int length = 1000000;
 4             UnmanagedArray<int> array = new UnmanagedArray<int>(length);
 5             UnmanagedArray<int> array2 = new UnmanagedArray<int>(length);
 6 
 7             long tick = DateTime.Now.Ticks;
 8             for (int i = 0; i < length; i++)
 9             {
10                 array[i] = i;
11             }
12             long totalTicks = DateTime.Now.Ticks - tick;
13 
14             tick = DateTime.Now.Ticks;
15             unsafe
16             {
17                 int* header = (int*)array2.FirstElement();
18                 int* last = (int*)array2.LastElement();
19                 int* tailAddress = (int*)array2.TailAddress();
20                 int value = 0;
21                 for (int* ptr = header; ptr <= last/*or: ptr < tailAddress*/; ptr++)
22                 {
23                     *ptr = value++;
24                 }
25             }
26             long totalTicks2 = DateTime.Now.Ticks - tick;
27             Console.WriteLine("ticks: {0}, {1}", totalTicks, totalTicks2);// unsafe method works faster.
28 
29             for (int i = 0; i < length; i++)
30             {
31                 if (array[i] != i)
32                 {
33                     Console.WriteLine("something wrong here");
34                 }
35                 if (array2[i] != i)
36                 {
37                     Console.WriteLine("something wrong here");
38                 }
39             }
40 
41             array.Dispose();
42             array2.Dispose();
43         }

 

 1                 unsafe
 2                 {
 3                     vec3* header = (vec3*)vec3Array.FirstElement();
 4                     vec3* last = (vec3*)vec3Array.LastElement();
 5                     vec3* tailAddress = (vec3*)vec3Array.TailAddress();
 6                     int i = 0;
 7                     for (vec3* ptr = header; ptr <= last/*or: ptr < tailAddress*/; ptr++)
 8                     {
 9                         *ptr = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
10                         i++;
11                     }
12                     i = 0;
13                     for (vec3* ptr = header; ptr <= last/*or: ptr < tailAddress*/; ptr++, i++)
14                     {
15                         var item = *ptr;
16                         var old = new vec3(i * 3 + 0, i * 3 + 1, i * 3 + 2);
17                         if (item.x != old.x || item.y != old.y || item.z != old.z)
18                         { throw new Exception(); }
19                     }
20                 }

2015-08-25

用StructLayout和MarshalAs支持复杂的struct

在OpenGL中我需要用UnmanagedArray<mat4>，其中mat4定义如下：

 1     /// <summary>
 2     /// Represents a 4x4 matrix.
 3     /// </summary>
 4     [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4 * 4)]
 5     public struct mat4
 6     {
 7         /// <summary>
 8         /// Gets or sets the <see cref="vec4"/> column at the specified index.
 9         /// </summary>
10         /// <value>
11         /// The <see cref="vec4"/> column.
12         /// </value>
13         /// <param name="column">The column index.</param>
14         /// <returns>The column at index <paramref name="column"/>.</returns>
15         public vec4 this[int column]
16         {
17             get { return cols[column]; }
18             set { cols[column] = value; }
19         }
20 
21         /// <summary>
22         /// Gets or sets the element at <paramref name="column"/> and <paramref name="row"/>.
23         /// </summary>
24         /// <value>
25         /// The element at <paramref name="column"/> and <paramref name="row"/>.
26         /// </value>
27         /// <param name="column">The column index.</param>
28         /// <param name="row">The row index.</param>
29         /// <returns>
30         /// The element at <paramref name="column"/> and <paramref name="row"/>.
31         /// </returns>
32         public float this[int column, int row]
33         {
34             get { return cols[column][row]; }
35             set { cols[column][row] = value; }
36         }
37 
38         /// <summary>
39         /// The columms of the matrix.
40         /// </summary>
41         [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
42         private vec4[] cols;
43     }
44 
45     /// <summary>
46     /// Represents a four dimensional vector.
47     /// </summary>
48     [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4)]
49     public struct vec4
50     {
51         public float x;
52         public float y;
53         public float z;
54         public float w;
55 
56         public float this[int index]
57         {
58             get
59             {
60                 if (index == 0) return x;
61                 else if (index == 1) return y;
62                 else if (index == 2) return z;
63                 else if (index == 3) return w;
64                 else throw new Exception("Out of range.");
65             }
66             set
67             {
68                 if (index == 0) x = value;
69                 else if (index == 1) y = value;
70                 else if (index == 2) z = value;
71                 else if (index == 3) w = value;
72                 else throw new Exception("Out of range.");
73             }
74         }
75     }

mat4

注意：UnmanagedArray<T>支持的struct，T的大小必须是确定的。所以在mat4里我们用 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4 * 4)] 指定mat4的大小为4个 vec4 * 4个 float * 4个字节(每个float) = 64字节，并且在 private vec4[] cols; 上用 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)] 规定了cols的元素数必须是4。之后在 vec4 上的 [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Size = 4 * 4)] 不写也可以，因为vec4只有4个简单的float字段，不含复杂类型。

下面是测试用例。

 1             mat4 matrix = glm.scale(mat4.identity(), new vec3(2, 3, 4));
 2 
 3             var size = Marshal.SizeOf(typeof(mat4));
 4             size = Marshal.SizeOf(matrix);
 5 
 6             UnmanagedArray<mat4> array = new UnmanagedArray<mat4>(1);
 7             array[0] = matrix;
 8 
 9             mat4 newMatirx = array[0]; // newMatrix should be equal to matrix
10 
11             array.Dispose();

如果matrix和newMatrix相等，就说明上述Attribute配置正确了。