MDL (memory descriptor list)

MDL (memory descriptor list)
2009年11月04日 星期三 下午 02:26

 

 

MDL (memory descriptor list):由于WDK文档中关于MDL的描述不算多,它只有在需要的时候才会提出一些操作MDL的function。今天遇到这个时,不知所措,搜集相关资料来解释一下MDL。以下是MDL的资料结构:

typedef struct _MDL {
    struct _MDL *Next;
    CSHORT Size;
    CSHORT MdlFlags;
    struct _EPROCESS *Process;
    PVOID MappedSystemVa;
    PVOID StartVa;
    ULONG ByteCount;
    ULONG ByteOffset;
} MDL, *PMDL;

MDL只是一个对物理内存的描述,但是因为系统跟Driver都是使用虚拟内存,所以MDL就是把虚拟内存『映射』到物理内存(from DDK)。这样讲是很模糊的,其实MDL的作用很简单:当Driver要存取某段内存位置时,确保MDL所描述的内存位置不会引起page fault。为什么?因为『虚拟内存』的关系。 Driver存取的是虚拟内存,所以当系统丢给Driver一块虚拟内存位置,而Driver如何知道这块虚拟内存所代表的资料到底是在RAM还是Disk(被paged out)内?

在NDIS里面,Driver是不需要去关心系统丢给Driver的内存到底是如何,反正一定是在RAM里面就是了。但是在I/O Control内的Direct I/O method里面,这可不一定,因为这个内存位置可能是指到某个应用程式的虚拟内存,而应用程式的虚拟内存是非常有可能被paged out。所以Driver得呼叫MmGetSystemAddressForMdlSafe()来锁定这块内存(系统得把把资料由Disk载入到RAM)。

写段代码就可以证明这一点:

 


PVOID    buf = NULL;
PMDL     mdl = NULL;
NDIS_STATUS    ndis_status;

ndis_status = NdisAllocateMemoryWithTag(&buf, 1000, 0x1111L);
if (ndis_status != NDIS_STATUS_SUCCESS){
    DbgPrint("NetVMini : Can not allocate test Memory for MDL.\n");
} else {
    mdl = IoAllocateMdl(buf, 1000, FALSE, FALSE, NULL);
    if (mdl != NULL) {
        MmBuildMdlForNonPagedPool(mdl);
        DbgPrint("buf = 0x%p\n", buf);
        DbgPrint("MDL.Size = 0x%X\n", mdl->Size);
        DbgPrint("MDL.MdlFlags = 0x%X\n", mdl->MdlFlags);
        DbgPrint("MDL.MappedSystemVa = 0x%p\n", mdl->MappedSystemVa);
        DbgPrint("MDL.StartVa = 0x%p\n", mdl->StartVa);
        DbgPrint("MDL.ByteCount = 0x%X\n", mdl->ByteCount);
        DbgPrint("MDL.ByteOffset = 0x%X\n", mdl->ByteOffset);
        IoFreeMdl(mdl);
         mdl = NULL;
     }
    NdisFreeMemory(buf, 1000, 0);
}

这个程式只是配置一块non-paged记忆体,然后用一个MDL来描述它。

下面是程式的结果:

00000008    0.00103058    buf = 0x85322008   
00000009    0.00104175    MDL.Size = 0x20   
00000010    0.00104985    MDL.MdlFlags = 0xC   
00000011    0.00106243    MDL.MappedSystemVa = 0x85322008   
00000012    0.00107919    MDL.StartVa = 0x85322000   
00000013    0.00109120    MDL.ByteCount = 0x3E8   
00000014    0.00110321    MDL.ByteOffset = 0x8   

buf = MappedSystemVa = StartVa + ByteOffset. Eureka!

看起来直接用NdisAllocateMemoryWithTag()所得到指标还比较好? MDL还是有好处的啦,至少它可以串起一堆MDL,当系统需要给Driver一堆内存区块,而这些内存区块全都是不连续的,那MDL就会很好用(NDIS_BUFFER就是如此)。而且在NDIS 6.0里面,NET_BUFFER只能使用MDL。

跟MDL相关的函式有:
PMDL IoAllocateMdl(
    IN PVOID VirtualAddress,
    IN ULONG Length,
    IN BOOLEAN SecondaryBuffer,
    IN BOOLEAN ChargeQuota,
    IN OUT PIRP Irp OPTIONAL
     );

VirtualAddress:这个MDL所描述的虚拟记忆体位置。

Length:这个MDL所描述的记忆体大小。

DDK里面特别强调,如果Driver希望建立的MDL是映射到Driver自己配置的Non-Paged记忆体的话,Driver还得呼叫MmBuildMdlForNonPagedPool()。这是因为IoAllocateMdl()只有配置记忆体,但是并没有Build MDL(好模糊的说法)。 NDIS 6.0有提供NdisAllocateMdl(),它提供一次呵成的服务。

 

VOID MmBuildMdlForNonPagedPool(
     IN OUT PMDL MemoryDescriptorList
     );

The MmBuildMdlForNonPagedPool routine receives an MDL that specifies a virtual memory buffer in nonpaged pool, and updates it to describe the underlying physical pages.

The MDL virtual address that is input must be within the nonpaged portion of system space, such as memory allocated by ExAllocatePoolWithTag with PoolType = NonPagedPool.

VOID IoBuildPartialMdl(
     IN PMDL SourceMdl,
     IN OUT PMDL TargetMdl,
     IN PVOID VirtualAddress,
     IN ULONG Length
     );

VOID IoFreeMdl(
     IN PMDL Mdl
     );

ULONG MmGetMdlByteCount(IN PMDL Mdl){
   return Mdl->ByteCount;
}

返回这个MDL的大小。

ULONG MmGetMdlByteOffset(IN PMDL Mdl) {
   return Mdl->ByteOffset;
}

返回这个MDL的StarVa偏移。

PVOID MmGetMdlVirtualAddress(IN PMDL Mdl) {
   return ((PVOID) ((PCHAR) ((Mdl)->StartVa) + (Mdl)->ByteOffset));
}

取得MDL的虚拟内存位置。 DDK特别讲明,Lower-Level Driver不可以直接把这个Address拿来使用,因为这有可能是user-space的内存位置。因此,Driver必须呼叫MmGetSystemAddressForMdlSafe()来取得并锁定这个Address所对应到的system-space的内存位置。

PVOID MmGetSystemAddressForMdlSafe(IN PMDL Mdl, IN MM_PAGE_PRIORITY Priority) {
   if (Mdl->MdlFlags & (MDL_MAPPED_TO_SYSTEM_VA | MDL_SOURCE_IS_NONPAGED_POOL)) {
       return Mdl->MappedSystemVa;
   } else {
       return MmMapLockedPagesSpecifyCache(Mdl, KernelMode, MmCached, NULL, FALSE, Priority);
   }
}

这个函式是个Macro。代码很简单,如果这个MDL已经被锁定并映射,直接返回MappedSystemVa,否则就锁定并映射它。

VOID MmInitializeMdl(IN PMDL MemoryDescriptorList, IN PVOID BaseVa, IN SIZE_T Length){
   MemoryDescriptorList->Next = (PMDL) NULL;
   MemoryDescriptorList->Size = (CSHORT)(sizeof(MDL) + sizeof(PFN_NUMBER) * ADDRESS_AND_SIZE_TO_SPAN_PAGES((BaseVa), (Length))));
   MemoryDescriptorList->MdlFlags = 0;
   MemoryDescriptorList->StartVa = (PVOID) PAGE_ALIGN((BaseVa));
   MemoryDescriptorList->ByteOffset = BYTE_OFFSET((BaseVa));
   MemoryDescriptorList->ByteCount = (ULONG)(Length);
}

这段代码码已经对MDL说明的很清楚了。 MDL的目的是要串起多个分散的内存块,而MnInitializeMdl()只是串起第一块内存位置。

VOID MmPrepareMdlForReuse(IN PMDL MDL) {
   if ((MDL->MdlFlags & MDL_PARTIAL_HAS_BEEN_MAPPED) != 0) {
       MmUnmapLockedPages( MDL->MappedSystemVa, MDL );
   }
}

Very few drivers call this routine (DDK says).

VOID MmProbeAndLockPages(
    IN OUT PMDL MemoryDescriptorList,
    IN KPROCESSOR_MODE AccessMode,
    IN LOCK_OPERATION Operation
     );

这似乎是用在Direct IO method上。它会映射并锁定MDL所描述的虚拟内存。

ULONG MmSizeOfMdl(
    IN PVOID Base,
    IN SIZE_T Length
     );

The MmSizeOfMdl routine returns the number of bytes to allocate for an MDL describing a given address range.

VOID MmUnmapLockedPages(
    IN PVOID BaseAddress,
    IN PMDL MemoryDescriptorList
     );

解除 MmProbeAndLockPages() 或 MmMapLockedPagesSpecifyCache() 所造成的影响。

 

 

What Is Really in That MDL?

Updated: January 11, 2008

 

A memory descriptor list (MDL) is a system-defined structure that describes a buffer by a set of physical addresses. A driver that performs direct I/O receives a pointer to an MDL from the I/O manager, and reads and writes data through the MDL. Some drivers also use MDLs when they perform direct I/O to satisfy a device I/O control request.

Driver writers should make no assumptions about the order or contents of pages that are described by an MDL. Drivers must not rely on the value of the data at any location that is referenced by an MDL and should not directly dereference a memory location to get the data. If the MDL describes a buffer for a direct I/O operation, the application that issued the I/O request might also have mapped a view of the same pages into its address space. If so, the application and the driver could try to modify the data at the same time, which causes errors.

Furthermore, in some cases the locations in the MDL do not reference the same physical pages that the memory manager retains. When the Microsoft Windows memory manager constructs an MDL for a device read, it locks physical pages to use for the transfer target. However, it is solely up to the memory manager to determine which pages to keep and which (if any) to discard. Why does the memory manager read data into these pages and then discard them? Because doing I/O in larger clusters provides better performance.

For example, in the following figure, the file offsets and VAs that correspond to pages A, Y, Z, and B are logically contiguous although the physical pages themselves are not necessarily contiguous. Pages A and B are nonresident, so the memory manager must read them. Pages Y and Z are already resident in memory, so it is not necessary to read them. (In fact, they might already have been modified since they were last read in from their backing store, in which case it would be a serious error to overwrite their contents.) However, reading pages A and B in a single operation is more efficient than performing one read for page A and a second read for page B. Therefore, the memory manager issues a single read request that comprises all four pages (A, Y, Z, and B) from the backing store. Such a read request includes as many pages as make sense to read, based on the amount of available memory, the current system usage, and so on.

MDL (memory descriptor list)_第1张图片

When the memory manager builds the memory descriptor list (MDL) that describes the request, it supplies valid pointers to pages A and B. However, the entries for pages Y and Z point to a single system-wide dummy page X. The memory manager can fill the dummy page X with the potentially stale data from the backing store because it does not make X visible. However, if a component accesses the Y and Z offsets in the MDL, it sees the dummy page X instead of Y and Z.

The memory manager can represent any number of discarded pages as a single page, and that page can be embedded multiple times in the same MDL or even in multiple concurrent MDLs that are being used for different drivers. Consequently, the contents of the locations that represent the discarded pages can change at any time.

Drivers that perform decryption or calculate checksums that are based on the value of data on the pages that the MDL maps must not dereference pointers from the system-supplied MDL to access the data. Instead, to ensure correct operation, such a driver should create a temporary MDL that is based on the system-supplied MDL that the driver received from the I/O manager. To create the temporary MDL:

1.

Call MmGetMdlVirtualAddress and MmGetMdlByteCount to get the base virtual address and length of the system-supplied MDL.

2.

Call ExAllocatePoolWithTag with PoolType=NonPagedPool to allocate a buffer from nonpaged pool. Specify a buffer size that is the length of the system-supplied MDL, rounded up to a page boundary.

3.

Call IoAllocateMdl to allocate an MDL with the base virtual address and length of the pool buffer that was created in step 2.

4.

Call MmBuildMdlForNonpagedPool to update the temporary MDL so that it describes the underlying physical pages of the pool buffer from step 2.

The driver should pass this temporary MDL in calls that read data from its hardware and then use the values of the data that was described by the temporary MDL in any required manipulations. By calling MmBuildMdlForNonPagedPool to update the temporary MDL, the driver ensures that the temporary MDL cannot possibly have any temporary pages, thus protecting it from any changes to the contents of the pages. In this way, even if the system MDL contains (potentially duplicated) pages that will be discarded, the driver avoids examining unstable contents. When the driver completes its manipulations, it should copy the changed data from the temporary MDL back to the system-supplied MDL by using RtlCopyMemory within a try/except or try/finally block.

Drivers that use MDLs as part of a typical I/O operation without accessing the data on the underlying pages do not need to create a temporary MDL. Internally, the memory manager keeps track of all the pages that are resident and how each is mapped. When a driver passes an MDL to a system service routine to perform I/O, the memory manager ensures that the correct data is used.

What should you do?

Do not assume that the contents of any memory location that is pointed to by an MDL are valid at any given time.

Always double-buffer data in system-supplied MDLs if your driver depends on the value of the data.

你可能感兴趣的:(职场,mdl,休闲)