【Win32多线程】异步I/O技术(Overlapped I/O),避免使用多线程
讨论：1.激发的文件handles
2.激发的event对象
3.异步过程调用
4.I/O completion ports(重要)适用于高负载服务器
----Win32文件操作函数
win32中有上那个基本的函数用来执行I/O,他们是：
CreateFile(),ReadFile(),WriteFile();
没有哪个函数用来关闭文件，只要调用CloseHandle()即可。
CreateFile()可以用来打开各式各样的资源：包括文件，串口和并行口，Named pipes,Console.
它的原型如下：
HANDLE CreateFile(
LPCTSTR lpFileName, // file name
DWORD dwDesiredAccess, // access mode存取模式(读或写)
DWORD dwShareMode, // share mode
LPSECURITY_ATTRIBUTES lpSecurityAttributes, // SD
DWORD dwCreationDisposition, // how to create 如何产生
DWORD dwFlagsAndAttributes, // file attributes 文件属性(使用overlapped I/O的关键)
HANDLE hTemplateFile // handle to template file一个临时文件，将拥有全部的属性拷贝
);
C runtime library不能使用overlapped I/O.Overlapped I/O的基本型式是以ReadFile()和WriteFile()完成的。这两个函数的原型如下：
BOOL ReadFile(
HANDLE hFile, // handle to file欲读取的文件
LPVOID lpBuffer, // data buffer 接收数据的缓冲区
DWORD nNumberOfBytesToRead, // number of bytes to read 欲读取的字节数
LPDWORD lpNumberOfBytesRead, // number of bytes read 实际读取的字节个数的地址
LPOVERLAPPED lpOverlapped // overlapped buffer指针，指向overlapped info
);
BOOL WriteFile(
HANDLE hFile, // handle to file欲写的文件
LPCVOID lpBuffer, // data buffer存储数据的缓冲区
DWORD nNumberOfBytesToWrite, // number of bytes to write欲写入的字节个数
LPDWORD lpNumberOfBytesWritten, // number of bytes written实际写入的字节个数
LPOVERLAPPED lpOverlapped // overlapped buffer 指针，指向overlapped info
);
这两个函数很像C runtime 函数的fread 和fwrite()，差别在于最后一个参数lpOverlapped.如果第六个参数设为
FILE_FLAG_OVERLAPPED,则必须在lpOverlapped参数中提供一个指针，指向一个OVERLAPPED结构。
---OVERLAPPED结构
它执行两个重要的功能，第一，它像一把钥匙，用以识别一个目前正在进行的overlapped操作。第二，它在你和系统之间提供了一个共享区域，参数可以在该区域中双向传递。
typedef struct _OVERLAPPED {
ULONG_PTR Internal;
ULONG_PTR InternalHigh;
DWORD Offset;
DWORD OffsetHigh;
HANDLE hEvent;
} OVERLAPPED;
由于OVERLAPPED结构的生命周期超越ReadFile()和WriteFile()函数。所以把这个结构放在局部变量不是安全的。应该放在heap上。
---被激发的File Handles
最简单的overlapped I/O类型，就是使用它自己文件handle作为同步机制。
首先，用FILE_FLAG_OVERLAPPED告诉Win32你不要使用默认的同步I/O。然后设立一个OVERLAPPED结构，其中内含“I/O请求”的所有必要的参数，并以此来识别这个“ I/O请求”，直到它完成为止。接下来，调用ReadFile()并以OVERLAPPED结构的地址作为最后一个参数。这时候，Win32会在后台处理你的请求。你的程序可以干其他的事情了。
如果需要等待overlapped I/O的执行结果，作为WaitForMultipleObjects()的一部分，在handle数组中加上这个文件的handle.文件handle是一个核心对象。一旦操作完毕即被激发。当完成操作之后，
调用GetOverlappedResult()确定结果如何。
BOOL GetOverlappedResult(
HANDLE hFile, // handle to file, pipe, or device
LPOVERLAPPED lpOverlapped, // overlapped structure 一个指针，指向OVERLAPPED结构
LPDWORD lpNumberOfBytesTransferred, // bytes transferred表示真正被传送的字节
BOOL bWait // wait option是否表示要等待操作完成
);
文件读取示例：
#define WIN32_LEAN_AND_MEAN
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
//
// Constants
//
#define READ_SIZE 512
//
// Function prototypes
//
void CheckOsVersion();
int main()
{
BOOL rc;
HANDLE hFile;
DWORD numread;
OVERLAPPED overlap;
char buf[READ_SIZE];
char szPath[MAX_PATH];
CheckOsVersion();
GetWindowsDirectory(szPath, sizeof(szPath));
strcat(szPath, "\\WINHLP32.EXE");
// Open the file for overlapped reads
hFile = CreateFile( szPath,
GENERIC_READ,
FILE_SHARE_READ|FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
FILE_FLAG_OVERLAPPED,
NULL
);
if (hFile == INVALID_HANDLE_VALUE)
{
printf("Could not open %s\n", szPath);
return -1;
}
// Initialize the OVERLAPPED structure
memset(&overlap, 0, sizeof(overlap));
overlap.Offset = 1500;
// Request the data
rc = ReadFile(
hFile,
buf,
READ_SIZE,
&numread,
&overlap
);
printf("Issued read request\n");
// Was the operation queued?
if (rc)
{
// The data was read successfully
printf("Request was returned immediately\n");
}
else
{
if (GetLastError() == ERROR_IO_PENDING)
{
// We could do something else for awhile here...
printf("Request queued, waiting...\n");
WaitForSingleObject(hFile, INFINITE);//这个调用其实是多余，因为GetOverlappedResult就可以等待 overlapped操作的完
成。
printf("Request completed.\n");
rc = GetOverlappedResult(
hFile,
&overlap,
&numread,
FALSE
);
printf("Result was %d\n", rc);
}
else
{
// We should check for memory and quota
// errors here and retry. See the samples
// IoByEvnt and IoByAPC.
// Something went wrong
printf("Error reading file\n");
}
}
CloseHandle(hFile);
return EXIT_SUCCESS;
}
//
// Make sure we are running under an operating
// system that supports overlapped I/O to files.
//
void CheckOsVersion()
{
OSVERSIONINFO ver;
BOOL bResult;
ver.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
bResult = GetVersionEx((LPOSVERSIONINFO) &ver);
if ( (!bResult) ||
(ver.dwPlatformId != VER_PLATFORM_WIN32_NT) )
{
fprintf(stderr, "IoByFile must be run under Windows NT.\n");
exit(EXIT_FAILURE);
}
}
Q:Overlapped I/O总是异步的执行吗？
虽然你要求overlapped操作，但它并不一定就是overlapped。如果数据已经被放进cache中，或如果操作系统认为它可以很快速的取得那份数据，那么文件操作就会在ReadFile()返回之前完成。而ReadFile()将传回TRUE.这种情况下，文件handle处于激发状态，而对文件的操作可以视作就像overlapped一样。
如果你要求一个文件操作为overlapped，而操作系统吧这个“操作请求”放到队列中等待执行，那么ReadFile()和WriteFile()都会传回FALSE表示失败。你必须调GetLastError()并确定它传回ERROR_IO_PENDING,那意味着“overlapped I/O请求”被放进队列之中等待执行。GetLastError()也可能传回其他的值，例如ERROR_HANDLE_EOF，那么就代表一个错误。
OVERLAPPED结构体有能力处理64位的偏移值，因此面对巨大的文件也不怕。为了等待文件操作的完成，把文件的handle交给WaitForSingleObject().一旦overlapped操作完成，该文件的handle就会成为激发状态。
---被激发的Event对象
用文件的handle作为激发机制，就没办法说出到底哪一个overlapped操作完成了。Win32有一个很好的解决办法。
OVERLAPPED结构中的最后一个栏位，是一个event handle.如果你使用文件handle作为激发对象，那么此栏设置为NULL.当使用event作为激发对象时，则赋值为event对象。系统核心会在overlapped操作
完成的时候，自动将此event对象给激发起来。由于每一个overlapped操作都有它自己独一无二的OVERLAPPED结构。所以每一个结构都有它独一无二的一个event对象，用来代表该操作。
Q:如何为overlapped I/O产生一个event对象？
你所使用的event对象必须是手动重置而非自动重置。如果使用自动重置，就有可能产生race condition,因为系统核心有可能在你有机会等待该event对象之前，先激发它。event对象的激发状态不能够保存的。于是这个event状态将遗失，而你的Wait...()函数永远不返回。但是一个手动重置的event，一旦激发，就一直处于激发状态。
使用event对象搭配overlapped I/O，可以对同一个文件发出多个读取操作和多个写入操作，每一个操作有自己的event对象；然后在调用WaitForMultipleObjects()来等待其中之一完成。
使用Event和overlapped I/O实现文件读取示例:
/* Demonstrates how to use event objects instead of
* file handles to signal multiple outstanding
* overlapped operation on a file.
*/
#define WIN32_LEAN_AND_MEAN
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
#include "MtVerify.h"
//
// Constants
//
#define MAX_REQUESTS 5
#define READ_SIZE 512
#define MAX_TRY_COUNT 5
//
// Function prototypes
//
int QueueRequest(int nIndex, DWORD dwLocation, DWORD dwAmount);
void CheckOsVersion();
//
// Global variables
//
// Need to keep the events in their own array
// so we can wait on them.
HANDLE ghEvents[MAX_REQUESTS];
// Keep track of each individual I/O operation
OVERLAPPED gOverlapped[MAX_REQUESTS];
// Handle to the file of interest.
HANDLE ghFile;
// Need a place to put all this data
char gBuffers[MAX_REQUESTS][READ_SIZE];
int main()
{
int i;
BOOL rc;
char szPath[MAX_PATH];
CheckOsVersion();
GetWindowsDirectory(szPath, sizeof(szPath));
strcat(szPath, "\\WINHLP32.EXE");
// Open the file for overlapped reads
ghFile = CreateFile( szPath,
GENERIC_READ,
FILE_SHARE_READ|FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
FILE_FLAG_OVERLAPPED,
NULL
);
if (ghFile == INVALID_HANDLE_VALUE)
{
printf("Could not open %s\n", szPath);
return -1;
}
for (i=0; i<MAX_REQUESTS; i++)
{
// Read some bytes every few K
QueueRequest(i, i*16384, READ_SIZE);
}
printf("QUEUED!!\n");
// Wait for all the operations to complete.
MTVERIFY( WaitForMultipleObjects(
MAX_REQUESTS, ghEvents, TRUE, INFINITE
) != WAIT_FAILED );
// Describe what just happened.
for (i=0; i<MAX_REQUESTS; i++)
{
DWORD dwNumread;
rc = GetOverlappedResult(
ghFile,
&gOverlapped[i],
&dwNumread,
FALSE
);
printf("Read #%d returned %d. %d bytes were read.\n",
i, rc, dwNumread);
CloseHandle(gOverlapped[i].hEvent);
}
CloseHandle(ghFile);
return EXIT_SUCCESS;
}
/*
* Call ReadFile to start an overlapped request.
* Make sure we handle errors that are recoverable.
* Properly set up the event object for this operation.
*/
int QueueRequest(int nIndex, DWORD dwLocation, DWORD dwAmount)
{
int i;
BOOL rc;
DWORD dwNumread;
DWORD err;
MTVERIFY(
ghEvents[nIndex] = CreateEvent(
NULL, // No security
TRUE, // Manual reset - extremely important!
FALSE, // Initially set Event to non-signaled state
NULL // No name
)
);
gOverlapped[nIndex].hEvent = ghEvents[nIndex];
gOverlapped[nIndex].Offset = dwLocation;
for (i=0; i<MAX_TRY_COUNT; i++)
{
rc = ReadFile(
ghFile,
gBuffers[nIndex],
dwAmount,
&dwNumread,
&gOverlapped[nIndex]
);
// Handle success
if (rc)
{
printf("Read #%d completed immediately.\n", nIndex);
return TRUE;
}
err = GetLastError();
// Handle the error that isn't an error. rc is zero here.
if (err == ERROR_IO_PENDING)
{
// asynchronous i/o is still in progress
printf("Read #%d queued for overlapped I/O.\n", nIndex);
return TRUE;
}
// Handle recoverable error
if ( err == ERROR_INVALID_USER_BUFFER ||
err == ERROR_NOT_ENOUGH_QUOTA ||
err == ERROR_NOT_ENOUGH_MEMORY )
{
Sleep(50); // Wait around and try later
continue;
}
// Give up on fatal error.
break;
}
printf("ReadFile failed.\n");
return -1;
}
//
// Make sure we are running under an operating
// system that supports overlapped I/O to files.
//
void CheckOsVersion()
{
OSVERSIONINFO ver;
BOOL bResult;
ver.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
bResult = GetVersionEx((LPOSVERSIONINFO) &ver);
if ( (!bResult) ||
(ver.dwPlatformId != VER_PLATFORM_WIN32_NT) )
{
fprintf(stderr, "IoByEvnt must be run under Windows NT.\n");
exit(EXIT_FAILURE);
}
}
对于以Remote Access Service连接到一个服务器，现在可以记录这些操作过程，并做成overlapped I/O,然后在住消息循环中使用MsgWaitForMultipleObjects()以便在取得数据后有所反应。
-----------异步过程调用(Asynchronous Procedure Calls,APCs)
使用overlapped I/O配合event对象，会产生两个基础性问题。
第一，使用WaitForMultipleObjects(),你只能够等待最多MAXIMUM_WAIT_OBJECTS个对象。在Win32 SDK中，最大值为64。如果要等待64个以上的对象，就会出现问题。
第二，你必须不断根据“哪一个handle被激发”而计算如果反应。你必须分配表格和WaitForMultipleObjects()的handles数组结合起来。以上两个问题可以用异步过程调用来解决。
只要使用“Ex”版本的ReadFile()和WriteFile()，就可以调用这个机制。这两个函数允许你指定一个额外的参数，是一个callback函数地址。当一个overlapped I/O完成时，系统应该调用该callback函数。这个callback函数称为I/O completion routine.
Q:一个I/O completion routine何时被调用？
你的线程必须在所谓"alertable"状态下才行。如果线程不处于“alertable”状态，那么对I/O completion routine的调用会暂时被保存下来。因此，当一个线程终于进入“alertable”状态时，可能已经有一大堆储备的APCs等待被处理。
如果线程因为下面五个函数处于等待状态，其“alertable”标记被设为TRUE,则该线程就是处于"alertable"状态：
SleepEx(),WaitForSingleObjectEx();WaitForMultipleObjectsEx(),MsgWaitForMultipleObjectsEx();
SignalObjectAndWait();
只有在程序处于“alertable”状态下，APCs才会被调用。
I/O completion routine的形式：
VOID CALLBACK FileIOCompletionRoutine(
DWORD dwErrorCode, // completion code 0表示操作完成，ERROR_HANDLE_EOF表示操作已经到文件尾 端。
DWORD dwNumberOfBytesTransfered, // number of bytes transferred真正被传输的字节数
LPOVERLAPPED lpOverlapped // I/O information buffer
);
Q：如何把一个用户自定义的数据传递给I/O completion routine？
使用APCs时，OVERLAPPED机构体重的hEvent位不需要用来放置一个event handle.hEvent栏位可以由程序员自由使用。最大的用途就是：首先配置一个结构，描述数据来自哪里，或是要对数据进行一些什么操作，然后将hEvent栏位设定为指向该结构。
overlapped I/O与APCs的应用示例：
/* Demonstrates how to use APC's (asynchronous
* procedure calls) instead of signaled objects
* to service multiple outstanding overlapped
* operations on a file.
*/
#define WIN32_LEAN_AND_MEAN
#include <stdio.h>
#include <stdlib.h>
#include <windows.h>
#include "MtVerify.h"
//
// Constants
//
#define MAX_REQUESTS 5
#define READ_SIZE 512
#define MAX_TRY_COUNT 5
//
// Function prototypes
//
void CheckOsVersion();
int QueueRequest(int nIndex, DWORD dwLocation, DWORD dwAmount);
//
// Global variables
//
// Need a single event object so we know when all I/O is finished
HANDLE ghEvent;
// Keep track of each individual I/O operation
OVERLAPPED gOverlapped[MAX_REQUESTS];
// Handle to the file of interest.
HANDLE ghFile;
// Need a place to put all this data
char gBuffers[MAX_REQUESTS][READ_SIZE];
int nCompletionCount;
/*
* I/O Completion routine gets called
* when app is alertable (in WaitForSingleObjectEx)
* and an overlapped I/O operation has completed.
*/
VOID WINAPI FileIOCompletionRoutine(
DWORD dwErrorCode, // completion code
DWORD dwNumberOfBytesTransfered, // number of bytes transferred
LPOVERLAPPED lpOverlapped // pointer to structure with I/O information
)
{
// The event handle is really the user defined data
int nIndex = (int)(lpOverlapped->hEvent);
printf("Read #%d returned %d. %d bytes were read.\n",
nIndex,
dwErrorCode,
dwNumberOfBytesTransfered);
if (++nCompletionCount == MAX_REQUESTS)
SetEvent(ghEvent); // Cause the wait to terminate
}
int main()
{
int i;
char szPath[MAX_PATH];
CheckOsVersion();
// Need to know when to stop
MTVERIFY(
ghEvent = CreateEvent(
NULL, // No security
TRUE, // Manual reset - extremely important!
FALSE, // Initially set Event to non-signaled state
NULL // No name
)
);
GetWindowsDirectory(szPath, sizeof(szPath));
strcat(szPath, "\\WINHLP32.EXE");
// Open the file for overlapped reads
ghFile = CreateFile( szPath,
GENERIC_READ,
FILE_SHARE_READ|FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
FILE_FLAG_OVERLAPPED,
NULL
);
if (ghFile == INVALID_HANDLE_VALUE)
{
printf("Could not open %s\n", szPath);
return -1;
}
// Queue up a few requests
for (i=0; i<MAX_REQUESTS; i++)
{
// Read some bytes every few K
QueueRequest(i, i*16384, READ_SIZE);
}
printf("QUEUED!!\n");
// Wait for all the operations to complete.
for (;;)
{
DWORD rc;
rc = WaitForSingleObjectEx(ghEvent, INFINITE, TRUE );
if (rc == WAIT_OBJECT_0)
break;
MTVERIFY(rc == WAIT_IO_COMPLETION);
}
CloseHandle(ghFile);
return EXIT_SUCCESS;
}
/*
* Call ReadFileEx to start an overlapped request.
* Make sure we handle errors that are recoverable.
*/
int QueueRequest(int nIndex, DWORD dwLocation, DWORD dwAmount)
{
int i;
BOOL rc;
DWORD err;
gOverlapped[nIndex].hEvent = (HANDLE)nIndex;
gOverlapped[nIndex].Offset = dwLocation;
for (i=0; i<MAX_TRY_COUNT; i++)
{
rc = ReadFileEx(
ghFile,
gBuffers[nIndex],
dwAmount,
&gOverlapped[nIndex],
FileIOCompletionRoutine
);
// Handle success
if (rc)
{
// asynchronous i/o is still in progress
printf("Read #%d queued for overlapped I/O.\n", nIndex);
return TRUE;
}
err = GetLastError();
// Handle recoverable error
if ( err == ERROR_INVALID_USER_BUFFER ||
err == ERROR_NOT_ENOUGH_QUOTA ||
err == ERROR_NOT_ENOUGH_MEMORY )
{
Sleep(50); // Wait around and try later
continue;
}
// Give up on fatal error.
break;
}
printf("ReadFileEx failed.\n");
return -1;
}
//
// Make sure we are running under an operating
// system that supports overlapped I/O to files.
//
void CheckOsVersion()
{
OSVERSIONINFO ver;
BOOL bResult;
ver.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
bResult = GetVersionEx((LPOSVERSIONINFO) &ver);
if ( (!bResult) ||
(ver.dwPlatformId != VER_PLATFORM_WIN32_NT) )
{
fprintf(stderr, "IoByAPC must be run under Windows NT.\n");
exit(EXIT_FAILURE);
}
}
Q:如何把C++成员函数当做一个 I/O completion routine?
一个C++成员函数不能拿来做一个I/O completion routine,除非它是static函数。如果它是static函数，它就没有this指针，也就不能直接调用那些需要this 指针的成员函数。
解决的方法是存储一个指针，指向用户自定义数据，然后经由此指针调用一个C++成员函数。由于static成员函数是类的一部分，还是可以调用private成员函数。与如何把C++成员函数当做一个线程函数类似。
另外一个I/O completion ports非常重要，对产生高效率的服务器很受欢迎，这里略，详细参考相关文献资料。