Windows异步I/O读文件的缓存限制问题?
在Windows下,使用OVERLAPPED方式,读取一个大文件(TEST.DAT, 200MB),示例代码如下(缩减):
HANDLE file_handle = 0;
DWORD numb_of_bytes = 0;
CHAR * file_buffer = new CHAR[1024 * 1024 * 128];
DWORD bytes_to_read = 1024 * 1024 * 64;
OVERLAPPED overlapped;
file_handle = CreateFile(
_T("D:\\TEST.DAT"),
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_ALWAYS,
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED
NULL);
overlapped.Internal = 0;
overlapped.InternalHigh = 0;
overlapped.Offset = 0;
overlapped.OffsetHigh = 0;
overlapped.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
BOOL result = ReadFile(file_handle, (void*)file_buffer, bytes_to_read, NULL, &overlapped);
result = GetOverlappedResult(file_handle, &overlapped, &numb_of_bytes, 60 * 1000);
.....
运行后发现:
(1)如果bytes_to_read < 64M字节,文件正常读取,numb_of_bytes输出值为64M字节。
(2)如果bytes_to_read >= 64M字节,那么GetOverlappedResult函数将等待到超时,并且numb_of_bytes输出为0。且在Windows任务管理器的进程IO读取字节中,同样显示为0。
(3)如果采用同步方式,则不存在上述现象,无论bytes_to_read多大,文件都将正常读取,且numb_of_bytes输出值正常。
(4)Win32SDK文档中,关于File I/O有一章File Caching,其中建议对于大数据量读写,最好使用FILE_FLAG_NO_BUFFING参数选项,但经测试,使用该选项时,结果与上述情况相同。
推测:
对于异步I/O,Windows内核设置了最大读取数据量的限制,以避免一个I/O长时间占用I/O通道,这个限制就在64MB。
不知上述推测是否属实,或者是哪里设置问题,查阅多方文档,均未解。