Cache Manager Overview

The Cache Manager is a software-only component that is tightly integrated with the Windows NT Memory Manager to integrate the caching of file system data with the Virtual Memory System. Some operating systems implement their file systems so they have a distinct data cache. However, because such caches must be managed from physical memory they are limited in size ¡V and memory used for such a cache is not available for use elsewhere in the system.

缓存管理器属于NT内存管理器紧密集成的软件组件，与虚拟内存系统集成文件系统缓存数据。

Thus one key advantage of using the Windows NT Cache Manager is that it allows for a balancing of the use of physical memory between file caching and programs running on the system. When an application is I/O intensive, the balance can be "tipped" towards caching data. When an application is consuming memory the amount of memory used for caching data can be reduced to practically zero. Thus, the net result is that the system makes better use of physical memory and ultimately provides better performance.

好处是物理内存的使用和文件缓存和系统运行程序在缓存管理器下达到均衡应用。

The other key reason for the file system to use the Cache Manager is that a file can be accessed either via the standard file system interface, such as read and write, or it can be accessed via the Memory Manager ¡V a "memory mapped" file. When both access methods are being used on the same file the Cache Manager provides a mechanism for bridging between the two to ensure consistency of the data.

另一个使用缓存关键原因是文件可被标准文件系统接口存取，如read和write等。或通过内存管理器形成一个“内存映射”文件。

Cache Manager Data Structures

The interface between the file system and the Cache Manager relies upon a procedural interface. Essentially all of the data structures within the Cache Manager are associated with a file, but the actual internal structure of those data structures is transparent to the file system. In this section we describe those key data structures which are shared between the file system and the Cache Manager.

缓存管理器数据结构：下段描述文件系统和缓存管理器共享的数据结构。

Buffer Control Bloxk (BCB)

A buffer control block is used internally by the Cache Manager to track when a portion of a file is mapped into the system address space. This is exposed to the file system because sometimes it is necessary for the file system to pin the data in memory while it is performing some critical operation.

Most of the buffer control block (BCB) is opaque. The first portion of the BCB is exposed to file systems:

typedef struct _PUBLIC_BCB {
    CSHORT NodeTypeCode;
    CSHORT NodeByteSize;
    ULONG MappedLength;
    LARGE_INTEGER MappedFileOffset;
} PUBLIC_BCB, *PPUBLIC_BCB;

The first two fields of the buffer control block are standard for Windows NT data structures ¡V they uniquely identify both the type and size of the data structure itself. The last two fields are of interest to the file system as they identify the range of the file managed by this particular buffer control block.

Buffer控制块(BCB)

BCB用来跟踪文件，当此文件影射到系统地址空间。在文件系统在内存锁定(Pin)数据以作关键操作。结构后两个字段是指定了此BCB文件管理的范围。

File Size Information

The file system and Memory Manager each maintain information about the size of the file. Whenever the file system establishes mapping for a file it indicates the current size of the file. Any subsequent changes to the size of the file are similarly indicated to the Cache Manager.

文件系统和内存管理维护文件大小信息。所有后续文件大小改动都同样被cache manager显示处理。

There are three values used by the Cache Manager to indicate the current size of the file:

typedef struct _CC_FILE_SIZES {
    LARGE_INTEGER AllocationSize;
    LARGE_INTEGER FileSize;
    LARGE_INTEGER ValidDataLength;
} CC_FILE_SIZES, *PCC_FILE_SIZES;

The names of these fields can be confusing. For instance the AllocationSize field is use, not to identify the actual physical space allocated for the file, but rather the amount of data which can fit in the presently allocated space. For some file systems, this turns out to be the same value. However, for a file system which supports compression or expansion of the actual data, this value represents the amount of data which could fit.

AllocationSize段并非实际文件实际分配的物理空间。而是适于分配空间的数据量。

The AllocationSize of the file is used by the Memory Manager to represent the size of the "section object." Since a section object is then used to determine how a file is mapped into memory it is essential that the AllocationSizealways be at least as large as the file. The Cache Manager and Memory Manager do not detect the case when the file system sets the AllocationSize to be smaller than the file size ¡V instead the system crashes due to the inconsistency in the data structures.

AllocationSize被内存管理器代表"section object"的大小。Section object 用于确定一个映射到内存文件，AllocationSize经常是最小为文件大小。

The FileSize of the file represents the last valid byte of data in the file ¡V logically it is the "End of File" marker.

The ValidDataLength of the file represents the last valid byte of data in memory. Thus, a file can be extended in memory prior to the data being written to disk.

Note that the CC_FILE_SIZES structure has precisely the same layout of the size fields as the FSRTL_COMMON_FCB_HEADER structure has. Typically, a file system does not maintain a separate CC_FILE_SIZES data structure but instead passes the address of the overlapping fields.

Cache Manager Callbacks

Interactions between the file system and the Cache Manager are manipulated via a series of callback functions. These callback functions are registered on a per file basis with the Cache Manager and are then used by the Cache Manager in order to ensure that the data structures are "locked" prior to performing a file system operation.

文件系统和缓存管理器通过一系列callback函数操纵和交互。

Windows NT assumes there is a strict ordering in how resources are acquired between the file system, Cache Manager, and Memory Manager. If followed, this ordering will ensure that deadlocks do not occur. Of course, if it is not followed, deadlocks can (and will) occur. Specifically, file system resources are acquired first. Then Cache Manager resources are acquired. Finally, Memory Manager resources are acquired.

Windows NT 假设在文件系统，资源获取有严格顺序。以保证不发生死锁。特别是文件系统的资源是在第一位获取，然后是缓存管理器资源，最后是内存管理器资源。

Thus, these callbacks are used by the Cache Manager to honor this hierarchy. The callbacks required by the Cache Manager are:

typedef BOOLEAN (*PACQUIRE_FOR_LAZY_WRITE) (
        IN PVOID Context,
        IN BOOLEAN Wait
        );

typedef VOID (*PRELEASE_FROM_LAZY_WRITE) (
        IN PVOID Context
        );

typedef BOOLEAN (*PACQUIRE_FOR_READ_AHEAD) (
        IN PVOID Context,
        IN BOOLEAN Wait
        );

typedef VOID (*PRELEASE_FROM_READ_AHEAD) (
        IN PVOID Context
        );

typedef struct _CACHE_MANAGER_CALLBACKS {
        PACQUIRE_FOR_LAZY_WRITE AcquireForLazyWrite;
        PRELEASE_FROM_LAZY_WRITE ReleaseFromLazyWrite;
        PACQUIRE_FOR_READ_AHEAD AcquireForReadAhead;
        PRELEASE_FROM_READ_AHEAD ReleaseFromReadAhead;
} CACHE_MANAGER_CALLBACKS, *PCACHE_MANAGER_CALLBACKS;

Note that the callbacks are used for two distinct parts of the Cache Manager. The first, the lazy writer, is responsible for writing dirty cached data back to the file system. The second is for read ahead handling ¡V reading data prior to an actual call from the user to obtain that information.

注意到callbacks用于Cache Manager 的两个不同部分。首先是Lazy writer有责任写回脏缓存数据到文件系统。第二是为预读功能。

First, in designing these it is important to note what you are protecting your file system against (and what you aren¡¦t.) There is no reason to serialize cached I/O operations from applications with I/O from the Cache Manager¡¦s lazy writer. However, you do need to protect against non-cached user I/O operations and user operations that modify the size of the file.

The NT file systems do this by using two ERESOURCE structures. Both of these can also be used (and located) by other components within the operating system by walking through the common header ¡V specifically the Resource and PagingIoResource fields within the common header. The Cache Manager does not directly acquire these resources ¡V instead it calls into the file system to acquire any necessary resources (typically these resources

需要保护non-cached用户I/O操作和用户修改文件大小操作。NT文件系统使用两个ERESOURCE结构，从Common header的PERESOURCE Resource
PERESOURCE PagingIoResource两个字段指定。Cache Manager并不直接获取资源相反调用文件系统获取必须资源。

Note that these routines must be provided by your file system ¡V they are not optional and the system will crash if you fail to provide them.
The following code is a sample implementation from an older version of the OSR FSDK on the implementation of a callback management routine:

下面代码是OSR FSDK的简单执行callback 管理routine.

static BOOLEAN OwAcquireForLazyWrite(PVOID Context, BOOLEAN Wait)
{
POW_FCB fcb = (POW_FCB) Context;
BOOLEAN result;

// Take out the lock on the file.

result = OwAcquireResourceExclusiveExp(&fcb->Resource, Wait);
if (!result) {

// We did not acquire the resource.

return (result);
}

// We did acquire the resource. We need to:
// (1) Store away the thread id of this thread (for the release)
// (2) Set top level irp to a pseudo value
// In both cases, the previous value should be zero.

OwAssert(!fcb->ResourceThread);
fcb->ResourceThread = OwGetCurrentResourceThread();
return (TRUE);
}

Each of the file systems in the Microsoft IFS kit also contains examples of routines like these. For the Lazy Writer these routines are located in the following locations:

File System
File
Routine

FAT
resrcsup.c
FatAcquireFcbForLazyWrite

CDFS
resrcsup.c
CdAcquireForCache

RDR2
rxce\resrcsup.c
RxAcquireFcbForLazyWrite

Other similar routines (for read ahead for example) can be located in the same file.

另外相似例程预读等可在同一个文件找到。

CcCanlWrite

Because an application program can modify data in memory at a rate that exceeds the ability to write the data to disk, the Virtual Memory system can "fill up" with data. This in turn can then cause fatal out-of-memory conditions to occur within the VM system. To avoid this, the file system must cooperate with the VM system to detect these conditions. One of the key operations provided by the Cache Manager for this support is CcCanIWrite. The prototype for this call is:

应用程序可修改内存数据的速率超出写数据到磁盘，虚拟内存系统可被数据填满。

判断是否可以写入数据到缓存。

NTKERNELAPI BOOLEAN CcCanIWrite (
    IN PFILE_OBJECT FileObject,
    IN ULONG BytesToWrite,
    IN BOOLEAN Wait,
    IN BOOLEAN Retrying
    );
未完

[注意]传递专业知识、拓宽行业人脉——看雪讲师团队等你加入！