Subscribe to receive all latest blog updates

This article would be useful for Windows developers, as it explains how to create virtual disk for Windows system.

Written by:

Gennadiy Maryanichenko
Development Leader of Driver Team

Windows virtual disks are used to be developed using kernel mode drivers. “Windows and disks” section will provide you necessary information about Windows disk drivers.

The widely known open source project named FileDisk (http://www.acc.umu.se/~bosse/) will also provide you ways for Windows virtual disk implementation.

As Apriorit frequently works on the advanced driver development projects, virtual disk technologies are an important part of our profile. The main differentiator of the Virtual Disk Development Kit, provided in this article, is the fact that it serves requests to the virtual disk in user mode in contrast to FileDisk, which processes them in kernel mode.

Pros of processing requests in user mode

Processing requests in user mode is rather useful if we have already developed user mode code that gives access to the data source (like disk image in memory, remote disk, cash register), and it’s difficult to port this code to the kernel mode; or we have no source code, for example during network access or if we use specific encryption library.

I have used SparseFile (file-container with data; in case of data accumulation, it will size up to the maximal size) as an example of such user mode library.

Article structure

In the «Windows and disks» section, we’ll consider the way Windows interacts with the disks; Windows virtual disk creation variants; useful tips for disk driver development.

In the «How to implement Windows virtual disk» section, we’ll consider the solution architecture, main implementation points and key stages of our disk’s lifecycle.

The «How to build Windows virtual disk Solution» section contains tips for building and testing the virtual disk project.

Used technologies

Project structure

There are several projects in the solution:

  • CoreMntTest (user mode, executable) – makes and mounts disk image using the CoreMnt_user code.
  • CoreMnt_user (user mode, library) – gets requests to virtual disks from the CoreMnt driver and processes them.
  • UsrUtils (user mode, library) – uses DeviceIoControl to contain auxiliary code of interaction with drivers.
  • CoreMnt (kernel mode, executable) – implements OS requirements for the disks; transforms requests; sends them to CoreMnt_user.
  • drvUtils (kernel mode, headers only library) – auxiliary kernel mode code, for example synchronization tools.

The following scheme shows the projects’ relations.

Structure of the project directory

  .\bin - folder with binary files

  .\lib - folder with library files

  .\obj - folder with object files

  .\src - folder with source files

     |

     |-> .\CoreMnt       - Kernel mode driver.

     |-> .\CoreMnt_user  - User mode mount library.

     |-> .\CoreMntTest   - User mode mount test.

     |-> .\drvCppLib     - Kernel Library to develop driver in C++.

     |-> .\drvUtils      - Kernel Library with utils for kernel mode projects.

     |-> .\mnt_cmn       - Files that are shared between projects.

     |-> .\STLPort       - Directory with STLPort 4.6 ported for

            utilizing in windows drivers.

     |-> .\usrUtils      - Win32 Library with utils for user mode projects.

 

Windows and disks

You can simply skip this section, if you have any experience of Windows driver development. I’m happy to appease the rest – this process is very simple. Windows sends “write” or “read” request to the disk. The disk returns read data or error code. That's all.

Of course, there are a few nuances.

We'll discuss a simple scheme of processing of requests to the disk. So, what happens after application calls, for example, the ReadFile function?

First, file system driver (for example ntfs.sys) receives the read file request. Scheme below illustrates this process:

File system driver defines the exact location of the requested file (the offset) and creates the read disk request. File can be divided into several parts, located at different places on the disk. Thus, a few requests will be created. Our virtual disk driver will receive requests like that from the file system driver. Virtual disk can be also implemented at the file system level. You can learn more from the http://www.codeproject.com/KB/system/fs-filter-driver-tutorial.aspx article.

Let’s consider technical terms we use:

  • IRP (I/O Request Packet) is a Windows kernel structure that stores parameters of the requests. In order to read data from device, we should indicate request type, buffer to read data to, size and offset. We can assume that IRP is a request to some device. Talking about IRP in this article, we always mean request. You can learn more about it at http://www.microsoft.com/whdc/driver/kernel/IRPs.mspx.
  • STATUS_PENDING is a special return code that alerts the request initiator about that IRP cannot be served now and will be served later on. There is a termination event for this case, device will set it when complete the request serving. Below we will describe the usage of this return code.
  • Device is a Windows kernel object that represents any device. It contains info about this device, for example its name. It also contains DeviceExtension.
  • DeviceExtension is a Device structure field that can be used by the device creator in his own way. We will consider the usage of DeviceExtension below.

How to implement Windows virtual disk

The solution consists of driver (CoreMnt.sys) and application (CoreMntTest.exe). The general scheme is shown below:

Driver helps to mount disks. Application creates data source and mounts it as a disk via the driver service. After receiving IRP, driver processes them in user mode and returns the result. Below you can find the general scheme of driver work:

CoreMntTest.exe processes requests from OS to the virtual disks. Below there is its structural scheme:

Now let’s stage by stage consider the source code.

1. Initialization

Now we are going to start the CoreMnt driver using the following command:

c:\>sc start CoreMnt

In DriverEntry, we should create the management device as an access point for CoreMntTest from user mode:

NTSTATUS DriverEntry(IN PDRIVER_OBJECT DriverObject,
                     IN PUNICODE_STRING RegistryPath)
{
...
    NTSTATUS status;
    status = IoCreateDevice(DriverObject,     // pointer on DriverObject
                            0,                // additional size of memory
                            &gDeviceName,     // pointer to UNICODE_STRING
                            FILE_DEVICE_NULL, // Device type
                            0,                // Device characteristic
                            FALSE,            // "Exclusive" device
                            &gDeviceObject);  // pointer do device object
    if (status != STATUS_SUCCESS)
        return STATUS_FAILED_DRIVER_ENTRY;
    status = IoCreateSymbolicLink(&gSymbolicLinkName,&gDeviceName);
    if (status != STATUS_SUCCESS)
        return STATUS_FAILED_DRIVER_ENTRY;

Then we should register the driver request handler. We are going to use one handler for all types of requests:

    for (size_t i = 0; i <= IRP_MJ_MAXIMUM_FUNCTION; ++i) 
        DriverObject->MajorFunction[i] = IrpHandler;

Finally, we are going to create MountManager:

    gMountManager = new MountManager(DriverObject);
    return STATUS_SUCCESS;
}

2. Virtual disk mounting

This is the stage, where we run the CoreMntTest.exe application. It sends the CORE_MNT_MOUNT_IOCTL management message to the driver:

    CORE_MNT_MOUNT_REQUEST request;
    request.totalLength = totalSize;
    request.mountPojnt = mountPoint;
    DWORD bytesWritten = 0;
    CORE_MNT_MOUNT_RESPONSE response;
    if(!m_coreControl.DeviceIoGet(CORE_MNT_MOUNT_IOCTL, 
                                  &request, 
                                  sizeof(request),
                                  &response,
                                  sizeof(response),
                                  &bytesWritten))
    {
        throw std::exception(__FUNCTION__" DeviceIoGet failed.&);
    }

The DispatchMount function deserializes request parameters and calls MountManager::Mount:

        if(inputBufferLength < sizeof(CORE_MNT_MOUNT_REQUEST) || 
            outputBufferLength < sizeof(CORE_MNT_MOUNT_RESPONSE) )
        {
            throw std::exception(__FUNCTION__" buffer size mismatch");
        }
        DISK_PROPERTIES diskProperties;
        CORE_MNT_MOUNT_REQUEST * request = (CORE_MNT_MOUNT_REQUEST *)buffer;
        diskProperties.totalLength.QuadPart = request->totalLength;
        CORE_MNT_MOUNT_RESPONSE * response = 
            (CORE_MNT_MOUNT_RESPONSE *)buffer;
  response->deviceId = gMountManager->Mount(&diskProperties);

In MountManager::Mount, we make and save the MountedDisk object. MountedDisk includes LogicIrpDispatcher, which constructor creates disk device. OS will send requests to this device:

LogicIrpDispatcher::LogicIrpDispatcher(PDISK_PROPERTIES diskProperties, 
                                       PDRIVER_OBJECT   DriverObject,
                                       MountManager* mountManager)
{
...
    //create device
    status = IoCreateDevice(DriverObject,sizeof(InThreadDeviceExtension),
        &deviceName,FILE_DEVICE_DISK,
        0,
        FALSE,&deviceObject_);
    if (!NT_SUCCESS(status))
        throw std::exception(__FUNCTION__" can't create device.");

After creating device, we should initialize DeviceExtension. We will use it for storing device identifier. Therefore, after getting IRP, we will simply find the corresponding MountedDisk:

    InThreadDeviceExtension* devExt = 
        (InThreadDeviceExtension*)deviceObject_->DeviceExtension;
    memset(devExt, 0, sizeof(InThreadDeviceExtension));
    devExt->mountManager = mountManager;
    devExt->deviceId = diskProperties->deviceId;

Therefore, MountManager has created the MountedDisk instance and saved it to the container. Then we are finishing the initialization stage in the user mode. For each disk, we create a thread, where all disk requests will be served. A thread sends IOCTL RequestExchange to the driver and proceeds to the request awaiting mode:

    while(true)
    {
        int type = 0;
        int size = 0;
        __int64 offset = 0;
        drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
                                 &type, &size, &offset);
        //do requested operation
        DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
        lastType = type;
        lastSize = size;
    }

Performance note: surely, processing requests in one thread is the «bottle neck». Thread pool should certainly be there in the real project.

3. Requests processing

Now our virtual disk is able to serve requests. I'll describe the complete chain of request processing. Everything starts from the IrpHandler function registered by our driver as the IRP processing procedure. Now we receive the device identifier from DeviceExtension (we saved it there during the initialization stage) and transmit IRP to MountManager:

NTSTATUS IrpHandler( IN PDEVICE_OBJECT fdo, IN PIRP pIrp )
{
...
        InThreadDeviceExtension* devExt = 
            (InThreadDeviceExtension*)fdo->DeviceExtension;
        return gMountManager->DispatchIrp(devExt->deviceId, pIrp);

MountManager receives IRP, finds the corresponding MountedDisk by device identifier, and redirects IRP to it. The code below makes decision whether it is possible to process this request at once or it should be processed in the user mode:

NTSTATUS MountedDisk::DispatchIrp(PIRP irp)
{
    IrpParam irpParam(0,0,0,0);
    irpDispatcher_.getIrpParam(irp, &irpParam);
    if(irpParam.type == directOperationEmpty)
    {
...
            irpDispatcher_.dispatch(irp);
...
        NTSTATUS status = irp->IoStatus.Status;
        IoCompleteRequest(irp, IO_NO_INCREMENT);
        return status;
    }
    IoMarkIrpPending( irp );
    irpQueue_.push(irp);
    return STATUS_PENDING;
}

In case of IRP_MJ_READ or IRP_MJ_WRITE, it should be processed in user mode. All other requests can be processed by the driver itself. For example, in case of IOCTL_DISK_GET_LENGTH_INFO, driver knows the disk size as well as that this size cannot be changed. In LogicIrpDispatcher::dispatchIoctl, you can find the complete list of requests, which Windows can send to the disk.

The thread, which serves this disk, chooses requests from the list:

void MountedDisk::RequestExchange(UINT32 lastType, UINT32 lastStatus, UINT32 lastSize, char* buf, UINT32 bufSize,
                     UINT32 * type, UINT32 * length, UINT64 * offset)
{
...
    NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny, 
        Executive, KernelMode, FALSE, NULL, 0);
...
            IrpParam irpParam(0,0,0,0);
            irpDispatcher_.getIrpParam(lastIrp_, &irpParam);
            *type = irpParam.type;
            *length = irpParam.size;
            *offset = irpParam.offset;

If it is IRP_MJ_WRITE, we should copy data to write to the buffer. After that, the solution passes this buffer to the user mode code:

        if(*type != directOperationEmpty 
            && opType2DirType(directOperationTypes(*type)) == directOperationWrite)
        {
            IrpParam irpParam(0,0,0,0);
            irpDispatcher_.getIrpParam(lastIrp_, &irpParam);
            if(irpParam.buffer)
                memcpy(buf, irpParam.buffer, *length);

After exiting the RequestExchange function, we get to the DispatchImage (cycle of requests processing) again:

    while(true)
    {
        int type = 0;
        int size = 0;
        __int64 offset = 0;
        drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
                                 &type, &size, &offset);
        //do requested operation
        DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
        lastType = type;
        lastSize = size;
    }

The type, size, and offset variables now contain the new request for processing. The DispatchImageOperation function will manage it:

void DispatchImageOperation(IImage * image, 
                            int type, int size, __int64 in_offset, char* buf,
                            int bufsize,
                            int* status)
{
    switch(type)
    {
...
    case directOperationRead:
        {
           image->Read((char*)buf, in_offset, size);
           *status = 0;
            break;
        }
    case directOperationWrite:
        {    
            image->Write((const char*)buf, in_offset, size);
            *status = 0;
            break;
        }

After serving the request, the RequestExchange function will be called again and the thread will proceed to the mode of waiting for new request.

4. Virtual disk unmounting

It starts in user mode from the UnmountImage function calling. Below you can find the code that checks whether the disk is being used now or not:

void UnmountImage(int devId, wchar_t mountPoint, DriverControl * drvCtrl)
{
...
        if (!DeviceIoControl(hVolume,FSCTL_LOCK_VOLUME,NULL,
                             0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to lock logical drive");
        }
        else if (!DeviceIoControl(hVolume,FSCTL_DISMOUNT_VOLUME,
                                  NULL,0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to dismount logical drive");
        }
        else if (!DeviceIoControl(hVolume,FSCTL_UNLOCK_VOLUME,NULL,
                                  0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to unlock logical drive");
        }

Then we eliminate the connection between mounting point and our device:

 
    if (UndefineLogicDrive(mountPoint))
        throw std::exception("Unable to undefine logical drive");

After that, we send message to all components that store list of disks in the system (for example, explorer.exe or any other file manager):

 SHChangeNotify(SHCNE_DRIVEREMOVED, SHCNF_PATH, root, NULL);

Finally, we notify our driver that it is time to delete the device:

    drvCtrl->Unmount(devId);
  }

MountManager::Unmount simply removes the corresponding MountedDisk from the container. It causes the calling of its destructor:

MountedDisk::~MountedDisk()
{

We set the stop event for the thread of requests processing for this disk:

 stopEvent_.set();

Then we terminate all IRPs, which were not processed and now are in a queue:

 
    if(lastIrp_)
        CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);
    while(irpQueue_.pop(lastIrp_))
        CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);
}

The thread of requests serving, which has been awaiting in MountedDisk::RequestExchange, reacts on stopEvent_ set and throws an exception:

    NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny, 
        Executive, KernelMode, FALSE, NULL, 0);
    if(status != STATUS_SUCCESS)
    {
        throw std::exception("MountedDisk::RequestExchange - mount stop.");
    }

We are catching the thrown exception in the DispatchException function catch block, and returning STATUS_UNSUCCESSFUL to the user mode:

NTSTATUS DispatchExchange(PVOID buffer, ULONG inputBufferLength, ULONG outputBufferLength)
{
    try
    {
...
        gMountManager->RequestExchange(request->deviceId, 
                                       request->lastType, 
                                       request->lastStatus, 
                                       request->lastSize, 
                                       request->data, 
                                       request->dataSize, 
                                       &response.type,
                                       &response.size, 
                                       &response.offset);
...
    }
    catch(const std::exception & ex)
    {
        KdPrint((__FUNCTION__" %s\n", ex.what()));
        return STATUS_UNSUCCESSFUL;
    }
}

The returned error state is processed in the DriverControl::RequestExchange function by the user mode code and throws an exception:

void DriverControl::RequestExchange(int deviceId, 
                                    int lastType, 
                                    int lastStatus, 
                                    int lastSize, 
                                    char * data, 
                                    int dataSize,
                                    int *type, 
                                    int *size, 
                                    __int64 * offset)
{
...
    if(!m_coreControl.DeviceIoGet(CORE_MNT_EXCHANGE_IOCTL, 
                                  &request, 
                                  sizeof(request), 
                                  &response, 
                                  sizeof(response), 
                                  &bytesWritten))
    {
        throw std::exception(__FUNCTION__" DeviceIoGet failed.");
    }
...
}

This exception is caught in the SyncMountmanager::mountDispatchThread catch block:

void SyncMountManager::mountDispatchThread(void* pContext)
{
...
    try
    {
        DispatchImage(dispContext->devId, 
                      image->GetMountPoint(),
                      dispContext->image, 
                      dispContext->mountManager->GetDriverControl());
    }
    catch(const std::exception& ex)
    {
        dispContext->mountManager->OnUnmount(dispContext->image, ex.what());
    }
...
}

Therefore, it will lead to requests processing thread termination and the Image destructor calling.

Steps to build Windows virtual disk Solution

1. Windows virtual disk building

  1. Install Windows Driver Developer Kit 2003 (http://www.microsoft.com/whdc/devtools/ddk/default.mspx)
  2. Set global environment variable "BASEDIR" to the path to installed DDK.
  3. Computer -> Properties -> Advanced -> Environment variables ->System Variables -> New
    Like this: BASEDIR -> c:\winddk\3790

  4. Download and install boost (tested with 1.40 version) (http://www.boost.org/users/download/ )
  5. Set global environment variable "BOOST" to the path to installed boost. Restart your computer after this.
  6. Build the solution by means of Visual Studio 2008.

2. Steps to test the Solution

  1. Build the solution by means of described instructions.
  2. Copy CoreMnt.sys to %windir%\system32\drivers.
  3. Register the driver in the system:
  4. sc create CoreMnt type=  kernel binPath= system32\drivers\CoreMnt.sys

  5. Start driver using the command:
  6. sc start CoreMnt
  7. Start CoreMntTest.exe.

If everything was OK, CoreMntTest.exe displays the message:

  Image was mounted. Press  any key for unmount.

Disk Z appears in the system.

Now we are able to format it.

The «tst_img» file appears in the directory along with CoreMntTest.exe.

3. Supported Windows versions

  • Windows XP SP2

Conclusion

In this article, I have described just one of the approaches for Windows virtual disk creation. We can call it logical virtual disk. You can also consider the following approaches:

  • File system virtual disk. Driver receives the CreateFile, ReadFile, WriteFile requests and so on.
  • Physical virtual disk. The approach of physical virtual disk is semantically similar to the logical one but it has some differences on the driver level: there is another format of requests to the virtual disk as well as PNP disk device.

 

Download sample sources

 

Read next: SaaS Application Development Team