This is the second part in our series File System Virtualization – The New Perspective.
In this part, we’ll demonstrate how to implement a virtual disk plugin for one of the most popular cloud storage services, Google Drive. This plugin will provide you with transparent access to cloud files and folders.
Note: This plugin requires our previously implemented virtual disk service.
First of all, we’ll need a common cloud plugin interface that can be used by all future plugins that we decide to implement:
Since a cloud file can represent either an actual file or a directory, we can work with such files using the generic
Plugins take the form of a Windows dll file. This allows you to load and unload them on demand through a virtual disk service. Here’s a possible wrapper for such a plugin:
And here’s an example of
LoadProtocolLib method implementation:
The dll plugin needs to export only three functions, and it works as a wrapper for
When you create a plugin API, you’ll need to implement a Google Drive protocol for your disk service. Using the
ICloudWorker interface, you can create a single common protocol class without the need to create separate implementations for each cloud plugin you may add in the future.
Google Drive provides a RESTful API for interacting with files in the cloud. Both API versions 2 and 3 are currently supported. You can learn more in Google’s API Reference. We’ll cover only part of this material for the sake of simplicity.
You should authorize every request coming to the Drive API using OAuth2 supported by cpprestsdk. That’s beyond the scope of this article, though.
The Drive API supports the following file and folder requests:
Your Virtual Disk plugin must implement at least the bare minimum of requests required to correct file I/O handling. These requests are create, update, delete, get, and list.
Once you define the necessary API functions, you can start implementing them.
You can represent a Drive cloud item using either a file or folder class.
Every item also has associated metadata serialized into the JSON object.
Finally, you can write the code for actual Drive API requests. You should use the get operation for this. All other operations can be implemented similarly. This example uses curl to handle the HTTP protocol:
Since the Drive API doesn’t support partial file downloads, you should synchronize all files locally before making changes. That way all file system operations will be performed in the cache once the file download is complete.
Here’s a possible implementation for
After applying changes to a file, you should synchronize it back to the cloud. You should do this asynchronously to ensure that multiple changes in the same file are consolidated and to reduce the number of network I/O operations.
To reduce latency and improve the user experience, the cache also utilizes the readahead technique: the plugin downloads all cloud files in advance starting from the drive root. At the time the system accesses the necessary file, it has already been located in the local storage.
When your plugin is complete, you can test it.
But first, you should enable the Drive API in the Google Developers Console. You can do this by following these simple steps:
- Use this wizard to create or select a project in the Google Developers Console and automatically enable the API. Click continue and then go to credentials.
- On the add credentials to your project page, click cancel.
- At the top of the page, select the OAuth consent screen tab. Select the email address option, enter a product name if not already set, and click save.
- Select the credentials tab, click the create credentials button, and select OAuth client ID.
- Select the other application type, enter the name “Virtual Disk plugin,” and click create.
- Click OK to dismiss the dialog box that opens.
- Save both the client ID and secret for use in the mounting tool.
Now you can mount and access the disk:
Your cloud storage will now be visible as a local disk in Windows Explorer:
You can seamlessly access all disk files.
In this article, we’ve shown the steps required to implement a cloud storage plugin for a virtual disk using file system virtualization. This plugin is designed to work with Google Drive, but what we’ve created is a solid interface foundation allowing for quick integration with other cloud plugins as well: DropBox, Box, Adobe Creative Cloud, and more.