blank Skip to main content

Applied OpenSSL: CTR mode in file encryption


Information security on PC becomes more and more popular, so I want to cover some issues of this topic – in particular, the using of ciphers to prevent data stealing from a physically removed hard disk.

Cipher is the sequence of steps to transform original text (plain text) to cipher text, which completely can’t be read by unauthorized person or the efforts to crack it will cost more than the information in the plain text costs.

Ciphers are documented and well-known, but the problem is how to make the good implementation, or in our situation, how to find and use the open source library. I will explain what advantages OpenSSL library has and how to use it.

File encryption methods

You can secure single files or folders by the user-mode applications, which will encrypt or decrypt information by your command. You can also use the complex kernel-mode on-the-fly encryption solutions based on the virtual drives (legacy disk device), encrypted file systems, file system filters and storage filter driver (full disk encryption), which work synchronously with OS read/write requests.

Application levels to encrypt information

Fig. 1. Application levels to encrypt information.

Cipher types by type of the key:

Symmetric key ciphers are preferable because encryption and decryption are performed by a single solution. At the same time, you should remember that the process can be successful only if nobody can get access to the processed data and the sources of the solution (this should be guaranteed by the solution itself).

Cipher types by type of input data:

  • Block ciphers – encrypt and decrypt blocks of data of fixed size;
  • Stream ciphers – encrypt and decrypt continuous streams of data (message).

lock ciphers are preferable because of the way the data is stored on HDD: it is divided into the sectors of fixed size.

In this article, I want to describe the problem of cipher implementation in file encryption.

For each kind of security application we should choose cipher and, in case of the file encryption, corresponding block cipher mode. Each case requires different cipher mode:

  • Legacy Disk Device and File System level could use any block cipher, because on this level, read and write requests are aligned to the sector size and you control and have access to all data. These disk encryption solutions use XTS and CMC cipher mode.
  • File System filters have less freedom to work with files, because of permissions, which are set on file creation by user or system. Also, on this level you closely connected to pure documented behavior of the cache manager and file system. Any wrong action with data, data length or permissions could cause errors or unexpected behavior. The best cipher mode for this purpose is CTR, which offers real random access to the file data. It allows you to encrypt or decrypt only the current chunk of non-granulated data, and only it.
  • User-mode applications can use any block cipher, but while using ECB or CBC, the applications should store the header in files, which contains the real file size, and should granulate the file size to the block size of cipher. Another approach is to use CTR, CFB or OFB. They differ only by error propagation in changed cipher text, so I would prefer CTR.


Crypto++ library is well-known, use of Crypto++ is described very well in the article “Applied Crypto++: Block Ciphers”. This library is very useful and has a big amount of ciphers and wrappers, but it also has a couple of disadvantages, which can prevent its usage:

  • Porting through compilers (different msvc and gcc)
  • Problems with using in driver projects

I would like to introduce the OpenSSL implementation, which has solid C style easily portable for different compilers and platforms.

Main folders for our purpose are crypto and include. The crypto folder contains all ciphers and the include folder contains library headers with definitions and constants:

Listing of OpenSSL folder

Fig. 2. Listing of OpenSSL folder.

Each cipher has its own subfolder in the crypto folder. Implementation of each mode for the cipher is located in the same folder, but some ciphers use common cipher mode implementation, which can be found in the modes folder. You can use all of them or easily copy a folder with the required algorithm and use it separately. I pick the AES cipher:

Listing of crypto folder

Fig. 3. Listing of crypto folder.

You should copy files to the new library project, change some include paths and build the library with AES cipher. See the example attached to the article.

To use AES cipher you should initialize AES key from the initialization vector with the help of AES_set_encrypt_key:

InitializeAesKey ( const unsigned char* aKey, 
	unsigned int KeyLength, CSecurityKey* key )
    AES_KEY bfKey;
    ::memset( &bfKey, 0, sizeof (bfKey) );
    AES_set_encrypt_key( aKey, KeyLength, &bfKey );
    *key = CSecurityKey ( (char*)&bfKey, sizeof(AES_KEY));

To encrypt or decrypt data with CTR you also need three variables:

void AES_ctr128_encrypt(...
			unsigned char ivec[AES_BLOCK_SIZE],
			unsigned char ecount_buf[AES_BLOCK_SIZE],
			unsigned int *num )
  • ivec – counter of the CTR mode.
  • ecount_buf – variable used for encrypted ivec.
  • num – index of byte from start of the data block to encrypt.


  • If you want to use CTR from the beginning of the file, you should use nullified ivec; if other, you should initialize your ivec with the value that corresponds to the size of the working block of data. ( for 128bit cipher, to crypt from the byte 48, you should increment ivec 3 times – it is only for example because this method is not very fast for the big files )
  • If you crypt data from the beginning of the block of your block cipher, you should use nullified ecount_buf and num; if other, you should manually assign ecount_buf to the encrypted ivec for this data block, assign num to the index of byte from the beginning of the data block, and assign ivec to the value of the next data block.

For example, let’s take the 128bit cipher. To start encryption from the 50th byte, we should:

  • Perform ivec incrementation 4 times.
  • Set ecount_buf to the encrypted value of ivec
  • Set num to 2 (as the 50th byte is the 2nd one from the beginning of the 4th block – 50=16*3+2)
  • Perform ivec incrementation.

You can find the example of using OpenSSL implementation in the attached files. There is solution for 2008 Visual Studio with 2 projects:

  • OpenSSLAes – library with the AES cipher plus CTR mode.
  • Crypt_Mode_CTR – executable test file. You can encrypt or decrypt any file you want and see the result.

Input file:


Encrypted file:


Tell us about your project

Send us a request for proposal! We’ll get back to you with details and estimations.

By clicking Send you give consent to processing your data

Book an Exploratory Call

Do not have any specific task for us in mind but our skills seem interesting?

Get a quick Apriorit intro to better understand our team capabilities.

Book time slot

Contact us