Reversing is complex. Yet software developers often turn to reverse engineering when faced with a challenging task: enhancing software security, making software compatible with third-party components, maintaining legacy code, and so on.
At Apriorit, we have more than a decade of reverse engineering experience. Over the years, our researchers have collected a rich selection of efficient tools for reverse engineering.
In this article, we describe the main tools our software reversers rely on in their work and show practical examples of how to use them. This article will be useful for readers who are familiar with assembly language and network interaction principles and who have experience programming for Windows using API functions.
There are so many different software applications where the source code is hidden from sight. But there are a number of situations when we do need to understand the logic of how a certain platform or application functions, its algorithms, and specifics. That’s when legal software reversing — a service provided by Apriorit for more than a decade — is needed.
The efficiency of the reversing process depends heavily on two factors:
- The expertise of the reverse engineer
- The set of tools and techniques used
To learn more about the skills you might need and the specifics of Windows software reversing, check out our previous article describing the reversing process step by step. In this post, we discuss some of the best Windows reverse engineering tools.
In the second half of the article, you’ll find a brief example of how to use each of these tools in practice.
It’s hard to name the best software reverse engineering program – there are quite a few of them, and each resolves a specific task in the multistep reversing process. Below, we overview the nine main tools used for reverse engineering by Apriorit researchers:
- IDA Pro, Hex Rays
- CFF Explorer
- API Monitor
- Relocation Section Editor
Let’s get started.
IDA Pro is one of the best and most popular reverse engineering software tools. It’s an interactive disassembler that has a built-in command language (IDC) and supports a number of executable formats for various processors and operating systems. IDA Pro also has a great number of plugins that can extend the disassembler’s functionality even further.
The main advantage of IDA Pro is that it allows you to interactively change any element of the displayed data:
- Give names to functions, variables, data structures, etc.
- Change data representation (as numbers, strings in various encodings, data structures)
- Build diagrams and code flow graphs to simplify the understanding of disassembled code
- Use type information about function arguments and structure definitions from C++ so that arguments and variables are automatically named
- Automatically recognize and name standard library functions in assembler code
- And much more
Aside from the disassembler itself, let’s also look closer at some IDA plugins.
This plugin can turn native processor code into a more readable, C-like version. The Hex-Rays Decompiler produces rather accurate C code comparable to that produced by a human reverse engineer. It correctly decompiles code produced by various C++ compilers, no matter the architecture. However, Hex-Rays Decompiler might have issues with processing complex assembler code, where the original code was specifically modified by adding the inline assembler or some manual optimization was made.
This plugin enables you to mark the execution path within the disassembler. As a result, you can understand which pieces of code are taking part in the execution and if they are involved in some algorithm or feature.
Basically, this plugin loads reports of code coverage tools into the IDA database and marks pieces of code depending on how many times they were executed. This makes it clear which part of the code is worth your attention while browsing the disassembly.
This plugin is intended to be used on binaries built by Visual Studio and searches for RTTI information stored in the data section of the executable file. RTTI information allows the plugin to find the class names and virtual methods of C++ classes and name them for the user. Also, ClassInformer can present you with a list of found classes.
BinDiff by zynamix
This tool uses the IDA engine to compare binaries as assembler code instead of a stream of bytes. BinDiff can pinpoint differences in the code of two versions of the same program (down to changes in a specific function) as a list of instructions which were added, removed, or replaced. Changes can also be represented as code flow graphs.
This plugin analyzes imported functions and functions that call them and then groups them by tags: cryptography-related, registry-related, network-related, etc. Such grouping makes it easier to find the part of the code responsible for specific operations.
This plugin emulates the execution of disassembled code without the need to run the application under analysis in a debugger. Using this plugin, you can emulate the result of executing any piece of code without the risk of modifying something in the system. All you need to do is specify the start values of CPU registers. Then you can do a step-by-step execution.
CFF Explorer is a suite of tools for portable executable (PE) editing which includes:
- PE and HEX editors
- Resource editor
- Import editor
- Signature scanner
- Address converter
- Dependency analyzer
- And more
API Monitor is an application for intercepting API function calls made by apps and services. This tool can also display input and output data.
By default, API Monitor contains definitions for over 13,000 API functions and more than 1,300 COM interface methods.
WinHex is a hex editor that provides a rich set of features and development tools for Windows.
WinHex can display checksums or code of software files, which is something a regular text editor is unable to do.
Hiew is a binary file editor focused on working with code. It has a built-in disassembler for x86, x86-64, and ARM as well as an assembler for x86 and x86-64.
The main features of Hiew include:
- Viewing and editing logical and physical drives
- Searching for assembler commands by template
- Keyboard macros
- Built-in 64-bit calculator
- Tools to create custom plugins
Fiddler is a proxy that you can use to intercept HTTP/HTTPS traffic going between your application and the server, monitor it, and analyze it. Fiddler can intercept HTTP/HTTPS traffic system-wide. You can also add plugins (e.g. wbxml view, which can decode wbxml) and display requests/responses in different views.
Fiddler has a built-in hex editor and can generate requests based on a selected request or create a custom request. In addition, the Request to Code plugin allows you to get ready code that executes requests in C#, Visual Basic, or Python.
Scylla is an application for dumping a running application process and restoring the PE import table. With its help, you can get a totally restored PE file that can be run by the operating system.
Relocation Section Editor is an application used for editing the relocation table in PE files. The main purpose of this tool is to modify the relocation table in case of patching relocatable pieces of code. But it’s often used to remove the relocation table altogether when restoring a protected file.
A protected file actually contains the relocation table for the unpacker code only. The relocation table for the real code is usually hidden within the unpacker data. Thus, in case a dump is being recovered, there are two ways to restore the missing relocation table for the real code:
- Force the application to be loaded at two different base addresses, then compare the dumps to see which parts of the code are being patched and make a new relocation table.
- Remove the relocation table completely and specify in the PE file header that the file is not relocatable.
PEiD is one of the best reverse engineering tools to detect the packer. By analyzing entropy, PEiD can detect whether an application is packed.
There are also various useful plugins that help to analyze PE files. For instance, the KANAL (Krypto Analyzer for PEiD) plugin analyzes a PE file for the presence of known encryption algorithms.
These are the nine tools that reverse engineers at Apriorit often turn to when working on Windows reversing projects. As you can see, each of these pieces of software for reverse engineering solves a very unique, specific set of tasks. In the next section, we provide practical examples that display the role and importance of each of these tools in Windows reversing.
Let’s see how we can use each of these tools to research a Windows application. As an example, we are going to use a test application that you can download and analyze on your own.
1. Opening the researched executable with IDA Pro
Let’s load our test application to IDA Pro. We receive the following message:
This means that something has gone wrong with the application: its import table can’t be found. At this point, we only need to press the OK button. Once we do that, IDA Pro provides us with the following results of application analysis:
As you can see, the import table is almost empty. Its upper part shows that it was possible to detect a small piece of code (the blue part), and the left part shows which functions were detected (in our case, very few).
There is also a set of undetected bytes above the start function. We suppose that the application is packed by means of some packer. PEiD will help us determine which packer was used.
2. Getting information about the packer with PEiD
Now, we need to load our application in PEiD.
In Screenshot 13 above, you can see that the Entry Point is located in the UPX1 section. However, this fact alone doesn’t really tell us much, so we need to run a scan.
To start the scanning process, go to Options, choose Hardcore Scan, and click Save:
Next, select the folder where the application is located. After scanning is complete, we receive the following result:
As you can see from Screenshot 15, the application is packed using the UPX tool. To unpack it, we are going to use CFF Explorer.
3. Unpacking the application with CFF Explorer
To unpack our sample application with CFF Explorer, we need to go to the UPX Utility page in the main menu of CFF Explorer and press the Unpack button:
After that, we can upload the already unpacked application to IDA Pro and restore the assembler code.
We upload our application to IDA Pro once more, and when the system asks us whether to upload symbols from the server, we agree. Here is the result of application analysis in IDA Pro:
You can see in Screenshot 17 that we now have some readable code, more detected functions, and an import table (Screenshot 18). At this point, we can run the application and debug it in IDA Pro. In the disassembler, select Debugger > Select Debugger > Local Win32 debugger and then press F9. After that, we receive the following warning message:
Our tested application detected that it was debugged. To continue with our analysis, we need to disable debugger detection first.
See the import table:
At once, we can notice the NtQueryInformationProcess function. After clicking on it, we get the following list of xref functions:
Clicking on the function, we can see where it’s called. The third parameter is an output parameter. If it equals 1, then a debugger is attached to the application; if it equals 0, there’s no debugger attached to the app.
Let’s see where the result of this function is written:
As you can see in Screenshot 22, the third parameter contains the address of the local variable (var_8). After a function call, the result of the function is checked (test eax, eax). Then the value of var_8 is checked, and if it’s not 0, the value is written to the byte_131443C variable.
Let’s check if the byte_131443C variable is used somewhere else in this function:
We’ll start from the end. This value contains the result from al (lower bytes). Before that, the esi result is written to eax, and 1 is written to esi. Above, we see the condition for writing 1 to esi: if ecx + 2 does not equal 0. The value in ecx is large fs:30h (and then + 2). It verifies that a debugger’s presence has bypassed the IsDebuggerPresent function (a field of undocumented PEB structure).
Let’s rename this variable. To do so, press N or right-click on the function and select Rename.
Now we need to check if the g_isDebbugerPresent variable is used anywhere else:
There is “…” in the end, so this variable is used in more than one place. Place the cursor over it and click X, or right-click and select Jump to xref to operand:
We already know the first four places where this variable is used, but not the last one. Let’s find it:
In Screenshot 26, we see that the variable checks if esi equals 0, and if it doesn’t, we receive the message that there is a debugger.
Gladly, this verification can be removed. We’ll consider one way to do that a bit later, when we get to working with the Hiew tool.
It’s noteworthy that IDA Pro also allows for patching memory and code. In order to quickly find the needed piece of code, we’ll run the Rebase program in IDA Pro to get the same offset as in Hiew. In IDA Pro, select Edit > Segments > Rebase program and enter the value 0x400000 in the opened window:
Let’s get the address of the code that performs the comparison. As you can see from Screenshot 28 below, the address is 0х401329:
We need to write down this address, as we are going to use it later in Hiew.
Now we need to load our application in Hiew. First, it looks as follows:
We can switch to the Decode mode by pressing F4 > Decode. Now we have to find the address we received in IDA Pro — 0х401329. Press F5 and set the address this way:
At this point, we get code that compares the values of our g_isDebbugerPresent variable:
Now we can replace this code with, say, jmp to a specific address so that this condition will never be satisfied (in real applications, it can be an exception to immediately close the application).
Press F3 and then F2 to switch to the Edit mode. Enter the address of the next command after if. Add “.” in the beginning of the address so it will be relative:
After editing, our modified command is highlighted in yellow. Press F9 to update and save the application.
If we try to run our test application now, it crashes.
When looking at the assembler code, we see that the new jmp will result in calling esi further down the code, and esi will contain garbage instead of the MessageBox function address. Since we’re jumping over the address 401330, we’re skipping mov esi, ds: MessageBox. Thus esi will not be initialized, and the application will crash at 401366.
Therefore, let’s swap jmp and the mov instruction, which puts the MessageBox address to esi. Now, we should set the relative address in jmp to 14, but no longer to 1E, because the command has become closer to the command we are going to.
Now, to make jmp, we need to save the MessageBox address to esi.
Let’s run IDA Pro again and go to the g_isDebbugerPresent variable’s address:
In Screenshot 35, you can see that there is an unconditional jump. If we run the application via the debugger, the application will crash because the previous command contains an absolute address, and after the application starts, the loader passes the relocation table and adds delta to each value to make all addresses valid. What we need to do now is remove this value from the relocation table.
Let’s load the original file to IDA Pro, find cmp (which was deleted earlier), and enable the option of showing commands in byte representation in IDA. To do this, select Option > General:
This is what we get:
cmp is a double-byte command starting with 1329, and the address starts with 132B. We can see that jmp is a one-byte command and that its address is relative (meaning that it shouldn’t be in the relocation table):
Thus, we have to remove the 132A value from the relocation table. To do so, we need to open the current version of our test application with Relocation Section Editor.
5. Deleting a value from the relocation table with Relocation Section Editor
We start by loading the test application and finding the target value: 0x0040132A.
Next, we remove the target value and save the test application.
If we run the application now, it still crashes, so we need to go back to IDA Pro.
After removing 132A from the relocation table and swapping jmp/mov, the application will also crash because we haven’t edited the value that was previously pointing to mov and now should point to jmp. CFF Explorer can help us fix this issue.
6. Modifying values in the relocation table with CFF Explorer
Now it’s time to open our test application in CFF Explorer. We have found the value — 1332 — on which delta for MessageBox used to be added.
Let’s replace the value 1332 with the value 132B, the new offset by which MessageBox can be found. If we run the test application now, it won’t crash or show a warning message about a detected debugger.
Next, we move to working with the MessageBox function calls.
7. API Monitor
This program is used to monitor API calls and contains a number of known API functions it can monitor. You can also add your own functions to API Monitor and use this tool to monitor network function calls and research passed parameters (of course, if traffic is not encrypted).
Let’s monitor our application. We’ll try to find the call for the MessageBox function:
We’re interested only in the API functions that can show a message window. So we select only them in User32.dll.
In API Monitor, select File > Monitor New Process and set the path to our file. After running our process, we see the list of called functions. Let’s try to find MessageBox:
API Monitor shows which parameters were passed to the MessageBox function. Also, we can set different breakpoints for a function:
If we run monitoring of our file, we’ll get the following results:
Screenshot 44 also shows the parameters passed to the MessageBox function.
Reverse Engineer Software
Now, we move to working with binaries so we can find the code that we previously detected with IDA Pro and API Monitor. But before exploring a binary, we need to determine its type with a hex editor. In our example, we use WinHex.
Open the file with WinHex:
The MZ signature at the zero offset corresponds to PE format files (executables or shared libraries), so this is an exe file or dll.
Most file formats have unique signatures. For example, here’s what a dump file looks like in WinHex:
To show you how to work with Scylla, we are going to once again use a packed application, but we won’t be unpacking it this time. Instead, we'll dump its memory and try to run it.
To do that, we open the packed executable file in IDA Pro. This time, we need to find the original entry point (OEP) into the application rather than the entry point of the packer.
The pusha command saves general-purpose registers to the stack. In the end, there should be the popa command which pushes the stored register values. After the popa command, there is a jmp to the original point of entry. You can use the Search for text option by pressing Alt + T and look for the popa command.
Below the popa command, there is jmp 40A191, which will eventually move to the original entry point.
Let’s put a breakpoint in jmp and run the debugger.
Now, we can try to follow jmp. However, IDA Pro displays another warning message:
This message means that there is no code at the point we’re going to. Therefore, IDA will create instructions in disassembled listings on the basis of bytes pointed to by the Extended Instruction Pointer (EIP).
0x00971A91 is the address of the original entry point after unpacking the application into memory.
Now, without closing IDA Pro, open Scylla in order to create the application’s dump and restore the application’s import table from it.
In the process list, choose our application and put the OEP address into the field. Then choose first IAT Autosearch and then Get imports. As a result, Scylla will show that the import table has been found.
Let’s make an application dump: press Dump, save the dump, then click Fix Dump and select the previously saved application.
If we run our application now, it will crash, so we need to remove the relocation table. To do so, open the modified dump file (it has the _SCY prefix) in CFF Explorer.
Set 0 in the Relocation Directory RVA field.
Next, we need to check whether the ImageBase is the same that the application loaded into memory.
You can find the ImageBase value in IDA Pro by going to Edit > Segments > Rebase program.
Let’s save and run the test application. Now, it will run as if it were packed.
When applied properly, reverse engineering can help you strengthen the security and improve the performance of your solution. However, quality reverse engineering is impossible without using the right set of tools and techniques.
In this article, we covered some of the best reverse engineering software that our team of professional reversers uses when researching software. Most of these tools can only be used for solving specific tasks, but when combined, they provide you with all the capabilities necessary for extensive software analysis.
Want to see if reverse engineering is the right approach for your next project? Get in touch with us using the form below.