Often companies developing ParentalControl or DLP solutions require access to private data from software, installed on Android devices. For example, messages from Skype/KIK/Viber, etc., or even an access to a browsing history are both good examples of such tasks. By default (and this is what is used most often due to security) all Android apps save data in a so-called
Sandbox – a storage area which other applications cannot access. And if an app doesn’t provide an API to get the data (or simply places the data in memory card where other applications can access it), then this data seemingly cannot be accessed. In this article I want to show one simple and fairly elegant way to access private data of third party applications.
The main idea behind the solution is very simple – to parse a layout tree of an active page and extract the necessary data.
One caveat is that, while this solution works well starting with Android 4.3 and older (including the latest Android N), to get a clean data without junk you will need to take your time to study each target application. There is no need for ROOT-access, since with it you would be able to read directly from a sandbox.
I will explain this approach based on one particular task: writing an Android app that can be used to monitor:
- Firefox browser history on Android 6.0 (public API for this was closed even for built-in browser)
- Skype instant messages (there was no official way to get access to them)
Basically, both tasks are very similar, since you’re trying to achieve the same thing, e.g., get access to personal data.
Android has a so called AccessibilityService – a built-in service and a set of API to develop applications for people with disabilities (mostly, people with eyesight and hearing problems).
This service is allowed to
- Subscribe to screen state change events, such as when screen focuses on a particular element of layout, when text changes, when image scrolls, etc.
- Read all content from the active screen
Therefore, our task is fairly simple – to write our own app similar to AccessibilityService, that will track any changes on the screen and react to them accordingly (for example, by saving all accessed web link to browsing history, or writing Skype messages into a log )
- Create service class in Java as always
- Declare our service in the manifest as an AccessibilityService and specify the config file for it. In the end, record about a service should look like this:
- Create configuration file with parameters that our service should use (
Here let us look into more detail, since these attributes show the system what kind of data we want to receive in our service.
android:accessibilityEventTypes – it states that we want to receive all accessibility events, because it’s important to not miss anything ;)
android:accessibilityFlags – here we tell the system that we want to receive all the data about information displayed on the screen: layout element id, insignificant View, basically, anything.
android:canRetrieveWindowContent – this is the most important attribute! It prompts the system to provide us with the layout three with all the data.
- Create inheritance from AccessibilityService in Java code. Implement onAccessibilityEvent and onInterrupt (you can leave this empty). onAccessibilityEvent – is exactly the method that the system itself will call to store View tree in the parameters.
Now, the rest is simple. We receive an AccessibilityEvent in the onAccessibilityEven parameter.
Secondly, we can call root tree node (getSource) and go through the whole tree (getChild, getChildCount), gathering information about each element of the tree, including its class (getClassName), id (getViewIdResourceName), and even text (getText). getText method is exactly what returns messages from instant messengers or URLs from browsers.
The main difficulty in gathering the data lies in the necessity to filer various tree nodes based on the class name, id, nesting, and maybe some other criteria, specific to a particular applications.
In this example, I went across the screen layout three to search for address bar simply by the element id. I also went through it to see the structure of the screen tree and dicover a pattern that will allow me to gather Skype messages.
For example, if user opens Skype and sees the following screen:
Then our parser can take it apart and get the messages (you can identify incoming and outgoing ones based on the structure of the tree)
Below is an example of our application log for this screen:
- This approach is very simple and reliable!
- We can use this approach to both capture user data (Skype, Viber, Twitter, WhatsApp, Facebook, etc.) and control user actions (for example, prohibit user from changing particular device settings).
- No need for any permissions in manifest.
- Public API that potentially doesn’t prohibit distribution via Google Play (There is at least one other app that I know off that uses this approach to get browser history – Qustodio).
- AccessibilityService is still active even after reboot.
- There is a potential for controlling the screen of the device.
- Since this approach relies on screen parsing, there is a need for an ability to parse the screen and identify the necessary information. This will heavily vary based on the application you decide to target.
- The need to create logic for storing data, since different pieces can be accessed at different times (for example, random scroll of the Skype chat, or switching between different chats).
- Parsing large volumes of data can affect battery life. You need to carefully set time intervals between receiving events, and correctly filter target applications for monitoring.
- User needs to explicitly (only once) activate AccessibilityService in the device settings (Settings → Accessibility → $TARGET_ACCESSIBILITY_SERVICE$ → Enable).
- When target apps are updated you may need to update your parser. If the structure of the layout three changes, or even if developer simply decides to change controls id, you will need to adapt the logic of your parser to these changes.
Android security policy directly states that access to private data can be illegal, which puts certain limitations on gathering such data, including the possibility of being prosecuted by law. Therefore, use this method to obtain data consciously and thoughtfully and make sure that you abide by laws and respect privacy of others.
Get the project source code: GitHub repository.