Case Study: MobileTriage – JPEG Metadata Triage – Automated Extraction of Information using Mobile Forensic Tools
Mobile devices are now becoming part of our daily lives. These devices can now perform tasks or activities that were normally being processed in a normal desktop or laptop computers. Attachment to such devices is immensely increasing in people’s lives.
From a digital forensic perspective, these devices hold an enormous wealth of information concerning someone’s activities whether it is personal or work related. The mobile devices are commonly being treated as mini computers as they possess a similar processing ability as normal computers would only on a smaller scale. However, in this setting, these devices are commonly portrayed as objects to the investigation. Suspected crimes or fraudulent activities can often be found in these phones or tablets, and they normally tell a story of what might have had happened.
My recent MSc. dissertation project investigates the possibility of mobile devices of becoming a subject or tool to aid digital forensic investigations. Somewhat shifting the paradigm of viewing the device from another angle or perspective and see it differently. In this particular setting, the device will no longer be treated as an object to investigate; instead it provides the investigators with digital forensic tools capabilities while embracing unique features of smartphones or tablets. Someone will not always carry their forensic workstations with them, how about the likelihood of a phone or tablet with a digital forensic app?
The initial idea was raised from a discussion I had in class during the course of my study at the University of Westminster. The functions and features of a portable forensic duplicator can potentially be brought to mobile devices, specifically Android which is built using a simplified Linux kernel. In addition, the USB On the Go (OTG) functionality has made plugging multiple USB sticks or memory cards (even portable HDD in some cases) possible to certain model of Android devices (http://www.usb.org/developers/onthego). Why not further utilise this capability in Android to mimic the portable forensic duplicator functionalities and possibly create forensic images of small storage devices? With this being said, some of the core bit level copy and verification tools such as dd and hashing commands are already made available through a number of Terminal apps in Android. Considering some of the existing open source forensic tools are built for GNU Linux and UNIX, I started off my experiments by trying to cross compile the source code of those tools into binaries that can be executed in Android. Some succeeded and some were not.
Following a further discussion with my project supervisor, emerged another idea of leveraging the computing capability of these small devices in a triage situation. A setting of where streamlining several processes together to reveal information fairly quickly. This with the assumption that all of the normal forensic workstations are occupied processing other stuff, some USB sticks or memory cards will still need to be analysed for information. The additional resource of mobile devices can potentially be utilised in these types of situation to extract contents of these small-scale storage devices.
This article contains the idea of the project and the implementation of the proof of concept tool that was built under the Android platform. The tool focuses on extraction and analysis of JPEG picture files from either digital cameras or smartphones. The result was a prototype of an Android app that can analyse the content of USB sticks, memory cards or raw forensic images by extracting image/picture files; visualise them as soon as they are extracted and reveals their metadata information as well as translating some of the metadata content into something meaningful. All of these activities are tied together in one simplified work flow.
A developed Android application package (.apk) for installation, an example dataset and complete source code of the tool is currently available on Github (https://github.com/andritona/MobileTriage) to download under GPL license. A simple documentation to install the app and build or compile the source code is included. The app is designed to work for Android version 2.2 to 4.4 for both phones and tablets. Although some of the layout elements may need to be readjusted on certain devices.
Requirements and Design Components
The proof of concept tool or app will need to have the ability to process picture files content, extract EXIF metadata and translate some of the items into information that can lead someone to reach certain decisions quickly. The following requirements were constructed in order to meet these objectives:
- Locate any JPEG picture files (both allocated and non allocated (deleted), both complete and partial or corrupted JPEG files).
- Decode the extracted JPEG picture files in parallel to the data extraction process in order to be visually analysed and bookmarked.
- Extract EXIF metadata content and thumbnail data from the selected or bookmarked images, provide the ability to convert the GPS tags into location coordinates and further translate the results into an actual address.
- Provide a method to view the extracted GPS location coordinates in a map application to visually pin point the address.
- Multi-threading capability to allow parallel processing of JPEG data and metadata extraction as well as visualisation.
- Provide a simple and intuitive user interface (UI).
Method of Data Extraction
Data carving was chosen as the preferred method to extract JPEG files data. Although the choosing did come with certain compromises such as the absence of file system metadata (filenames, timestamps, size and deletion status) which presumably could provide further insights to an investigation. However, due to the limited time constraints I had with the project, the approach seemed to be ideal at the time.
An example implementation of data carving project using Java was incorporated into the tool (http://web.cs.du.edu/~mitchell/forensics/projects/data_carving/example_java_carver.zip). The code was examined, modified and optimised thus enabling the function to run properly within Android. This was a huge time saver for me.
Android provides the ability of parallel processing using native Java thread method or a simple and ready to use Android thread pool method (http://developer.android.com/reference/android/os/AsyncTask.html). The latter will provide an Android managed threads or to execute any batch processes that we throw at it. Basically, it will just process any incoming requests and assign any of the available threads. Luckily, there are enough resources on the Internet and numerous good video tutorials on how to understand and achieve both approaches. (For someone who has no prior experience of using both Java and Android development, I found video tutorials to be most helpful).
Visualisation + Metadata Extraction and Translation
Fortunately (again), Android provides various built-in libraries required to perform the remaining tasks or activities for the proof of concept tool. Guidance on how to use these libraries are well documented in the official Android website, however finding examples on how to apply certain methods was a different story. One of the libraries used in this implementation was for the EXIF extraction functionality (http://developer.android.com/reference/android/media/ExifInterface.html). The built-in library in Android only provides a small number of EXIF tags available, however I found that the tags were sufficient enough to represent the capability and well within the project scope.
Putting Everything Together
A simple pre-configured work flow was constructed with the idea that the overall process has to be simple enough so that no one would struggle in using the tool. Therefore, allowing even the less experienced individuals to perform such task. Individuals can focus more on the outcome or results rather than spending precious time on how to navigate the tool. This is the base concept that shaped and formed the intuitive user interface (UI) requirement.
From the outline of the work flow, the individuals will need to do the following:
- Load the evidence (this can be a block/raw device or a raw dd image file)
- Visually review and bookmark images of interests.
- View the final report containing bookmarked images along with their metadata content.
The rest of the processes in the work flow are simplified and automated by the proof of concept tool.
The Proof of Concept Tool – MobileTriage App
1. Load the Evidence File
After opening the app, the user will be presented with a prompt to select electronic evidence file. This can be a raw dd image file or a block device file in Android, assuming the appropriate permission is set for the corresponding file. After selecting the file, the carving process will immediately execute and read the file input searching for JPEG EXIF signature.
Accessing raw block device from an app in Android requires root level privilege over the device, otherwise the intended block device will not be accessible to read. There also a number of good tutorials on how to obtain root level access over a specific device, however this may void the manufacturer’s warranty of the device.
I have discovered that using a block device to be read in Android app requires the permission of the block device file to be modified in order to allow other users apart from root to read the file. Mainly because each app is treated as individual user in the ecosystem (https://source.android.com/devices/tech/security/). This can be achieved by incorporating the elevated chmod command in the shell through the Android app for the corresponding block device. However, I do believe that this is not ideal but can be considered as a temporary or workaround solution. I have not yet found other alternatives to this at the moment.
For the purpose of the project, I decided to use a dd image file placed in the internal storage of the device instead of reading from raw/block device in order to get more testing done on different types of phone and tablet. Therefore, the non-rooting route was the preferred method of choice.
2. Data Extraction and Visualisation
After the process finds and extracts a carved file, the file will immediately be decoded as a picture to be visually analysed while continuing the carving process. These both processes run in parallel to one another with the use of threading. Each individual picture can be selected and will invoke the Android internal picture viewer application (another app can be selected as default, for example the Gallery app). This will give the user further flexibility of using the pinch to zoom in functionality for closer inspection of picture files.
User will also be able to select or bookmark images of interests to be short listed for report generation which will immediately trigger the metadata extraction and translation process. The swipe gesture functionality provides a method in browsing or scrolling through the number of images in a horizontal movement fairly quickly.
3. Metadata Extraction and Translation (Report)
EXIF metadata content and thumbnail data of the bookmarked images are then extracted in the report view of the app. Some of the values are translated into useful information. In this particular instance, the EXIF GPS tags are converted into location coordinates format which further translated into addresses values by using the built-in Android’s reverse geocoding capability (http://developer.android.com/reference/android/location/Geocoder.html). The ability to display the app in both landscape or portrait mode and the vertical gesture swiping functionality also provides greater flexibility in viewing the report.
Each of the item that has GPS or location coordinates can also be tapped or selected to invoke the built-in Google Maps application. This will provide the visual pinpoint location of where the picture was presumably taken.
In one of the examples, a possible corrupted or edited picture may still has the original thumbnail data intact. In addition to this, the address location of the where the picture was taken can potentially be more useful than the set of numbers in the GPS tags. All these values can possibly give inferences to reach certain decisions quickly.
4. Visual Location in Map Application
The visual location or address of a picture may provide certain benefits to the individuals. Surrounding areas in close proximity displaying various landmarks, bars, restaurants, schools, offices and any other place of interests can be observed. This might end up giving further clues that can be followed up in certain event reconstruction scenarios. Of course these are some of the few assumptions that I had to take in when constructing the project as they might differ in real investigations.
Known Issues and Findings
1. Data Carving Performance
The performance of the data carving process is not satisfactory. The same carving code takes almost 10x faster in a laptop running OpenJDK on GNU Linux 64bit. Although the hardware and software specifications might not be an apple to apple comparison but this can roughly represent the behaviour. This is likely because the use of DalvikVM or Android Runtime that act as VM to the Java code is not meant to perform complex computational operations (https://developer.android.com/tools/sdk/ndk/index.html). It also adds another layer of execution between the code and the device.
One way to possibly mitigate this is by outsourcing the intensive portion of the computation into a more low-level language such as C in Android. In theory, this can be done using the Android Native Development Kit (NDK) to allow interoperability between the normal app and the native code. An experiment of performing simple Fibonacci algorithm was conducted using both Java and C natively in Android (https://thenewcircle.com/s/post/96/using_ndk_for_performance_dalvik_versus_native). The native code was almost 17x faster in its execution. The results seem promising enough to be applied for the concept and therefore bypassing the requirements of data carving through Android VM using Java code.
Another way to resolve the issue is probably to implement a FAT32 file system parser. In theory, this would provide both allocated and non allocated (deleted) files and skip the need for the data carving functionality. In addition, it will also provide file system metadata that contains filenames, timestamps, size, location and also deletion status. The latter can be quite useful to demonstrate the existence of deliberate intent to remove certain files.
The assumption taken in this regard is that the data is stored in a contiguous non fragmented manner. Both approaches can potentially worth to be considered, however, due to time constraints and my lack understanding of C programming language, both of them were not pursued further in the project.
2. Access to Raw/Block Device
I have not yet been successful in finding ways to allow Java Android app to read the content of the block device or files that are owned by root. However, on a rooted Android device, it allows the su -c command to be invoked with another binary to elevate its privilege from within the app, but not elevating the app’s privilege itself. The chmod command to modify the permission of the block device was tested using this method.
A mount command can be invoked using a Terminal app or adb shell to identify the location of a block device and its mount point. It was observed that the raw or block device for the USB OTG will appear under the /dev/block/vold/ on the Android device using a numbered scheme. For example /dev/block/vold/8:0 will be the entire storage device and /dev/block/vold/8:1 is the first partition on that particular device. Root level privilege is required to access the content of the parent location.
Anther alternative will be to have the whole code developed natively using C and not the Java SDK within Android. This approach will likely loose the Android’s graphical user interface unique capabilities as everything will need to be executed through a Terminal app. This is done by issuing series of commands under root user privilege as they would on GNU Linux or UNIX systems.
3. USB On The Go (OTG) Auto Mount Feature
In my testing, I found that the USB OTG functionality tries to auto mount a device when it is plugged or inserted to the interface such as USB OTG cable or powered hub. By default this behaviour only happens on FAT file systems variants. Unfortunately, these are exactly what most USB sticks or memory cards from digital cameras or smartphones be formatted under. This behaviour will ultimately tamper the integrity of the evidence in question.
There are some good discussions in the forum (http://forum.xda-developers.com/nexus-4/orig-development/usb-otg-externally-powered-usb-otg-t2181820) on ways to enable or disable the auto mounting feature. The discussion talks about the techniques to modify initialisation files for mounting block device as well as performing minor modification to the kernel itself. Both methods likely require the mobile device to have root level privilege. Disabling the auto mount feature should be the main aim of the approach, although I believe that performing an auto mount with read only access will still be an acceptable solution.
Another alternative that I could possibly think of is to integrate a portable or mini USB write blocker to the USB OTG interface. In theory, this could work and will not need any sort of modifications to the Android kernel or configuration files. Although, root access may still be required for accessing the raw or block device. Again, due to resource constraints I have not personally tested both of these approaches.
Challenges and Ideas for Improvements
There are a number of ideas that I believe can be useful and further implemented within the app. These items include:
- FAT32 File system parser to extract file content and system metadata, allowing for faster retrieval and processing of small-scale storage devices.
- Ability to extract more EXIF metadata tags from various different manufacturers or to integrate the existing EXIF extraction open source tools into the app.
- Ability to create forensic duplication from the raw or block device which would provide bit stream copy of the device along with verification information such as timestamps and hash values.
- Support for dynamic user configuration settings from within the app, currently some of the values are still hard-coded into the source code. These values are set as global variables that can be modified.
- Ability to use compact database such as SQLite to store the extracted results. This would provide scalability and further flexibility.
- Ability to extract the multiple coordinates into Google Earth’s KML format (https://developers.google.com/kml/documentation/mapsSupport). This would enable multiple locations of the images to be displayed in the map application, thus might be helpful in certain location-based pattern analysis.
- Ability to export the report results into known formats such as Portable Document Format (PDF) and send or share the results through email or other app. In theory, this would provide information exchange to happen quickly between individuals (Assuming preventive measures were taken beforehand to protect and secure the information both at rest and in transit).
The app is far from perfection. There are still a number of findings and improvements that need to be resolved before thinking about putting the tool to a good use.
The MobileTriage app is just an example of how the concept can be implemented. Mobile devices with their unique features in various types of platform can potentially be utilised to assist in processing electronic evidence. Development of digital forensic tools under this platform can open up new possibilities, methods and approaches that can be applied following certain limitations. Being aware of such limitations can provide new opportunities as well as challenges for digital forensics practitioners, students and community to develop good mobile forensic tools that are acknowledged by the profession and industry.
Thank you all very much for reading the article, I welcome any discussions or comments regarding the idea. I certainly hope that the article can be useful and wish that more ideas can be derived from this work.