This is a record of my process trying to intercept SSL traffic on Android apps.
Obtaining APK files
Most tutorials would instruct you to download the APK file from third-party sites like APKpure and APKmirror, or install the APK first on your device then pulling it to the computer using adb pull. Here I use another way: manually downloading from Aurora Store. Aurora Store is an unofficial, FOSS client to Google’s Play Store. This gives some advantages:
- Third-party APK sites might not have latest version of the APK.
- Third-party APK sites might not have the APK variant (CPU architecture, screen resolution, locale, etc) that fits your device.
- Aurora Store allows you to easily spoof device models and regions.
Steps:
- If you want to download apps only available in a specific region, connect to a VPN of that region.
- Select “Anonymous" when using Aurora Store
- Search for the app. The search result will contain apps available in that region.
- Go to the app info page.
- In the upper right 3-dot menu, select “Manual Download"
- The text field will be filled with the latest version code. If you want to download an older version you can change the code here. Note: a) old versions are not always available, b) every app has their own scheme naming the code, c) the code corresponds to
android:versionCode property in AndroidManifest. - Click “Download". This will download all split APKs (or Dynamic Delivery) that fits your device profile into
Internal Storage/Aurora/. - You don’t have to install the app, because we’re going to modify it.
Using rootless Xposed
Unmodified version of the app failed to run (black screen) on VirtualXposed.
TaiChi even failed to install the app. Taichi will ask to uninstall the app outside Taichi’s container, then after uninstallation nothing follows.
Modifying the APK
Modifying APKs usually involves these steps:
- Decompile: apktool does a good job
- Modify:
- Source code: edit smali, I don’t need it here
- Resources:
AndroidManifest.xml and other XML files, turns out there are quite some difficulty with this part.
- Rebuild package: apktool usually does a good job, but there seems to be some methods that an app developer can take to make apktool fail rebuilding.
- Signing: no problem
network_security_config
Since Android 7, user supplied CAs are not trusted in apps anymore. Most guides (like this one) will suggest you to modify networkSecurityConfig in AndroidManifest to make the app trust user supplied CAs.
Unfortunately when I do this for some apps, apktool will fail to rebuild the app. There are many strange errors, some can be solved by instructing apktool to use aapt2, but I encountered many cases where even aapt2 wouldn’t even help. So I had to seek other ways.
AndroidManifest have a complex binary format. It is usually quite easy to decode binary AndroidManifest into readable XML text, in the process the decoder (apktool) will flatten special “binary pointers to external resource files" into text paths and handle other special structures to text. But when encoding text AndroidManifest into binary, there are ambiguities to external references, causing the encoding to fail. (I don’t fully understand this part.) Fortunately, for most apps, there is another way: mark the app debuggable.
debuggable
For many apps, the networkSecurityConfig is defined like this:
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<base-config cleartextTrafficPermitted="true">
<trust-anchors>
<certificates overridePins="true" src="system" />
</trust-anchors>
</base-config>
<debug-overrides>
<trust-anchors>
<certificates src="user" />
</trust-anchors>
</debug-overrides>
</network-security-config>
This means that if the app is debuggable, it will accept user supplied CAs.
Modifying debuggable property in AndroidManifest.xml is a much smaller change than modifying networkSecurityConfig path. In the worse case I can just use a hex editor to flip a few bits. And since the change is minimal, apktool should be able to rebuild the app.
In order to modify debuggable property, I tried many tools:
But in the end I had success with Ele7enxxh‘s AmBinaryEditor. (Documentation: http://ele7enxxh.com/AndroidManifest-Binary-Editor.html )
I also wrote some shell scripts to help handle decoding, building and signing split APKs: https://github.com/pellaeon/AddSecurityExceptionAndroid
Full process
Assumption: you already have all split APKs stored in a directory apk1/
1. Decode all APKs:
~/projects/AddSecurityExceptionAndroid/splitApktool.sh decode apk1/
# Decoded apks are put under apk1_tmp/
2 . Binary edit AndroidManifest.xml using AmBinaryEditor:
cd apk1_tmp/xxxx.apk_unpack/
~/projects/AmBinaryEditor/bin/Release/ameditor attr --modify application -d 1 -n debuggable -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1
mv AndroidManifest.xml1 AndroidManifest.xml
3 . Build the APKs
~/projects/AddSecurityExceptionAndroid/splitApktool.sh build apk1/ # It will build from apk1_tmp/
4 . Install the split APKs onto the device
~/projects/AddSecurityExceptionAndroid/adbinstallsplitapk.sh apk1_new/
5 . Check if the package is installed as debuggable:
$ adb shell
j3y17lte:/ $ for p in $(pm list packages | cut -d : -f 2); do (run-as $p id >/dev/null 2>&1 && echo $p); done
com.xxx # If it shows the package id, you have success
Notes
# Uninstall APK using pm. Sometimes a package will not remove completely when you use the GUI, causing the installation to fail.
adb shell pm uninstall <com.xxx.packageid>
Forking AmBinaryEditor
During testing I fixed a few quirks of AmBinaryEditor, they are documented in the readme. https://github.com/pellaeon/AmBinaryEditor
For apps that doesn’t have debuggable property already defined
In the aforementioned scenario, debuggable property already exists in AndroidManifest.xml. But if it does not already exist, we need to add an attribute using AmBinaryEditor.
# WON'T WORK: Use this command to add an debuggable attribute to the application tag
~/projects/AmBinaryEditor/bin/Release/ameditor attr --add application -d 1 -n debuggable -r 16842767 -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1
Note: when adding attributes, we need to specify the resource id using -r and a decimal number. Refer to the android source code for system global resource ids. Resource id for debuggable is 0x0101000f , so in decimal it is 16842767.
Unfortunately the debuggable attribute would not be accepted when modified in this way. (It would still install fine but not debuggable.)
To solve this problem, I inspect the APK using aapt:
$ aapt list -v -a apk2.apk
[...SNIP]
E: application (line=106)
A: android:theme(0x01010000)=@0x7f12001e
A: android:label(0x01010001)=@0x7f110bbe
A: android:icon(0x01010002)=@0x7f0e0000
A: android:name(0x01010003)="[REDACTED]"
A: android:persistent(0x0101000d)=(type 0x12)0x0
A: android:launchMode(0x0101001d)=(type 0x10)0x3
A: android:alwaysRetainTaskState(0x01010203)=(type 0x12)0xffffffff
A: android:allowBackup(0x01010280)=(type 0x12)0x0
A: android:largeHeap(0x0101035a)=(type 0x12)0xffffffff
A: android:supportsRtl(0x010103af)=(type 0x12)0xffffffff
A: android:resizeableActivity(0x010104f6)=(type 0x12)0x0
A: android:networkSecurityConfig(0x01010527)=@0x7f150003
A: android:roundIcon(0x0101052c)=@0x7f0e0000
A: android:appComponentFactory(0x0101057a)="android.support.v4.app.CoreComponentFactory" (Raw: "android.support.v4.app.CoreComponentFactory")
A: android:isSplitRequired(0x01010591)=(type 0x12)0xffffffff
A: android:debuggable(0x0101000f)=(type 0x12)0x1
Compared with aapt output from using ameditor attr --modify on another APK with existing debuggable attribute:
$ aapt list -v -a apk1.apk | less
[...SNIP]
E: application (line=111)
A: android:theme(0x01010000)=@0x7f12002e
A: android:label(0x01010001)=@0x7f110c19
A: android:icon(0x01010002)=@0x7f0e0000
A: android:name(0x01010003)="[REDACTED]"
A: android:persistent(0x0101000d)=(type 0x12)0x0
A: android:debuggable(0x0101000f)=(type 0x12)0x1
A: android:launchMode(0x0101001d)=(type 0x10)0x3
A: android:alwaysRetainTaskState(0x01010203)=(type 0x12)0xffffffff
A: android:allowBackup(0x01010280)=(type 0x12)0x0
A: android:largeHeap(0x0101035a)=(type 0x12)0xffffffff
A: android:supportsRtl(0x010103af)=(type 0x12)0xffffffff
A: android:resizeableActivity(0x010104f6)=(type 0x12)0x0
A: android:networkSecurityConfig(0x01010527)=@0x7f150003
A: android:roundIcon(0x0101052c)=@0x7f0e0000
A: android:appComponentFactory(0x0101057a)="android.support.v4.app.CoreComponentFactory" (Raw: "android.support.v4.app.CoreComponentFactory")
A: android:isSplitRequired(0x01010591)=(type 0x12)0xffffffff
One difference I spotted is that, in the previous one, the debuggable attribute is positioned last, and in the latter one, it is positioned in the middle. And in the latter one, attributes are sorted in their resource ID (for debuggable the resource id is 0x0101000f, see the android source code for all resource ids).
Next, looking at AmBinaryEditor’s source code, in function AddAttribute:
while(1)
{
if (list->next == NULL)
{
break;
}
list = list->next;
}
list->next = attr;
attr->prev = list;
It appears that attributes are stored in linked lists, and when adding a new attribute, it is added to the end of the list. This fits our observation from aapt output.
So, in order to make it work, I need to insert the debuggable attribute in correct position. I quickly modified the AmBinaryEditor source code with a hard-coded position index 3:
for ( int i=0; i<=3; i++ )
{
if (list->next == NULL)
{
break;
}
list = list->next;
}
ATTRIBUTE *attr_orignext = list->next;
list->next = attr;
attr->prev = list;
attr->next = attr_orignext;
Then try to insert the attribute and build the APK again:
$ ~/projects/AmBinaryEditor/bin/Release/ameditor attr --add application -d 1 -n 'debuggable' -r 16842767 -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1
$ mv AndroidManifest.xml1 AndroidManifest.xml
$ cd -
$ ~/projects/AddSecurityExceptionAndroid/splitApktool.sh build apk2/
This time the attribute is correctly inserted in the middle, and successfully parsed upon installation! Success!
References
Root
With root, everything is possible. But I didn’t need to go down this path. Well, I had to admit that intercepting traffic in unrooted environment is time spent digging unimportant hole. Eventually I still rooted the phone to intercept the traffic, that is, after all, my real goal.
Magisk only
Xposed doesn’t work on Android 8.1+ yet, so if you need to get this to work, use Magisk, it works on most versions of Android.
This Magisk module will copy the user CA store into system CA store: https://github.com/NVISO-BE/MagiskTrustUserCerts
As of Magisk 20.3 . the repo above doesn’t take any effect, seemingly because it’s using an older Magisk module template that is no longer supported. A pull request is opened to fix it but has not been accepted by the original author yet. In the mean time download the module from here https://github.com/giacomoferretti/MagiskTrustUserCerts to get the working version. (Follow the Installation section to generate installable Magisk zip file.)
Note: How did I discover that the module was not working?
Go to /data/adb/modules/trustusercerts , and I found that post-fs-data.sh , which does exist in the repo, missing. If it installed properly, it should have existed in that directory.
In the end, after I finally got it to work, I found that the particular app that I was looking at seemingly employs some custom SSL pinning, so the Magisk module only allowed me to intercept some of the HTTPS messages. So I had to move on to the next approach.
- Running
android sslpinning disable from Objection shell – doesn’t work - Tried some popular Frida scripts, doesn’t work either.
EdXposed
In the end, I got everything working with EdXposed and this module: https://github.com/Fuzion24/JustTrustMe
I was able to intercept all traffic from the app.
References
General walk-through articles
Other tools
Deobfuscation
噴火獸號：裴列恩之艦 