LXC Simple Bridge Networking with existing iptables rules

Environment: RPi 3B+ , Raspbian 9

I set up LXC with simple bridge networking according to the instructions here: https://ubuntu.com/blog/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan .

Same instruction but other ways are documented as well: https://wiki.debian.org/LXC/SimpleBridge#Host_device_as_bridge

The problem I had was, after setting up, the container still couldn’t communicate with outside. I found this Q&A that taught me a nice way to test if it is firewall rules that is interfering:

echo 0 > /proc/sys/net/bridge/bridge-nf-call-iptables

This will make bridge networking bypass iptables.

After testing, simply change it back to 1.

Then on the second answer it describes my exact same problem.

So I installed iptables-persistent. Initially I couldn’t figure out the format of /etc/iptables/rules.v4. So I issued:

netfilter-persistent save

This will save the current rule into that file.

The next answer is exactly my issue. So I integrated their rule into mine and it worked:

*filter
-A FORWARD -o lxcbr0 -m comment --comment "allow packets to pass from lxd lan bridge" -j ACCEPT
-A FORWARD -i lxcbr0 -m comment --comment "allow input packets to pass to lxd lan bridge" -j ACCEPT
COMMIT

Different methods to intercept Android app SSL traffic (public version)

This is a record of my process trying to intercept SSL traffic on Android apps.

Obtaining APK files

Most tutorials would instruct you to download the APK file from third-party sites like APKpure and APKmirror, or install the APK first on your device then pulling it to the computer using adb pull. Here I use another way: manually downloading from Aurora Store. Aurora Store is an unofficial, FOSS client to Google’s Play Store. This gives some advantages:

  • Third-party APK sites might not have latest version of the APK.
  • Third-party APK sites might not have the APK variant (CPU architecture, screen resolution, locale, etc) that fits your device.
  • Aurora Store allows you to easily spoof device models and regions.

Steps:

  1. If you want to download apps only available in a specific region, connect to a VPN of that region.
  2. Select “Anonymous" when using Aurora Store
  3. Search for the app. The search result will contain apps available in that region.
  4. Go to the app info page.
  5. In the upper right 3-dot menu, select “Manual Download"
  6. The text field will be filled with the latest version code. If you want to download an older version you can change the code here. Note: a) old versions are not always available, b) every app has their own scheme naming the code, c) the code corresponds to android:versionCode property in AndroidManifest.
  7. Click “Download". This will download all split APKs (or Dynamic Delivery) that fits your device profile into Internal Storage/Aurora/.
  8. You don’t have to install the app, because we’re going to modify it.

Using rootless Xposed

Unmodified version of the app failed to run (black screen) on VirtualXposed.

TaiChi even failed to install the app. Taichi will ask to uninstall the app outside Taichi’s container, then after uninstallation nothing follows.

Modifying the APK

Modifying APKs usually involves these steps:

  • Decompile: apktool does a good job
  • Modify:
    • Source code: edit smali, I don’t need it here
    • Resources: AndroidManifest.xml and other XML files, turns out there are quite some difficulty with this part.
  • Rebuild package: apktool usually does a good job, but there seems to be some methods that an app developer can take to make apktool fail rebuilding.
  • Signing: no problem

network_security_config

Since Android 7, user supplied CAs are not trusted in apps anymore. Most guides (like this one) will suggest you to modify networkSecurityConfig in AndroidManifest to make the app trust user supplied CAs.

Unfortunately when I do this for some apps, apktool will fail to rebuild the app. There are many strange errors, some can be solved by instructing apktool to use aapt2, but I encountered many cases where even aapt2 wouldn’t even help. So I had to seek other ways.

AndroidManifest have a complex binary format. It is usually quite easy to decode binary AndroidManifest into readable XML text, in the process the decoder (apktool) will flatten special “binary pointers to external resource files" into text paths and handle other special structures to text. But when encoding text AndroidManifest into binary, there are ambiguities to external references, causing the encoding to fail. (I don’t fully understand this part.) Fortunately, for most apps, there is another way: mark the app debuggable.

debuggable

For many apps, the networkSecurityConfig is defined like this:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <base-config cleartextTrafficPermitted="true">
        <trust-anchors>
            <certificates overridePins="true" src="system" />
        </trust-anchors>
    </base-config>
    <debug-overrides>
        <trust-anchors>
            <certificates src="user" />
        </trust-anchors>
    </debug-overrides>
</network-security-config>

This means that if the app is debuggable, it will accept user supplied CAs.

Modifying debuggable property in AndroidManifest.xml is a much smaller change than modifying networkSecurityConfig path. In the worse case I can just use a hex editor to flip a few bits. And since the change is minimal, apktool should be able to rebuild the app.

In order to modify debuggable property, I tried many tools:

But in the end I had success with Ele7enxxh‘s AmBinaryEditor. (Documentation: http://ele7enxxh.com/AndroidManifest-Binary-Editor.html )

I also wrote some shell scripts to help handle decoding, building and signing split APKs: https://github.com/pellaeon/AddSecurityExceptionAndroid

Full process

Assumption: you already have all split APKs stored in a directory apk1/

1. Decode all APKs:

~/projects/AddSecurityExceptionAndroid/splitApktool.sh decode apk1/
# Decoded apks are put under apk1_tmp/

2 . Binary edit AndroidManifest.xml using AmBinaryEditor:

cd apk1_tmp/xxxx.apk_unpack/
~/projects/AmBinaryEditor/bin/Release/ameditor attr --modify application -d 1 -n debuggable -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1
mv AndroidManifest.xml1 AndroidManifest.xml

3 . Build the APKs

~/projects/AddSecurityExceptionAndroid/splitApktool.sh build apk1/ # It will build from apk1_tmp/

4 . Install the split APKs onto the device

~/projects/AddSecurityExceptionAndroid/adbinstallsplitapk.sh apk1_new/

5 . Check if the package is installed as debuggable:

$ adb shell
j3y17lte:/ $ for p in $(pm list packages | cut -d : -f 2); do (run-as $p id >/dev/null 2>&1 && echo $p); done
com.xxx # If it shows the package id, you have success

Notes

# Uninstall APK using pm. Sometimes a package will not remove completely when you use the GUI, causing the installation to fail.

adb shell pm uninstall <com.xxx.packageid>

Forking AmBinaryEditor

During testing I fixed a few quirks of AmBinaryEditor, they are documented in the readme. https://github.com/pellaeon/AmBinaryEditor

For apps that doesn’t have debuggable property already defined

In the aforementioned scenario, debuggable property already exists in AndroidManifest.xml. But if it does not already exist, we need to add an attribute using AmBinaryEditor.

# WON'T WORK: Use this command to add an debuggable attribute to the application tag
~/projects/AmBinaryEditor/bin/Release/ameditor attr --add application -d 1 -n debuggable -r 16842767 -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1

Note: when adding attributes, we need to specify the resource id using -r and a decimal number. Refer to the android source code for system global resource ids. Resource id for debuggable is 0x0101000f , so in decimal it is 16842767.

Unfortunately the debuggable attribute would not be accepted when modified in this way. (It would still install fine but not debuggable.)

To solve this problem, I inspect the APK using aapt:

$ aapt list -v -a apk2.apk
[...SNIP]
    E: application (line=106)
      A: android:theme(0x01010000)=@0x7f12001e
      A: android:label(0x01010001)=@0x7f110bbe
      A: android:icon(0x01010002)=@0x7f0e0000
      A: android:name(0x01010003)="[REDACTED]"
      A: android:persistent(0x0101000d)=(type 0x12)0x0
      A: android:launchMode(0x0101001d)=(type 0x10)0x3
      A: android:alwaysRetainTaskState(0x01010203)=(type 0x12)0xffffffff
      A: android:allowBackup(0x01010280)=(type 0x12)0x0
      A: android:largeHeap(0x0101035a)=(type 0x12)0xffffffff
      A: android:supportsRtl(0x010103af)=(type 0x12)0xffffffff
      A: android:resizeableActivity(0x010104f6)=(type 0x12)0x0
      A: android:networkSecurityConfig(0x01010527)=@0x7f150003
      A: android:roundIcon(0x0101052c)=@0x7f0e0000
      A: android:appComponentFactory(0x0101057a)="android.support.v4.app.CoreComponentFactory" (Raw: "android.support.v4.app.CoreComponentFactory")
      A: android:isSplitRequired(0x01010591)=(type 0x12)0xffffffff
      A: android:debuggable(0x0101000f)=(type 0x12)0x1

Compared with aapt output from using ameditor attr --modify on another APK with existing debuggable attribute:

$ aapt list -v -a apk1.apk | less
[...SNIP]
    E: application (line=111)
      A: android:theme(0x01010000)=@0x7f12002e
      A: android:label(0x01010001)=@0x7f110c19
      A: android:icon(0x01010002)=@0x7f0e0000
      A: android:name(0x01010003)="[REDACTED]"
      A: android:persistent(0x0101000d)=(type 0x12)0x0
      A: android:debuggable(0x0101000f)=(type 0x12)0x1
      A: android:launchMode(0x0101001d)=(type 0x10)0x3
      A: android:alwaysRetainTaskState(0x01010203)=(type 0x12)0xffffffff
      A: android:allowBackup(0x01010280)=(type 0x12)0x0
      A: android:largeHeap(0x0101035a)=(type 0x12)0xffffffff
      A: android:supportsRtl(0x010103af)=(type 0x12)0xffffffff
      A: android:resizeableActivity(0x010104f6)=(type 0x12)0x0
      A: android:networkSecurityConfig(0x01010527)=@0x7f150003
      A: android:roundIcon(0x0101052c)=@0x7f0e0000
      A: android:appComponentFactory(0x0101057a)="android.support.v4.app.CoreComponentFactory" (Raw: "android.support.v4.app.CoreComponentFactory")
      A: android:isSplitRequired(0x01010591)=(type 0x12)0xffffffff

One difference I spotted is that, in the previous one, the debuggable attribute is positioned last, and in the latter one, it is positioned in the middle. And in the latter one, attributes are sorted in their resource ID (for debuggable the resource id is 0x0101000f, see the android source code for all resource ids).

Next, looking at AmBinaryEditor’s source code, in function AddAttribute:

        while(1)
        {
            if (list->next == NULL)
            {
                break;
            }
            list = list->next;
        }
        list->next = attr;
        attr->prev = list;

It appears that attributes are stored in linked lists, and when adding a new attribute, it is added to the end of the list. This fits our observation from aapt output.

So, in order to make it work, I need to insert the debuggable attribute in correct position. I quickly modified the AmBinaryEditor source code with a hard-coded position index 3:

        for ( int i=0; i<=3; i++ )
        {
            if (list->next == NULL)
            {
                break;
            }
            list = list->next;
        }
        ATTRIBUTE *attr_orignext = list->next;
        list->next = attr;
        attr->prev = list;
        attr->next = attr_orignext;

Then try to insert the attribute and build the APK again:

$ ~/projects/AmBinaryEditor/bin/Release/ameditor attr --add application -d 1 -n 'debuggable' -r 16842767 -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1
$ mv AndroidManifest.xml1 AndroidManifest.xml
$ cd -
$ ~/projects/AddSecurityExceptionAndroid/splitApktool.sh build apk2/

This time the attribute is correctly inserted in the middle, and successfully parsed upon installation! Success!

References

Root

With root, everything is possible. But I didn’t need to go down this path.

Xposed doesn’t work on Android 8.1+ yet, so if you need to get this to work, use Magisk, it works on most versions of Android.

Magisk module: https://github.com/NVISO-BE/MagiskTrustUserCerts

References

General walk-through articles

Other tools

Deobfuscation

KDE Plasma VNC remote session

  • Remote VNC get new session separate from local console
  • Session will remain after disconnect by closing the window
  • Server will be terminated upon logging out from start menu
  • Need SSH forwarding for security
  • Screen resolution cannot be changed
  • Remote user logs in directly with VNC password, no user password needed
  • Need to generate VNC password using `vncpasswd` for the first time

Reference: https://wiki.archlinux.org/index.php/TigerVNC#System_mode

 

Upgrading my LineageOS on SM-G930F

  1. Download and install latest version: https://forum.xda-developers.com/galaxy-s7/development/lineageos-15-android-8-0-0-herolte-t3710107
  2. Check if TWRP is updated: https://dl.twrp.me/herolte/
  3. Uninstall Magisk (seems that sometimes you need to do it twice)
  4. Install latest Magisk: https://forum.xda-developers.com/apps/magisk/official-magisk-v7-universal-systemless-t3473445
  5. Update OpenGapps – arm64 nano
  6. Install Xposed-magisk from Magisk Manager
  7. Install modified Xposed installer https://forum.xda-developers.com/xposed/unofficial-systemless-xposed-t3388268/post67074423#post67074423

6/29

After installing new version of LineageOS. Apps that rely on Google Service Framework always crashes. Usually disabling Magisk and re-enabling works, but this time it doesn’t. In the end upgrading the XprivacyLua module solved the issue.

 

台灣的數位政委談如何不使用審查來打擊不實訊息—個人想法

後話

(因為我覺得如果有人只看我原文沒看後話的話會誤會所以我把後話放在前面)

寫完上面那些之後,我去搜尋了一下有沒有唐鳳跟 FB、LINE 就報導主題對談的紀錄,因為唐鳳大部分的對談都是有公開逐字稿的。結果我沒找到,只有 2017 年 FB 去拜訪他的紀錄。

不過我倒是找到當初受訪的逐字稿

我只有快速看過一次,發現幾個亮點:

一,其實記者有問到這個政府快速澄清的機制是否有可能被濫用。

但唐鳳的回答我覺得很糟,根本沒有談到怎麼防止濫用,記者也沒有追問下去。目前就我所知,沒有防止政府濫用的機制。

二,記者有問到,政務委員羅秉成防治不實訊息的框架(大概是指這個是否在未來有可能傷害媒體自由

唐鳳的回答基本上是說這個機制不會影響大部分的媒體,而且羅委員背景是人權律師,所以他很重視人權。這回答我覺得也蠻糟的,這個法律框架目前看起來沒有考慮到要怎麼對付未來上台的惡意執政者。

原文

看到 CPJ 這篇很紅,再看了 HN 底下很多留言,記一下想法。

留言區有很多我認同的留言,先畫一些重點。

What is described is better than China, but it’s not ideal. However I do think they should be commended from explicitly avoiding censorship to the degree that they do.

However, communications from the government to counter other communications identified as disinformation are government propaganda. If they are not identified as such, then they have the same problem as the original disinformation.

Secondly, making it easy to flag propaganda in apps and then blocking it as she describes is in fact building in a type of censorship. It’s just a brilliant type of crowd-sourced censorship. I know the theory is that that would only be applied to false information. However, censorship doesn’t work like that. The end result is going to be a worldview that is reduced and shaped by a dominant group rather than the free flow of information. Especially if it relates to all of the information that people in a country can see. This can be very dangerous.

So I will just say what I think would be ideal even if it may not be very realistic. The history of government propaganda and censorship should be part of the public context. I think it is good to be able to flag articles as being propaganda and from what source. However I think that removing things from being visible entirely is very dangerous for freedom. Taiwanese/US/Chinese/etc. government propaganda or counter-propaganda should be identified as such if possible.

 

This gives me some weird feelings. I mean, if you ask a website to turn some content “unpreferred" without user consensus, isn’t that also some type of censorship?

 

> How is that reconciled? How does one know when the government is being truthful versus being propagandists?

If there are multiple conflicting narratives, at least some people will be motivated to do more research to resolve the contradiction. If the government is being more truthful, its statements can point to resources that provide independent verification, accelerating the process. At a minimum that will blunt the effect of the lies.

這段我倒是認同,如果有多個衝突的說法,至少有一小部分人會開始自己研究。有一部分人則是相信任何最新的反駁論點;當然還有最後一部分是相信他們所信任的來源的一切訊息。可惜後面兩種人加起來是多數。不過我想至少政府澄清可以增加衝突的說法,促使更多人開始去自己研究。

> Once they do that, Facebook promises, by June, that this will inform the Facebook’s algorithm so that it will stop being preferred to show on people’s newsfeed, but it’s not censorship. If you look specifically for that friend, that post is still there, but they have a warning that says it’s already fact-checked as false.

That’d be very interesting if the Taiwanese government was in direct communication with Facebook over something like this. CPJ wasn’t able to get a hold of Facebook, I wonder if anyone here has visibility into something like that? Given that the US government took a more “combative" approach (dragging executives in front of Congress) I’d be curious how more tame approaches like this were being received.

我覺得台灣政府和 FB 在這件事情的合作上面透明度不夠。

This is much better than removing the post, but it still makes me nervous. Just imagine that we’re in the early 2000s, and you can envision a banner saying “The claims in this article are FALSE. As confirmed by the intelligence community and the New York Times, Iraq is actively working on its nuclear weapons program." Perhaps they should have a policy limiting the use of this tool when it comes to issues that may lead to war.

 

>notably, for all its merits, the taiwanese system still positions the government in an information vetting capacity, which may lead some to think that it is merely another system of propaganda or quasi-censorship

This is kind of how I see it too. I think if your goal is to combat disinformation or propaganda you should just come out straight up and say it because it’s a perfectly defensible position to hold, but creating a government counter-narrative isn’t any more harmless than just censoring something, it’s just sounds nicer than having to use the word “censor" which tends to make people in some parts of the word jump up in panic.

整篇讀起來我其實覺得一部分她做的、這件事情的影響,是負面的。

  1. 權力傾向被濫用

政府有了這樣跟 FB 和 LINE 合作的權力,現在打擊不實訊息效果可能很好,現在的政府可能也意圖良善,但是哪天換了一個政府呢?就想,如果國民黨上台,他們會怎麼用這個機制?這些合作的公司又有多少獨立的力量可以抵抗來自政府的要求?

這機制雖然不是審查,但是確實是政治宣傳(Propaganda),現在政治宣傳還沒有被濫用,但政府已經獲得做政治宣傳更大的能力了。

  1. 知名度集中在唐鳳身上,知名度會給一個人更大的權力

建立這個機制的背後肯定是很多人,首先肯定是有總統的支持,還有各個行政部門首長也要配合,忽略團隊不是一件好事。尤其這篇報導是針對外國人的場合,唐鳳也並未利用此機會宣傳能夠讓她這麼做的政治系統和內部機制和合作伙伴。

  1. 台灣政府和 FB 和 LINE 的合作並不透明

這些應該要是透明的:

  • 合作的範圍和具體定義
    • 原文 “Facebook promises, by June, that this will inform the Facebook’s algorithm so that it will stop being preferred to show on people’s newsfeed” 這裡的 preferred 是什麼意思?減低多少程度?
      • 不過從另一方面想起來,依照 FB 過去的糟糕紀錄,台灣政府恐怕也不知道這個詞是什麼意思,因為 FB 公共關係部門不會跟他們講清楚的。而且 FB 的公共關係部門大概也不知道這詞到底代表什麼具體的程式行為。
      • FB 演算法相關資訊的透明度大概比北韓政府還低,台灣政府選擇跟這個不透明的公司建立合作而且也沒有立法要求透明,很失敗。(放任壟斷市場的企業繼續不透明,還跟他們合作)
    • LINE 也是
  • 政府和平臺中間具體的溝通機制
    • 溝通管道
    • 主管
    • 實行細則、SOP
    • 決策流程
  • 民間團體的角色
    • 如何監督政府和 FB?
    • 是否具有制衡力?

整體而言,我覺得這件事情有助於打擊不實訊息,但是減損我們的民主和自由。

 

Setting up Nextcloud 16 on Synology Webstation

Webstation configuration

Follow http://andreasschmid.com/2018/12/24/nextcloud-on-synology-nas/ , but I used nginx and PHP 7.2 instead. And of course don’t use the mysql root user for nextcloud.

Configure SSL certificate

Basically follow : https://stefandingemanse.nl/how-to-use-lets-encrypt-ssl-certificate-on-synology-dsm/

But in the “Security > Certificate" settings you need to enable the certificate for Webstation. Do this by selecting the Let’s Encrypt certificate, click Configure, and select to use this cert for your virtual host.

Failed to start session error

It is caused by Webstation has enabled PHP open_basedir restrictions. In the PHP profile you need to configure /var/services/tmp:/dev/urandom:/tmp

I got this from: https://webcache.googleusercontent.com/search?q=cache:A8qy2RWRhUgJ:https://www.synology-forum.de/showthread.html%3F99836-Nextcloud-Problem-mit-PHP-7-2+&cd=3&hl=zh-TW&ct=clnk&gl=tw&client=firefox-b-d

‘SQLSTATE[HY000] [2002] No such file or directory’ on Nextcloud initial config page

Initially for the database host field I entered localhost:3307 and had the error ‘SQLSTATE[HY000] [2002] No such file or directory’ . Following the idea here to change it to 127.0.0.1:3307 gave a success.

`nginx` configs for Nextcloud

In the beginning the login page index.php/login will be 404, because all the rewrite rules are not in place yet.

Take reference from Nextcloud documentation, put configs without the server block into /etc/nginx/conf.d/YOUR_UUID/user.conf like this. YOUR_UUID will be different on each machine.

Also need to put in php-handler setting in /etc/nginx/conf.d/http.php-handler.conf like this. It is actually included from nginx.conf with include conf.d/http.*.conf

Restart nginx: sudo synoservice --restart nginx

Now you should be able to access the login page!