burp proxy doesnt see the application's requests

I have followed instructions on how to add fiddler certificate on android emulator, using both nox and memu emulators, as well as my android phone running marshmallow, I set the WiFi proxy to point to my PC over the local network, when I open a website using a web browser, things work fine, I receive the warning, I choose to proceed and the connection is successfully tunneled and decrypted using fiddler.

But, when I try to use other apps, connections fail! I see the tunnel connections, and then connection fails. My bet is, it's due to the invalid HTTPS certificate, so my question is, is there a way for me to install fiddler to the trusted authorities so connecting to it will go through without the warning? So I can finally debug HTTPS traffic from and to those apps.

I found similar questions here on SO, but none of them were exactly the same as mine, nor did they have the right answers, so I'm not sure if this question does in fact qualify as a duplicate.

Thanks

On modern Android devices using apps developed for target API Level 24 (Android 7) or higher sniffing traffic is not that simple anymore. The target API level of an app is defined it's AndroidManifest.xml file in the entry <uses-sdk android:targetSdkVersion="??"/>.

The main problem is that if you install the Fiddler root CA certificate in Android it is marked as user certificate (not system certificate). And unless explicitly configured in an app those user certificates are not trusted.

One of those rare apps that respect user CA certificates is Chrome. So using Chrome for testing if the proxy and the installed root CA certificate works is a bad idea, as it may only work in Chrome but not for apps.

Note that some apps further use certificate pinning (leaf or root CA pinning). Therefore even if the Fiddler root CA certificate is installed as system certificate the app won't trust this certificate as it fails on the certificate pinning.

Certificate pinning is also a web site feature, hence some sites save a certificate hash in the web browser cache that pins the site to a certain certificate. In such a case clearing the browser cache is usually removing those pinning data.

Rooted devices

If your device is rooted you can try to install the Fiddler root CA certificate as system certificate. The Mitmproxy documentation contains a how-to for manually installing the mitmproxy certificate.

If you have rooted the phone using Magisk, there is a Magisk module that seems to be able to install user certificates automatically as system certificates: https://github.com/NVISO-BE/MagiskTrustUserCerts

Alternatively you can install Magisk + Edxposed + TrustMeAlready Xposed module. This allows to disable certificate checking system wide - WARNING: this eliminates the security of SSL/TLS against active attacks, for all apps on the phone. Therefore only do this on a device you use just for hacking!

Also possible is installing and run Frida-Server on the device and hook into the app you are interested to modify the SSL/TLS certificate checking at run-time. AFAIK the Frida based framework Objection has some scripts to do so.

Non-rooted device

On a non-rooted device there is only the option to modify the application before you install it onto the device. Note that some apps will detect that they have been modified and will refuse to work.

To let the app trust user certificates you have to modify network_security_config.xml (see e.g. here) included in the app. You can use apktool to decompile/recompile the app. Don't forget to re-sign the recompiled/repackaged app e.g. using apksigner from Android SDK.

There are some tools available that automate the decompiling , modification and signing like apk-mitm.

There is also the possibility to modify an app by including the Frida gadget for Android into the app. This would allow to use Frida for this specific app on a non-rooted device.