ikev2provider blocking connections - misconfigured vpn settings - stuck on the same IP

I am currently struggling with very weird macOS problems. I am running Catalina on a macbook pro 2018. For the past few days I have experienced connection issues:

• The first problem was when I returned to work after vacation and could not connect to any of our internal services.
• A quick check showed that I was getting an external IP from Denmark. Alright that is unexpected. Then I used some ip checker tools to get some more informations, and I saw that the IP was one of NordVPNs servers.
• Alright - so I don't seem to have a malware proxy (what I first thought) but rather a misconfigured VPN connection. The VPN client app was not active at this point and every manual vpn connection in the network app was inactive or disabled.
• So now I knew the problem seems to be NordVPN, so I deleted the app, deleted every network adapter vpn setting, deleted all places where I could find nordvpn files

Places where I deleted nordVPN related stuff:

- ~/Library/Application\ Support/  
- ~/Library/Caches
- ~/Library/Logs
- ~/Library/Preferences
- ~/Library/Containers

I also tried restarting the mDNSResponder several times with sudo killall mDNSresponder and I tried deleting the network configs in ~/Library/Preferences/SystemConfiguration:

com.apple.airport.preferences.plist
com.apple.network.identification.plist
NetworkInterfaces.plist
preferences.plist

• Afterwards I restarted the machine and it seemed to work, but only for some time (minutes to hours) before my IP changes again and a VPN connection is active.

• So it seems like somewhere in the system there are still credentials/certificates stored. So I opened the mac keychain and deleted every nordVPN credential and root certificates. Another restart later the problem has changed a little bit:

Now I could not get ANY connection (I could use ping and would get the expected output but could not use curl or any browser). So somehow the approach with deleting all credentials seemed to have some effect.

• For the next step I wanted to find out which process was starting up the VPN connection. So I consulted the system.log and the macOS console. I've seen many many entries from the IKEv2Provider service (the mac vpn service process) with a status like "Tunnel:NordVPN:"some hex code here" with the Description: "Received a start command from "programXY[pid]"" Unfortunately I don't have the exact wording as I restored the complete machine in the meantime to factory settings and did not save the logs. There was another logfile where it also mentioned the IKEv2Provider process quite often with something like: process remained dirty for too long. Killing which showed up many times too.

So the problem seems to be this process, but it seems to get the start command from random apps, I was seeing processes like calendar.agent, spotify, firefox etc. starting it up.

• So that was the last step before I said screw it and restored the machine to factory settings. Another problem is that the issue persists when I restore from my timemachine backup, either I need to find an older backup which is not compromised or I need to restore only files and apps but not system settings? I will try that out later today.

I've already somehow settled with a factory reset and a partial-recovery of my important things like dotfiles, IDE settings and stuff, but my system settings will be lost, so some kind of solution would be preferred.

I can always go back to the "corrupted" backup, so if anyone here has a idea what else to do or has every heard of something similar I would love to hear your tips, and if it's promising I can restore the machine to the faulty state and try out some more things.

Additional Info:

• At some point deleting and re-adding all wifi connections seemed to work, but only for some time until the VPN kicked in again.
• I have contacted NordVPNs support, but so far I only received the usual tips, maybe they will escalate the ticket to a dev.
• When I had the NordVPN client still installed and clicked on Quick Connect it connected to the server I was already on! So my IP did not change at all but the Client reported that it is connected. Connecting to different countries still worked, I would get a UK-IP when I connected to the UK, but when I disconnected I still was on the NordVPN IP from Denmark. What I would expect from Quick Connect (And experienced in the past) is that it connects to a German or Switzerland server (I am located in southern Germany) but never to a server in Denmark.
• All of this stuff happened after I was vacationing in the Netherlands. I heavily used my VPN there (Lots of open wifis) and it feels like that is part of the reason it won't disappear afterwards. (When located in the Netherlands getting a quick-connect server in Denmark would make a little bit more sense)
• What bugs me out the most is that the logs showed Tunnel:NordVPN AFTER I deleted every file/keychain entry/certificate mentioning NordVPN, so there has to be some kind of hidden entry somewhere which I can't find.

I am sorry for the long text, but I tried collecting all the steps I already tried out, I think it is not an common problem but I hope that we somehow find a solution, I would love to avoid restoring only part of my files, and use my complete backup.

Solution 1:

I found the problem myself:

The culprit was /Library/Preferences/com.apple.networkextension.plist - there were entries for the IP I was always conecting too, my vpn login email and strings like "nordvpn.osx". (Not to be confused with the path ~/Library/Preferences!) I deleted everything related to vpn and after a restart it worked again. I needed to download XCode to edit the .plist, and you have to copy it to a directory with r+w permission, and copy it back to the protected Preferences directory after editing.