iptables flushed on firewalld reload

I’m running on Rocky Linux (RHEL8 based) and am kind of struggling with firewalld and iptables. Well, not really iptables anyway… Let me explain.

I setup a K8S cluster on VMWare ESXi with six virtual machines, all the same. I need to have firewalld running, given this environment. I’m using Weave CNI, which is using iptables in order to create its networking rules.

Here are my initial findings:

• on weave docker containers and VM hosts, iptables with nf_tables backend is used
  • this is confirmed by iptables -V output which gives me iptables v1.8.4 (nf_tables)
• firewalld is setup to use nftables as its backend (FirewallBackend)
• iptables service doesn’t exists (Unit iptables.service could not be found.)
• nftables service is loaded, but inactive
• firewalld service is running
• if I start nftables service, firewalld service is stopped, and vice-versa

As of my understanding of firewalld, because it uses nftables, it shouldn’t flush iptables at all, as per their blog post about nftables backend.

When weave sets its iptables rules (for routing and network policies), I can see them either with iptables-save or nft list ruleset. This means that even if weave is using iptables to set its rules, they are in fact set into nftables.

Now, my problem is that if I reload firewalld (with firewall-cmd --reload or systemctl reload firewalld.service) after weave set its rules, all rules are flushed! I can confirm that with iptables-save or nft list ruleset.

May it be related to using iptables commands instead of nft? Even if iptables uses nf_tables backend?

Please note that this problem is partially addressed by weave, because they monitor rules to see if a special (empty) chain WEAVE-CANARY still exists. If it’s not there, they recreate routing routes, but nothing is done for network policy rules. This is currently a feature request.

What are the possible solutions? I thought about hooking into firewalld service to add ExecStopPre and ExecReloadPre to save other rules than firewalld’s one into /etc/sysconfig/nftables.conf (or another file, included from there), but it’s sort of hackish and I’m not fond of this “solution”. Furthermore, I’m not even sure that this will work, given that nftables service is stopped.

Edit: I made some tests and behavior is really weird.

I added a rule with iptables: iptables -I INPUT 3 -s 10.1.1.9 -j DROP, then reloaded the firewall and the rule is not there anymore! I tried the same with nft: nft add rule ip filter INPUT ip saddr 10.1.1.9 drop, reloaded firewall and result is the same…

This is driving me crazy, firewalld shouldn’t touch other rule tables than its own, but this is not the case here!

Solution 1:

Source: GitHub issue reply from a firewalld’s collaborator

There are a couple things going on here.

• The nft variant of iptables will add rules to nftables in the known tables: filter, nat, raw, etc.
• firewalld always flushes all iptables rules (e.g. iptables -F -t filter) which flushes all chains in the filter table. This corresponds to the known nftables table names mentioned above.

Firewalld's nftables backend only touches the firewalld table, but tables filter, nat, raw, etc. are touched indirectly via iptables-nft to support iptables direct rules. This behavior cannot be changed due to compatibility. Firewalld will not touch other nftables tables though, e.g. a table named foobar would not get flushed.

Unfortunately we can't do anything here. We either a) maintain compatibility and let the iptables named tables get flushed or b) drop iptables direct rule support. Option A is the most user friendly.