Cannot port forward with UFW on Ubuntu

I am trying to get port fowarding to work to expose a LXC container, when using ufw on Ubuntu, but to no avail. I have constructed the following workflow from guides around the Internet ([1][2][3]); most notably the section about /etc/ufw/before.rules I see repeated all over the place but .... I cannot get it to work for me.

If anybody follows this guide and finds success.... do let me know (so I may first weep, and then fall back and write my own firewalling tool)

Steps to reproduce/test

Setup

1. Test environment

   • New VM (I used virtualbox) "host1" with both a NAT and NAT Network interface
     • Install Ubuntu server 16.04 - just do a default installation
   • Second VM instance "host2", on same NAT Network

2. LXC installation on host1 - I am doing this with plain LXC, not introducing LXD setups yet

   • sudo lxc-create -n web -t download -- -d centos -r 7 -a amd64
   • sudo lxc-start -n web
   • sudo lxc-attach -n web
     • yum install httpd ; yum enable httpd ; yum start httpd
     • exit

We assume now that

• NAT Network interface on host1 is enp0s8 and
• the container IP on host1 is 10.0.3.101
• the IP of host1's enp0s8 is 10.0.2.6
• See that with sudo iptables -L -t nat that the masquerading has already been configured by the LXC install

You may want to snapshot this base state.

At this point two routes: A-route with plain iptables, and B-route, the problematic one, with ufw

3. A - iptables
   • sudo iptables -t nat -A PREROUTING -i enp0s8 -p tcp --dport 8080 -j DNAT --to "10.0.3.101:80"
   • On the second host I can wget http://10.0.2.6:8080 and this gives me the standard CentOS landing page. Success!

The problem

3. B - this can be done after a reboot of host1 or, if you snapshotted earlier, just revert. Either way, it behaves the same
   • Edit /etc/default/ufw
     • set MANAGE_BUILTINS=yes
     • set DEFAULT_FORWARD_POLICY="ACCEPT"
   • Edit /etc/ufw/sysctl.conf
     • uncomment net/ipv4/ip_forward=1
   • Edit /etc/ufw/before.rules
     • Insert the nat lines (see below [4]) under the lead comments, but above the *filter line
     • Adjusting of course for the relevant configurations
   • ufw allow 8080
   • ufw enable
   • At this point, I cannot reach host1 from host2 with wget, firefox, etc - connection cannot be established
   • If I do nc -l 8080 on host1 and try connecting again from host2, I get the HTTP header dump - indicating that no port forwarding is occurring. FAIL!

I've gone over my steps about short of 5 different times now with fresh VMs, digitalOcean droplets and AWS EC2 instances - no luck, always the same.

Can anybody point out what I am doing wrong ...?

• [1] https://askubuntu.com/questions/370599/forward-port-to-lxc-guest-using-ufw#435286
• [2] https://gist.github.com/ShawnHuang/2726ef5e01a76e2eef55
• [3] https://help.ubuntu.com/lts/serverguide/firewall.html#ip-masquerading

• [4] Nat lines:

  *nat
  :PREROUTING ACCEPT [0:0]
  -A PREROUTING -i enp0s8 -p tcp --dport 8080 -j DNAT --to 10.0.3.101:80
  COMMIT
  
  # comments etc and the rest of the file, like,
  *filter
  ...
  

Turns out there are a few more configuration steps that are needed to allow general traffic to flow:

At the top of /etc/ufw/before.rules, before the *filter section

*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -i enp0s8 -p tcp --dport 8080 -j DNAT --to 10.0.3.101:80
COMMIT

And near the bottom, before the final COMMIT:

-A FORWARD -o lxcbr0 -j ACCEPT
-A FORWARD -i lxcbr0 -j ACCEPT

-A INPUT -p udp --dport 53 -i enp0s8 -m state --state NEW -j ACCEPT
-A INPUT -p tcp --dport 53 -i enp0s8 -m state --state NEW -j ACCEPT
-A INPUT -p udp --dport 67 -i enp0s8 -m state --state NEW -j ACCEPT
-A INPUT -p tcp --dport 67 -i enp0s8 -m state --state NEW -j ACCEPT

At the top of /etc/ufw/after.rules, before the filter section:

*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 10.0.3.0/24 ! -d 10.0.3.0/24 -j MASQUERADE
COMMIT

That did it for my test environment, as outlined in the problem description.