lxc bandwidth control using tc

The problem here is not well documented but I've experienced it before. On 64 bit systems, the value you echo is not represented as a 16bit integer but a 32bit integer.

Try replacing:

echo 0x1001 > A/net_cls.classid   # 10:1
echo 0x1002 > B/net_cls.classid   # 10:2

With

echo 0x00100001 > A/net_cls.classid   # 10:1
echo 0x00100002 > B/net_cls.classid   # 10:2

This should fix your problem.

Note: Its not actually necessary to provide the leading zeroes at the start but for clarity I added them.

Many of the non-hypervisor containers (lxc, jails) tend to have dodgy/incomplete metering (disk iops, net) that impacts other containers. If limiting external transit is the main concern, stick a transparent firewall (i.e., ArmorLogic, Barracuda, etc.) in front of these first.

In any case, definitely load test to see if it makes a difference.

(ProTip: Heroku runs LXC. heroku run bash)