Why does vm.swappiness not working?

On the contrary, 5.4 kernel has something off in the VM system.

In 5.4 kernel series, I can have like 4GB of applications running on my 16GB system at home, do something involving like 5-10GB+ of disk I/O, and eventually I have like 2GB+ swapped out... and a nice big swapin pause if I pull down menus or whatever. It does seem setting vm.swappiness=1 versus vm.swapiness=100 versus default has little to no effect on 5.4.x kernels.

In pre-5.4 kernels I ran in Ubuntu 18.04, and also with 5.7.x kernel I ran in Ubuntu 18.04 with mainline (Ubuntu mainline kernel installer), if you set vm.swappiness=100 it was still not quite as aggressive with the swap as 5.4 is. Not that I recommend a value as low as vm.swappiness=1, but in 5.4.x series it seems to still swap pretty heavily, in pre-5.4 and post-5.4 kernels if I ran enough apps it would let applications virtually fill RAM before it swapped, which lead to pretty bad performance due to the tiny amount of RAM left for the disk cache; it did indeed show near-zero swap usage until RAM was pretty full.

Solution is to increase vm.watermark_scale_factor (great article on this topic)

sudo sysctl -w vm.watermark_scale_factor=1000 should have the effect that you're looking for :

When free memory goes below 10% (where 10 := watermark_scale_factor / 100), the kernel will wake kswapd and will thus swap out memory pages to the swap (or discard cache pages, this actually depends on vm.swappiness) until free memory goes up to 20%

Useful reference : https://www.kernel.org/doc/Documentation/sysctl/vm.txt