Nginx not showing 404 when attempting a nonexistent folder

Solution 1:

What's missing is the =404 in:

try_files $uri $uri/ /index.php?$args;

Change to:

try_files $uri $uri/ /index.php?$args =404;

That fixed it.