sudo apt-get remove does not remove config files
Solution 1:
Use sudo apt-get purge <program> instead, to remove config files as well.
Solution 2:
You're looking for purge:
sudo apt-get purge vsftpd
will also delete the config files. You can do this even after you've already removed the package.