Should you edit/delete the default apache site config?
Solution 1:
Just disable them if you're not using them
a2dissite default
a2dissite default-ssl
apachectl graceful
This simply removes the link to sites-enabled, so they're not included in the config anymore
Solution 2:
You can disable the default files which is equivalent to removing the link from sites-enabled directory.
a2dissite default
If you don't want to disable default site but want to hide the directories then you can put up a home page (index.html) in /var/www/ or hide the directory indexes.