Are restrictive file permissions important when you're the only user on a VPS?

I'm setting up a VPS to be used as a web server. Primarily I will host a Wordpress blog, and in the official Hardening WordPress guide, they are quite particular about which folders should have which permissions. Several guides about setting up web servers also (naturally) seem to recommend using the most restrictive permissions you can get away with while still preserving functionality.

However, when you're the only user on a VPS, are file permissions really important? If you harden the server in other ways, such as disabling root login over SSH, changing the SSH port, or even disabling password-based SSH login, wouldn't it be fairly safe to use less restrictive file permissions?

wouldn't it be fairly safe to use less restrictive file permissions?

Often online a little bit insecure equates to completely insecure.

Common online building blocks, like Wordpress sites (and many others), are almost under continuous attack and although I couldn't care less about your site getting defaced and any potential dataloss you might suffer, but if your server gets hacked it might be turned into another node (that gets rented out) as part of a botnet attacking other targets, which I do care about.

On the whole file-permissions aren't your only protection nor it is likely that incorrect filesystem permissions are what allows your site to get hacked. Much more likely is that incorrect file system permissions provide an attacker who already hacked your site to get a foothold from which to attack and gain access to your server.

In perspective: on the whole keeping up with patches (both for your OS, your Wordpress deployment and the plugins you use) is usually relatively much more important. SELinux/AppArmor potentially protect you despite any excessive file permissions but if a thing is worth doing, it is worth doing well, so please take the time to set up your server with correct permissions.