Solution 1:

nginx is a metapackage, so you need to remove whatever was installed by this package. If doing sudo apt-get autoremove doesn't do the trick, you can run sudo apt-get remove nginx-core nginx-full nginx-light nginx-extras nginx-naxsi nginx-common (you probably have only one of these packages installed, but the command shouldn't fail).

Solution 2:

sudo apt-get remove --purge nginx* will remove whatever is installed and is related to nginx, including configuration files. It will also list packages available in repositories which match the regex nginx* and are not installed.

There is a difference between apt-get remove and apt-get purge. The --purge option removes also the configuration files. It is useful for a clean reinstall.

Typing the command:

sudo apt-get remove --purge nginx*

will result in:

The following packages will be REMOVED:
nginx-common*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 0 B of additional disk space will be used.
Do you want to continue [Y/n]? 
(Reading database ... 55416 files and directories currently installed.)
Removing nginx-common ...
Purging configuration files for nginx-common ...