Set nginx.conf to deny all connections except to certain files or directories

The easiest path would be to start out by denying all access, then only granting access to those directories you want. As ring0 pointed out, you can use the default (default_server in 0.8) flag on the listen directive. However, if you already have a server you want to use as a default for unknown named access to your host, you can also just catch requests without a host header or with your server's ip address with something like this (replacing 1.2.3.4 with your server's ip:

upstream _php {
  server unix:/var/run/php-fpm/php-fpm.sock;
}

server {
  server_name "" 1.2.3.4;

  root /path/to/root;
  index index.php;

  include fastcgi_params;
  fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

  # deny everything that doesn't match another location
  location / { deny all; }

  # allow loading /index.php
  location = / { } # need to allow GET / to internally redirect to /index.php
  location = /index.php { fastcgi_pass _php; }

  # allow access to phpmyadmin
  location /phpmyadmin/ { } # Allow access to static files in /phpmyadmin/
  location ~ ^/phpmyadmin/.*\.php$ { fastcgi_pass _php; } # phpmyadmin php files
}

the fastcgi_params will be inherited by both locations that fastcgi_pass, and only /index.php and /phpmyadmin/ are allowed. I've also added an upstream block for php, which makes it easier should you ever need to add to or change it in the future.

Creative use of the location and deny rules negated in regex, something like:

    location / {
        root   html;
        index  index.html index.htm index.php;
    }

    location ~* !^/(index.(php|html|htm)$)|(phpmyadmin/) {
        deny all;
    }

    location ~ \.php$ {
        root           html;
        fastcgi_pass   unix:/var/run/php-fpm/php-fpm.sock;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME /srv/http/nginx/$fastcgi_script_name;
        include        fastcgi_params;
    }

That's untested but you get the idea.

http://wiki.nginx.org/HttpAccessModule

http://wiki.nginx.org/HttpCoreModule#location

Also, this may help you write it:

http://www.regextester.com/

It is easier to declare servers that are called with a domain name, and declare a default server that will include direct-IP accesses.

For instance your domain configuration could be used for, say, mydomain.com

server {
        listen       80;
        server_name  mydomain.com *.mydomain.com;
        root /var/www/html/mydomain.com
        ...

Having your regular files in the directory /var/www/html/mydomain.com.

And the default entry would only allow access to some specific files in a different location.
Note the default keyword after the listen 80.

server {
        listen       80 default;
        server_name  _ *;
        root /var/www/html/restricted
        ...

And you put the default files in /var/www/html/restricted, that will be served for localhost and raw IP addresses.