Denying folder in Nginx causing PHP-FPM to not take precedence

The title says it all, but I have a very interesting problem that I am quite confused on. Basically, I have a simple WordPress installation, and I wish to deny a folder (along with all files in the folder), let's say /wp-content/themes/default/scripts but to allow certain IPs.

I can deny this folder with location ^~/wp-content/themes/default/scripts {deny all;} like any Nginx guru will tell you.

But from my understanding, the "^" takes higher precedence and will stop searching for other location blocks once it finds the regex match. As I do not want to deny the folder for everyone (of course, using allow (IP Address);, my ^~/wp-content/... location block entirely whipes out my PHP-FPM location block to pass the file over to the FastCGI server. And of course, when I try to view any file in the folder, the PHP file is directly downloaded as PHP is not parsing it.

Does anyone have an idea? I want to block the folder, but for PHP to work at the same time for the users I decide to allow. It is quite the tricky question; I cannot find any answer to the problem.

Thanks guys! Really appreciate your help!

For anyone wondering, my current Nginx vhost config looks like the following:

server {
    #..all of the common stuff you would expect

    include /folder/nginx.conf; #including some WordPress rewrites for W3 Total Cache

    # pass the PHP scripts to FastCGI server listening on UNIX socket
    location ~ \.php$ {
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include /etc/nginx/fastcgi_params;
        fastcgi_read_timeout 1900;
    }

    location / {
        #allowing some IPs...

        #deny everyone else
        deny all;

        #WordPress rewrite for slugs
        index index.php;
        if (!-e $request_filename) {
            rewrite ^(.*)$  /index.php last;
        }

        #rewrite for sitemaps
        rewrite ^(.*/)?sitemap.xml /wp-content/sitemap.php last;

    }

    # denying access to .htaccess files
    location ~ /\.ht {
            deny  all;
    }
}

Finally found the answer. When you're doing something like this, you need to redeclare the PHP-FPM settings (all the stuff that was in the location ~ \.php$ { (this code) } block.

So to avoid redundancy, I put those values in another file, and was left with something like this:

server {

        # pass the PHP scripts to FastCGI server listening on UNIX socket
        location ~ \.php$ {
                include /etc/nginx/fastcgi_php_text;
        }

        location / {
                index index.php;
        }

        location ^~/folder/ {
                deny all;
                allow ...;

                include /etc/nginx/fastcgi_php_text;
        }
}

No idea if this is the best way to do such, but this is the only way that I have figured out.

You could also try defining a location block by @name and referencing it. From nginx Pitfalls page

server {

        location ~ \.php$ {
            try_files      @fascgi;
        }

        location ^~ /folder {
            deny           all;
            allow          127.0.0.1;
            location ~ \.php$ {
                try_files  @fascgi;
            }
        }

        location @fascgi {
            fastcgi_pass   127.0.0.1:9000;
            include        fastcgi_params;
        }

}