Changing proxy_cache_valid value based on GET Query

I'm currently trying to setup a nginx cache that should cache a location for a long time but if query parameters are present for only a few minutes.

So basically:

http://example.com/mypath -> long cache
http://example.com/mypath?param=1 -> short cache

My current config looks something like this:

location "~^(/mypath)"
{
        proxy_cache        example.com_my_cache_http;
        proxy_cache_valid 404 15m;
        ...
}

I can't use diffrent locations because locations can't match query parameters, I have tried using an if (I know it's bad practice)

if ($args ~ param) {
            proxy_cache        example.com_my_cache_http;
            proxy_cache_valid 404 15m;
} else {
            proxy_cache        example.com_my_cache_http;
            proxy_cache_valid 404 2d;
}

This results in:

Testing nginx configuration: nginx: [emerg] "proxy_cache" directive is not allowed here

I also tried using variables:

    set $time "1h";

    if ($args ~ param) {
        set $time "2m";
    }

    proxy_cache        example.com_my_cache_http;
    proxy_cache_valid 404 $time;

This results in:

Testing nginx configuration: nginx: [emerg] invalid time value "$time" 

I'm really running out of ideas any help or hint is appreciated.

My first thought would be to use a map.

map $arg_param $cache_valid_404 {
    ""      2d;
    1       15m;
}

Then you set:

proxy_cache_valid 404 $cache_valid_404;