Does CURL cache requests?
This is a pretty long question, so bear with me.
I wanted to stress my Akamai Server logged in from an AWS instance. So, I started running ab benchmark. However, they seemed ridiculously fast to download ~3 MB video files. Naturally I wanted to see what's going on. This is what I did to get the file
curl -v -o /dev/null
The above completed in ~5 seconds.
Next, I ran the same command again. This time, it completed in ~200ms! Naturally, my intuition says the file is being cached somewhere.
My questions:
- Does curl cache files? If so, is there a way to ignore it?
- If curl doesn't, does the ubuntu abstracts a cache beneath curl? If so, is there a way to ignore it?
- Given the requirements, do you think there could be a benchmarking tool apart from ab that could serve the purpose?
Thank you, Akshay
Solution 1:
The curl client isn't caching files, but the remote server network might well be. Try adding an arbitrary query string variable to the URL to see if you can reproduce it.
Solution 2:
Belatedly, try:
curl -v -H "Cache-Control: no-cache"
That will tell the web server to not cache. Doesn't stop layers below caching unless it's coded to obey the headers.
Solution 3:
You can use add a random query string using the $RANDOM environment variable:
curl --location --silent "https://git.io/lsf-e2e?$RANDOM"
This worked for me on github raw files.
Solution 4:
I've used this curl command with a cache buster parameter.
curl http://example.com/static/changing_file?_=$(date +%s)
date +%s prints the seconds since the epoch, if you call the url more than once a second use date +%s.%N to add in nanoseconds.