How to detect in Bash when Curl fails downloading non-existent file?

I'm using curl to download a zip file. curl is called from a bash script:

curl --user joedoe http://1.2.3.4/files/joedoe.zip -o /tmp/downloaded.zip

I'd like to detect when the target zip file does not exist.

I tried

if ! [[ $? == 0 ]] ; then
  echo "ERROR: problem with Curl."
  exit 1
fi ;

but it seems that $? returned by curl is 0, so I am unable to detect when the zip files does not exist.

How can it be done?

Solution 1:

This is almost a duplicate of Can I make cURL fail with an exitCode different than 0 if the HTTP status code is not 200? The most upvoted answer (i.e. most upvoted when I'm writing this) advises -f. This solution will catch 4xx or 5xx HTML error codes, but not 301 Moved Permanently. In your case the HTML error code may or may not be 301.

If I were you, I would capture and analyze this code. Fortunately your curl uses -o and writes to a regular file, not to stdout; so we can use stdout for our extra task. -w is what you need:

http_code="$(curl --user joedoe -w '%{http_code}\n' -o /tmp/downloaded.zip http://1.2.3.4/files/joedoe.zip)"
exit_status="$?"

Now you have $exit_status and $http_code to analyze. Decide if you want -f and build the logic of your script accordingly. Basically you want $http_code to be 200. Even if it's 200, check anyway if $exit_status is 0. Imagine the link is good, but you cannot write to /tmp/downloaded.zip (maybe somebody put a rogue file there in the hope you will use it). This is an example case when you can get 200 from the server and still an error (non-zero $exit_status) from curl.

Even if $exit_status is 0 and $http_code is 200, you may still want to check if /tmp/downloaded.zip is a valid zip file. A basic test may be the exit status of:

[ "$(file -b --mime-type /tmp/downloaded.zip)" = application/zip ]

(if your file supports these options). A "full" test is by checking the exit status of:

unzip -t /tmp/downloaded.zip