How to use curl to compare the size of the page with deflate enabled and without using it

Solution 1:

I think the only reliable way to get the size, is to actually download the file. However, curl offers a very convenient option for only outputting data of interest

-w/--write-out <format>
    Defines what to display on stdout after a completed and successful operation.

[...]

size_download  The total amount of bytes that were downloaded.

which means you can do something like this:

curl -so /dev/null http://www.whatsmyip.org/http-compression-test/ -w '%{size_download}'

Output:

8437

And to get the compressed size:

curl --compressed -so /dev/null http://www.whatsmyip.org/http-compression-test/ -w '%{size_download}'

Output:

3225

After that your comparison should be trivial.

Solution 2:

Copy/paste ready and human readable

Based on @flesk answer and on this here is a human readable version of the script:

#!/usr/bin/env bash
set -e

bytesToHuman() {
    b=${1:-0}; d=''; s=0; S=(Bytes {K,M,G,T,E,P,Y,Z}iB)
    while ((b > 1024)); do
        d="$(printf ".%02d" $((b % 1024 * 100 / 1024)))"
        b=$((b / 1024))
        let s++
    done
    echo "$b$d ${S[$s]}"
}

compare() {
  echo "URI: ${1}"

  SIZE=$(curl -so /dev/null "${1}" -w '%{size_download}')
  SIZE_HUMAN=$(bytesToHuman "$SIZE")
  echo "Uncompressed size : $SIZE_HUMAN"

  SIZE=$(curl --compressed -so /dev/null "${1}" -w '%{size_download}')
  SIZE_HUMAN=$(bytesToHuman "$SIZE")
  echo "Compressed size   : $SIZE_HUMAN"
}

compare https://stackoverflow.com/q/9190190/1480391

Output:

URI: https://stackoverflow.com/q/9190190/1480391
Uncompressed size : 106.69 KiB
Compressed size   : 24.47 KiB