How to view EXTREMELY large images?

I have a 1.8GB TIF image that I'm trying to open on my machine with Ubuntu 18.04.1 LTS. I have 8GB of RAM and 2GB of swap space. However, I can't open the image on the stock Gnome Image Viewer (says "Dimensions of TIFF image too large"), or Shotwell (says "Photo source file missing: [path to the file]"), or Gimp (bugs out), or on my Windows 7 VM with 4GB of RAM (says too little RAM even though it had over 2.5GB free), or feh, which behaves like this:

$ feh DO_NOT_OPEN.tif
TIFFFetchNormalTag: Warning, Incompatible type for "RichTIFFIPTC"; tag ignored.
feh WARNING: DO_NOT_OPEN.tif - No Imlib2 loader for that file format
feh: No loadable images specified.
See 'man feh' for detailed usage information

Is there any way to open this image on Ubuntu?

(For the curious, the image is this one. I named it like that because the first time i tried to open it crashed my computer ¯_(ツ)_/¯ )

Solution 1:

Why can't I open the image in any viewer?

The issue is probably RAM and/or CPU.

The TIFF image is LZW-compressed. When an application wants to display it, it needs to uncompress the image first. And that's where RAM and CPU come into play. Furthermore, depending on the application, there may be buffers that also need RAM.

I did some (non-scientific) experiments and noticed differences between applications in three phases: when loading the image, after loading the image, when zooming in.

Shotwell took up to 5GB of RAM while opening the file and then dropped to 2GB. When I zoomed in, it went up to 7GB.

Gimp went up to 5GB and stayed there when viewing the image and zooming in. The RAM usage gradually increased when editing the image.

GwenView went up to 3.8GB while loading and dropped to 2GB afterwards. When zooming in, it went up to 2.7GB.

Your best bet is trying GwenView (the default image viewer in Kubuntu, but can be installed in stock Ubuntu as well).

What options do I have if the image is way too big for RAM?

In general, if the image is too big to fit into RAM, you can split it using convert which is part of the imagemagick package.

You need to increase the default limits which are defined in /etc/ImageMagick-6/policy.xml (the number 6 may change in future versions). You need to adjust the maximum size in pixels, which are defined in the following lines:

<policy domain="resource" name="width" value="16KP"/>
<policy domain="resource" name="height" value="16KP"/>

KP means KiloPixels, i.e. thousand pixels. In your case, the image has 40000x12788 pixels, so you only need to change the width to something greater than 40KP, e.g. 41KP.

Additionally, you need to increase the maximum disk space:

<policy domain="resource" name="disk" value="1GiB"/>

to something reasonable like 10GiB.

To split the image into 5x3 parts and produce one file per part, use the following command:

convert -crop 5x3@ inputfile.tif outputfile%0d.tif

For more information about convert, see man convert or take a look at the documentation.

Maybe a bit off topic, but here's a quick&dirty bash script that tiles an image and creates an html page that displays an overview where you can click on a tile to open it in full resolution. Save it as e.g. tile_image.sh and invoke it like ./tile_image.sh originalImage tilesX tilesY, where tilesX is the number of tiles horizontally and tilesY is the number of tiles vertically.

#!/bin/bash
INPUT_FILE=$1
TILES_X=$2
TILES_Y=$3
OUTPUT_FILE=${INPUT_FILE%.*}_tile_%0d.png
convert -crop $TILES_X"x"$TILES_Y\@ $INPUT_FILE $OUTPUT_FILE
HTML_FILE=${INPUT_FILE%.*}_tile_view.html
echo "<html><body><style type="text/css">table,tr,td,a {padding:0;border-spacing:0} img:hover {opacity:.9}</style><table>" > $HTML_FILE
X=0
Y=0
while [ $Y -lt $TILES_Y ]; do
    echo "<tr>" >> $HTML_FILE
    while [ $X -lt $TILES_X ]; do
        TILE_NUMBER=$(echo $Y*$TILES_X+$X | bc -l)
        TILE_NAME=${INPUT_FILE%.*}_tile_$TILE_NUMBER.png
        THUMBNAIL=${INPUT_FILE%.*}_tile_$TILE_NUMBER"_thumb.png"
        convert -resize 100x100 $TILE_NAME $THUMBNAIL
        echo "<td><a href=\""$(basename $TILE_NAME)"\"><img src=\""$(basename $THUMBNAIL)"\"></a></td>" >> $HTML_FILE
        let X=X+1
    done
    let X=0
    echo "</tr>" >> $HTML_FILE
    let Y=Y+1
done
echo "</table></body></html>" >> $HTML_FILE

Solution 2:

Test

I tested to view the 1.7 GiB Tiff file in a Dell Precision M4800 with 16 GiB RAM and Ubuntu 18.04.1 LTS with Wayland. I tested with gimp also in a Dell Latitude E7240 with 8 GiB RAM and 18.04.1 LTS persistent live with Xorg.

The total RAM usage including operating system is measured with free -m.

gimp - works best for me also at full resolution (without delay), 6143 MiB RAM. It works well both with 16 GiB RAM and 8 GiB RAM.
shotwell - works also at full resolution (without delay), 7920 MiB RAM (after some horizontal and vertical scrolling at full resolution)
gpicview - works but slower (failed to show full resolution directly, but works when gradually zooming in to full resolution, total RAM usage when showing and zooming 3016 MiB RAM, but 4219 MiB RAM during loading.
feh - does not work 'No ImLib2 loader for that file format'
eog - does not work 'too big image'
ristretto - does not work (silently, shows a thumbnail, but no picture)

Conclusion

It should work with Ubuntu and 8 GiB RAM to view this picture with several viewers, and the editor gimp works really well for me. With 6 GiB RAM, you should select a light-weight viewer, and with 4 GiB probably be prepared to wait for some swapping.

Please notice that I have measured the total usage of RAM (including the operating system).

Final comment

This is a great picture well worth zooming into with the tools, that you have available. I should add that the zoom tool of the web site works well with Firefox for me in a computer with 4 GB RAM, I could reach a zoom level corresponding to full resolution of the 1.7 GB Tiff file and I could scroll horizontally and vertically to see the whole picture.