Why is it impossible to compress on the fly images by ddrescue?

Solution 1:

gddrescue dosen't image the file in order - it goes back and retries, and fills in the blanks, and I suspect this is why you can't effectively pipe it into something else. The man page goes into that in detail

GNU ddrescue manages efficiently the status of the rescue in progress and tries to rescue the good parts first, scheduling reads inside bad (or slow) areas for later. This maximizes the amount of data that can be finally recovered from a failing drive.

The standard dd utility can be used to save data from a failing drive, but it reads the data sequentially, which may wear out the drive without rescuing anything if the errors are at the beginning of the drive.

Other programs switch to small size reads when they find errors, but they still read the data sequentially. This is a bad idea because it means spending more time at error areas, damaging the surface, the heads and the drive mechanics, instead of getting out of them as fast as possible. This behavior reduces the chances of rescuing the remaining good data.

Its by design, and the solution, unfortunately, is to get a bigger drive. For the -S argument to work, I believe the used space on the source drive must be smaller than the destination drive.

Solution 2:

It is possible to compress images created with ddrescue on-the-fly. Thing is, ddrescue requires a seekable destination, since it will go over in several passes (and therefore must have the ability to jump backwards to fill in earlier gaps, as @JourneymanGeek explains in his answer). This means you cannot use a pipe as the output, since a pipe is not seekable. Therefore you cannot pipe to compression programs.

One way to get around this is to use transparent compression. Some filesystems (notably, Btrfs, amongst others) provide this built-in. Alternatively, you can use filesystem drivers to provide transparent, seekable compressed storage, e.g. fusecompress.