Copying raw partition sequentially, in chunks, with error recovery
This is technically possible. The filesystem on the USB stick must support sparse files for my method to work.
To tell ddrescue
to recover a part of a file, use -m
:
-m file
--domain-mapfile=file
Restrict the rescue domain to the blocks marked as finished in the mapfilefile
. […]
(source)
You need to prepare many domain-mapfiles, each describing one rescue domain, one chunk. Altogether they should cover the entire device. Read about the mapfile structure. The 0th domain-mapfile chunk00.map
will be:
# status line is a formality
0 + 1
# the below line describes the chunk
# pos size status
0 104857600 +
This means the chunk starts at offset 0 and takes 104857600 bytes. The latter number is interpreted as decimal. ddrescue
creates mapfiles with hexadecimal values (0x…
) but it understands decimal. A leading zero denotes octal, so don't use leading zeros.
Alternatively you can specify these numbers with -i
and -s
command line options; but because we will need the exact same numbers on PC_A and PC_B, it's better to have them in a file you can copy. If you use the right values on PC_A and copy the file to PC_B then you will use the right values on PC_B. Typing commands with -i
and -s
on both systems independently is more error-prone.
104857600 bytes is 100 mebibytes. I did not use 100000000 (100 megabytes) because we're going to use either -b 512
or -b 4096
(depending on the physical sector size of the source device) and the chunk should take an integer number of sectors. 104857600 is a multiple of 4096 (and therefore a multiple of 512), 100000000 is not.
chunk01.map
will be:
0 + 1
104857600 104857600 +
chunk02.map
:
0 + 1
209715200 104857600 +
and so on. The starting position for the chunk N+1 is the starting position of the chunk N plus the size of the chunk N. In general the chunks may overlap but we don't want them to overlap.
Read the 0th chunk:
# on PC_A
ddrescue -b 512 -c 32768 -m chunk00.map /dev/sdaX /mnt/usb_PC_A/chunk00.raw common.map
You need to do this for each chunk, you need to generate chunk01.raw
from chunk01.map
, chunk02.raw
from chunk02.map
and so on. common.map
should be the same mapfile each time, each invocation of ddrescue
on PC_A should update the common mapfile.
Resulting chunks with non-zero offset (i.e. all chunks but chunk00.raw
) will be sparse.
If the filesystem on the USB stick did not support sparse files then each chunk would be bigger than the previous one due to explicit zeros in the beginning. The last chunk would be as big as /dev/sdaX
, this defies the purpose. A workaround can be to write a chunk to a local filesystem that supports sparse files, pack it (with or without compression) somehow and finally unpack on PC_B as sparse. It's way easier if the filesystem on the USB stick supports sparse files.
After reading each chunk you should transfer the chunk to PC_B. You need to transfer chunk*.raw
and its corresponding chunk*.map
. If you need to copy a sparse chunk to another filesystem, use cp --sparse=always …
. You may not need to copy though; building the final image can be done by reading directly from the USB stick.
Before transferring the 0th chunk, make sure the final image final.raw
on PC_B is empty or nonexistent. Transfer the 0th chunk from the USB:
# on PC_B
ddrescue -b 512 -c 32768 -m /mnt/usb_PC_B/chunk00.map /mnt/usb_PC_B/chunk00.raw final.raw
Do this with all chunks as they come, writing to the same final.raw
; just remember to use the right chunk*.map
each time. Here ddrescue
reads regular files, we don't expect read errors, there's no need for a mapfile.
After transferring all the chunks this way, final.raw
on PC_B should be complete. The file common.map
on PC_A is the corresponding mapfile. It's as if you did
ddrescue -b 512 -c 32768 /dev/sdaX final.raw common.map
on PC_A, except final.raw
is now on PC_B.
Notes:
-
In case of read errors on PC_A, a chunk may be smaller than you expect. See "conclusions" in this answer of mine (where "chunk" means "one or more sectors", "a fragment", it's different than our chunk):
Unreadable chunk [=fragment] at the very end of the input file doesn't contribute to the overall size of the output file.
If our last chunk is smaller then
final.raw
on PC_B will be smaller than/dev/sdaX
on PC_A. It would be exactly like this if you letddrescue
write tofinal.raw
on PC_A in one run. -
In my tests a chunk that starts within
/dev/sdaX
and reaches beyond the end ofsdaX
generates adequately smallerchunk*.raw
file. A chunk that starts beyond the end ofsdaX
makesddrescue
on PC_A complain and fail. This means you don't need to pay attention to the size ofsdaX
, you don't need to define the last chunk accurately. While buildingchunk*.map
files you can increase the starting position by your desired size each time, until somechunk*.raw
is created smaller because of the end ofsdaX
. Then you try toddrescue
the next chunk, getCan't start reading at pos … . Input file is only … bytes long.
from the tool and this way you find out the wholesdX
has been read.common.map
should confirm this.