Convert NTFS partition to ext4 - How to copy the data?

Solution 1:

1. FreeFileSync GUI tool (Windows, Mac, Linux)

To ensure you don't miss any data, I highly highly recommend using a good file-copy tool like FreeFileSync. It is graphical, no cost, free and open source, and cross-platform (Windows, Mac, Linux). Just be sure not to screw it up and copy backwards, deleting everything! That's not hard to get right, however, since it's graphical and very user-friendly.

Just set up your folders with source on the left and destination on the right, then click the gear at the top right and choose the "Mirror" option to mirror from left to right, then press the "Compare" button at the top left. Once it determines what it needs to copy, it will pop up an info screen showing what it's going to do (how many files it will create, overwrite, or delete, etc.). Ensure you see nothing next to the trash can (indicating what is to be deleted), or that you see expected files to be deleted, as marked with the trash can and a minus sign, and then press the "Synchronize" button to begin. Voila! Easy to use. Here's a screenshot:

2. rsync Command-line tool (Linux, Windows with Cygwin) (AKA: "How to use rsync"):

If you are doing the copy from Linux, you may also use FreeFileSync of course, but if your'e a command-line guru you might like rsync better. It's a command-line-only tool, and harder to use, but still very powerful, albeit not quite as powerful as FreeFileSync since it doesn't handle two-way syncing very well. Here's a few of my favorite rsync commands:

1. COPY from source to destination, leaving anything in the destination that isn't conflicting still intact:

    sudo rsync -rah --stats --info=progress2 '/path/to/source' '/path/to/destination'
   

   Optionally, add --verbose, or just -v (or -rahv in my case) to also get a full list of exactly what will be copied.

   Optionally, add --dry-run to show what would happen but NOT ACTUALLY DO IT YET.

   Optionally, you may also log the entire list of files to be copied to a stdout.log file like this, and any errors to a stderr.log file like this. Here I remove the progress indicator by removing --info=progress2, just so the log files don't get polluted with progress info everywhere:

    (sudo rsync -rah --verbose --dry-run --stats '/path/to/source' '/path/to/destination' | tee -a '/path/to/stdout.log') 3>&1 1>&2 2>&3 | tee -a '/path/to/stderr.log'
   

   Remove --dry-run from the above when ready to actually make the copy. Be sure to review the stats paragraph printout (which printed because you had --stats) at the end of the stdout.log file before running the script for real, to make sure it all looks good.

   Here's an example of a --stats output obtained from doing the MIRROR option below. Notice it is deleting some files. Pay careful attention to that, as deleting files can't be undone (at least not without special software recovery tools). Just keep in mind it is permanently deleting them, NOT sending them to some sort of recycle bin or something, so watch out!

   Number of files: 102,033 (reg: 89,638, dir: 9,867, link: 2,502, special: 26)
   Number of created files: 3 (reg: 3)
   Number of deleted files: 9,861 (reg: 9,360, dir: 474, link: 27)
   Number of regular files transferred: 25
   Total file size: 8.12G bytes
   Total transferred file size: 35.30M bytes
   Literal data: 35.30M bytes
   Matched data: 0 bytes
   File list size: 917.43K
   File list generation time: 0.007 seconds
   File list transfer time: 0.000 seconds
   Total bytes sent: 38.33M
   Total bytes received: 1.02M

   sent 38.33M bytes received 1.02M bytes 7.15M bytes/sec
   total size is 8.12G speedup is 206.37

2. To MIRROR from source to destination, which means also deleting anything in destination that is not in source (notice the "Number of deleted files" line in the stats output shown just above), add --delete --delete-excluded (even though we aren't excluding anything yet, once you learn about the --exclude=PATTERN syntax you'll also realize you want the --delete-excluded flag).

   First, the safe dry-run version:

    (sudo rsync -rah --verbose --dry-run --stats --delete --delete-excluded '/path/to/source' '/path/to/destination' | tee -a '/path/to/stdout.log') 3>&1 1>&2 2>&3 | tee -a '/path/to/stderr.log'
   

   And now the danger danger danger--could delete everything if you do source and destination backwards version (so check the stats output at the end of stdout.log before continuing):

    (sudo rsync -rah --verbose --stats --delete --delete-excluded '/path/to/source' '/path/to/destination' | tee -a '/path/to/stdout.log') 3>&1 1>&2 2>&3 | tee -a '/path/to/stderr.log'
   

   And now the version that doesn't save to any log files in case you want to just see the live progress and the stats paragraph at the end, but NOT have a log of individual files copied or deleted in stdout.log, or errors printed in stderr.log:

    sudo rsync -rah --info=progress2 --stats --delete --delete-excluded '/path/to/source' '/path/to/destination'
   

3. And this is just the beginning of rsync. Read the manual: man rsync, or online: https://linux.die.net/man/1/rsync, and keep in mind you can mix and match many of the options I've shown above. Other really useful things to look into include setting up a source.txt file, and an exclude.txt file, to include and exclude, respectively, lists of files and paths. Do this with the option --files-from='/path/to/source.txt' and --exclude-from='/path/to/exclude.txt', respectively. This is extremely useful. Lastly, this goes really well with the --relative option. In the end, a full Bash script might look something like this:

     (sudo rsync $OPTIONS $SRC $DEST | tee -a $LOG_STDOUT) 3>&1 1>&2 2>&3 | tee -a $LOG_STDERR
   

   ...where $SRC is your source path (in this case, a relative path for all files in the source.txt file, such as simply / to make everything relative to your root directory), $DEST is your destination path, $LOG_STDOUT is your path to where you'd like to log your stdout info, $LOG_STDERR is your path to where you'd like to log your stderr info, and $OPTIONS might look something like this:

    -rah --verbose --stats --relative --info=progress2 --delete --delete-excluded --dry-run --files-from='/path/to/source.txt' --exclude-from='/path/to/exclude.txt'
   

   And, of course, as stated above, you might want to remove --verbose, --relative, --info=progress2, --delete and --delete-excluded, and --dry-run, as desired, depending on what you're doing.

Related:

1. [MY ANS] https://askubuntu.com/questions/17275/how-to-show-the-transfer-progress-and-speed-when-copying-files-with-cp/1275972#1275972
2. [MY ANS] https://unix.stackexchange.com/questions/65077/is-it-possible-to-see-cp-speed-and-percent-copied/567828#567828
3. [MY ANS] https://askubuntu.com/questions/527304/how-to-change-permission-of-a-drive-in-an-external-hard-disk/1302532#1302532