Could hardlinks be used for backup deduplication purposes?

What you describe is already in use, with rsync and its --link-dest= option, via dozens of wrapper programs, such as dirvish, among others.


Seems like a viable plan. It would decrease the amount of time taken to view and use the backups. If the backups are used frequently and one needs to see complete snapshots, this would be very handy.

I would change the wording "moved from L to C" to simply say "hard linked from L to C".

One consideration - deleting a file with large numbers of links (in reference to your last bullet point) means locating all of those links and removing them. So, recovering space selectively in that manner would be more challenging, but easy enough to do with the find command.


I think that's basically what you call a Delorean-copy. There is, for example, the Link Shell Extension for Windows, that implements this behaviour. They have a quite good explanation in their documentation:

http://schinagl.priv.at/nt/hardlinkshellext/linkshellextension.html#deloreancopy


HardLinkShellExtension with its "Delorean-Copy" (see other answer) isn't the only "ready to use" solution. There are alternatives:

  • the console tool ln.exe from the same programmer with the same functionality. The author offers a pre-written batch-file for timestamped DeLorean copies too.
  • the GUI Backup solution HardLinkBackup which does pretty much exactly what you want.
  • use ln.exe from 1. to make a hardlink copy of the old backup into the new backup folder and then use xcopy or robocopy to only copy new files and remove old ones (i thinks it's --mirror for robocopy). Test it to make sure changed files are deleted and then copied and not just modified (the latter would change the file in older backups too because of the hardlinks).
  • using xcopy or robocopy to make normal backups and then run dfhl.exe /l /r /w /s /h "X:\Backups-parent-folder\." to hardlink all identical files.
  • same as 3. but finddupe -hardlink X:\Backups-parent-folder\** instead of dfhl.


Disclaimer: I've used all of the above mentioned programs except for finddupe, but not necessarily in the same way. And I've no monetary connection or investment or any other connection to any of the programs.