How to sync large, disconnected file sets with a small USB flash drive?
I want to keep a large file set (700 GB) synchronized between two computers at two different locations (A and B). The computers are not networked in any way. The file sets will be synchronized initially at one location.
Then I want to be able to work on various files in location A, and transfer the gist of the file system changes (files added, changed, deleted, renamed or moved) to a small, 8 or 16 GB USB flash drive. Then plug the flash drive into the computer at location B, and have those changes applied there prior to commencing additional work. Then repeat the same process at location B back to location A.
Obviously, the system will be limited by the size of the flash drive, which in my case should be very infrequent. The goal is to keep disconnected file sets synchronized without having to carry the entire 700 GB back-and-forth between locations or rely on a network connection. These are Windows computers. Any ideas of how to accomplish this?
This is a slightly odd idea, but it would work. So I'm throwing it out there: Use a version control system and export patches to USB. This would probably depend on your VCS being able to produce reasonably efficient binary patches though.
Also, why use such a small USB stick? It's possible to buy 750 GB and 1 TB external 2.5" hard drives. That would let you sync the entire dataset.
I'm relatively new to all this, but I'll give it a shot.
I have written several batch files that "sync" (mostly copy) different hardware I have. It recognizes the serial numbers for my external hard-drive, my flash drives, and my mini-SD memory card.
With my mini-SD card, since it is in my phone, I copy the files both ways, but since I don't want to constantly fill up the tiny memory card, I wrote in the batch file to only synchronize files after the last date the file was run. This enables me to delete files on either side, but still use the same folders as the sources and destinations.
Basically, the batch file reads a txt file with the last date the batch was run. Then the batch file copies files modified on or after that date, based off of %Last_run_date%, and then calls another batch file to update the %Last_run_date%.
@echo off
set /p Last_run_date=<"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.txt"
@echo.
@echo.
@echo.
@echo Last Update %Last_run_date%
@echo.
@echo Sync card to folder!
xcopy "F:\" "C:\Users\Owner\Desktop\Random\Phone Sync Folder" /s /y /h /r /e /c /i /d:%Last_run_date%
@echo.
@echo.
@echo.
@echo Sync folder to card!
echo Videos
xcopy "C:\Users\Owner\Desktop\Random\Phone Sync Folder\Video" "F:\Video" /s /y /h /r /d:%Last_run_date% /e /c /i
echo Audio
xcopy "C:\Users\Owner\Desktop\Random\Phone Sync Folder\Audio" "F:\Audio" /s /y /h /r /d:%Last_run_date% /e /c /i
echo Pictures
xcopy "C:\Users\Owner\Desktop\Random\Phone Sync Folder\Picture" "F:\Picture" /s /y /h /r /d:%Last_run_date% /e /c /i
@echo.
@echo Update LastRun.txt
"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.bat - Shortcut.lnk"
@echo off
set /p Last_run_date=<"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.txt"
echo.
echo Current Update %Last_run_date%
Here's the lastrun.bat.
@echo off
set DD=%date:~7,2%
set MM=%date:~4,2%
set YY=%date:~10,4%
echo %MM%-%DD%-%YY%>"C:\Users\Owner\Desktop\Stuff I don't use\Shortcuts\Batch Files\lastrun.txt"
Finally, I also have another batch file that synchronizes one of my flash drives that I use for school. It deletes and remakes the print folder directory so that I don't constantly fill up my flash drive
echo Empty Print Folder (F)
rmdir /s /q "F:\Stuff that needs printed"
mkdir "F:\Stuff that needs printed"
echo Sync to F Drive
xcopy "C:\Users\Owner\Desktop\Random\4GB Flashdrive" "F:" /s /y /h /r /d /e /c /i
Also, little side note, if any of this looks familiar, some of the pieces of code I found via Google and then I re-arranged to suit my needs.