How to backup millions of small files?

What is the best way to backup millions of small files in a very small time period? We have less than 5 hours to backup a file system which contains around 60 million files which are mostly small files.

We have tried several solutions such as richcopy, 7z, rsync and all of them seems to have a hard time. We are looking for the most optimal way...

We are open to putting the file in an archive first or transferring the file to another location via network or hard disk transfer

thanks

I'd strongly suggest using a file storage system that allows you to snapshot the volume and backup from the snapshot. This way there's little impact on the actual server and the backup system can take its time doing the backup without concern for the main system. You don't mention an operating system but something like ZFS or a NetApp filer would allow this and both are being used for this exact function all over the place. I'm sure there are other file systems that offer this but I know these work.

Hope this helps.

I worked with a server that stored about ~20 million files where 95% are less than 4k in size and about 50% are deleted every 90 days. They use raw disk image for backup. They also create a index file of names, md5 hash and date created via a script and use that to track the contents.

The original backup solution was to load the files as blobs in a database by the md5 signature. This was phased out since creating millions of md5 hashes took longer than just making a raw image backup.