S3fs performance improvements or alternative?

Solution 1:

S3FS may not be the best choice for a large amount of smaller files. The overhead with S3FS is pretty high as well. I'd suggest using something like S3Curl

You can even get parallel transfers going. Just remember it will never be fast like EBS / local storage.

If you need to have it as a 'mountable' storage, the only alternatives to S3FS that I know of is S3Backer or s3ql

Solution 2:

I just released v0.0.1 of https://github.com/kahing/goofys which was partly motivated by performance problems in s3fs. File creation speedup is 3-6x and time to first byte is 58x. Feedback welcome!