Is there a practical limit to the number of files in a Swift container?
(Copied over from StackOverflow as I was told it is more appropriate here)
A solution I work on stores files in OpenStack's Swift but the files are spread out into multiple containers. I have been told this is because the performance is awful after about a million files in a container. The only reference I could find is from a blog of 2010 http://adrianotto.com/2010/09/openstack-os-is-great-for/.
Since
- I cannot find any other documentation about such a limitation/performance issue
- It has been more than 10 years since the blog post
Is there still a limitation (or performance issue) to the number of objects in object storage in recent versions of OpenStack?
Also copying over my answer from Stackoverflow.
The page that documents Swift's relatively new container sharding feature makes that point as well. The fact that developers implemented this feature indicates that this is a real problem.
Container sharding could be a better method for your use case than storing objects in several containers, but it is still marked as experimental.