What databases do the World Wide Web's biggest sites run on? [closed]
This question is meant to serve as a list of databases and their configurations that the major web sites use and would be a great reference for anyone thinking of scaling their web site to the size of Twitter, Facebook or even Google.
Please keep your answers to a minimum and be sure to cite any sources used.
EDIT:
Also, please bold both the web-site name and the database for easier scanning.
Solution 1:
Facebook.com
- MySQL with MyRocks. Used to store user info and social activities such as likes, comments, and shares.
- Hive (Data warehouse for Hadoop, supports tables and a variant of SQL called hiveQL). Used for "simple summarization jobs, business intelligence and machine learning and many other applications"
- Cassandra (Multi-dimensional, distributed key-value store). Currently used for Facebook's private messaging.
Currently running 610 (soon to be 1000) Hadoop nodes in a single cluster with Hive datastore. Both Hive and Cassandra have been open-sourced by Facebook.
Facebook stats:
- More than 200 million active users
- More than 100 million users log on to Facebook at least once each day
- More than 30 million users update their statuses at least once each day
- Average user has 120 friends on the site
Sources:
- http://www.dbms2.com/2009/05/11/facebook-hadoop-and-hive/
- http://www.facebook.com/note.php?note_id=89508453919
- http://www.facebook.com/press/info.php?statistics
- http://hadoop.apache.org/hive/
- http://wiki.apache.org/hadoop/Hive/Design
- http://www.facebook.com/note.php?note_id=24413138919
- https://code.facebook.com/posts/190251048047090/myrocks-a-space-and-write-optimized-mysql-database
Solution 2:
Stack Overflow - SQL Server.
Jeff Atwood wrote a nice blog post on this
https://blog.stackoverflow.com/2008/09/what-was-stack-overflow-built-with/
Solution 3:
LinkedIn.com
- Oracle (Relational Database)
- MySQL (Relational Database)
Databases replicated on multiple servers for high availability. Each specific Service uses its own domain-specific DB.
LinkedIn stats:
- 22 million members
- 4+ million unique visitors/month
- 40 million page views/day
- 2 million searches/day
Sources:
- http://hurvitz.org/blog/2008/06/linkedin-architecture/