Is it possible to Managing 20 TB data using MySQL?
Solution 1:
Thoughts:
- If you are asking these question on a public forum, hire experts to do it for you
- Consider Postgres and SQL Server which will scale to this volume too
- Do you need ACID? No = consider NoSQL
- Design and hardware matter more than the platform
- Don't virtualise or cut other hardware corners
- What is your RPO/RTO?
- Maintenance window? a.k.a are you really 24/7/365? a.k.k 30k rows per second all the time
- Archiving?
- Do you need older (say 6 months) online?
- Budget?
- Realistic testing required to validate architecture and design for stated load
- 20 TB is probably too small
- 6k per RFID per day but 30k per second? There are 86.4k seconds per day so only 1 in 14 RFIDs write per second: what about potential peak loads of 420k+ rows per second
Finally
- This isn't a database question but an architecture question
- You're asking the wrong questions, too soon for this requirement