Is it possible to Managing 20 TB data using MySQL?

Solution 1:

Thoughts:

  • If you are asking these question on a public forum, hire experts to do it for you
  • Consider Postgres and SQL Server which will scale to this volume too
  • Do you need ACID? No = consider NoSQL
  • Design and hardware matter more than the platform
  • Don't virtualise or cut other hardware corners
  • What is your RPO/RTO?
  • Maintenance window? a.k.a are you really 24/7/365? a.k.k 30k rows per second all the time
  • Archiving?
  • Do you need older (say 6 months) online?
  • Budget?
  • Realistic testing required to validate architecture and design for stated load
  • 20 TB is probably too small
  • 6k per RFID per day but 30k per second? There are 86.4k seconds per day so only 1 in 14 RFIDs write per second: what about potential peak loads of 420k+ rows per second

Finally

  • This isn't a database question but an architecture question
  • You're asking the wrong questions, too soon for this requirement