How well does authorized_keys scale?
You can actually see the efficiency on GitHub as to how fast this is. You are not going to cause a significant bottleneck with that many keys.
Though as documented in their blog from 2009, they have changed how ssh keys are retrieved, from a database. Hat Tip: @Jeremey
But, you created over 8k keys, you can test again with 50k keys.
Those keys don't need to be valid keys, just write a generator and write the file and then append yours to the end.