Why does the built-in ssh-keygen on M1 Mac run significantly slower

I had the same issue with macOS Big Sur on MacBook Air M1.

Though I haven't investigated much, I confirmed that the Apple tree of OpenSSH also runs fast if built by myself, and that the issue now seems to be fixed in macOS Monterey.