What does "key_load_public: no such file or directory" mean?

Solution 1:

It means literally what it says: there is no such file or directory that ssh wanted to access.

However, it talks about the file mentioned below, not above. You have just the regular public keys, but you do not have the SSH certificates for them (presumably because you just don't need them). OpenSSH however will always try to load the associated .pub-cert file for each identity key.

The ssh-keygen(1) manual talks about creating an OpenSSH cert authority and signing certificates, should you be interested. (Note: this doesn't use X.509, only OpenSSH's own cert format.)

Usually the certificates are only useful if you have a massive amount of users (and/or servers) but don't want to use Kerberos.