As of 2020-05-09 Filippo Valsorda has released yubikey-agent. I am now recommending this method over using PKCS#11, however if you still wish to use the native ssh-agent, read on.
All other guides I've seen (https://github.com/drduh/YubiKey-Guide being the most prolific) tell you to use the Yubikey's smartcard (PKCS#11) features with GnuPG via gpg-agent.
STOP THE MADNESS!
OpenSSH has supported OpenSC since version 5.4. This means that all you need to do is install the OpenSC library and tell SSH to use that library as your identity.