So this is stupidly absurd to achieve right. Here are the steps I go through to get a key-based auth into the default WSL2 shell.
https://www.concurrency.com/blog/may-2019/key-based-authentication-for-openssh-on-windows
https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_server_configuration