that makes no sense - you need the key

But if it’s stored in a keyring or similar (like on windows) and the client reads from it you don’t need the file with the plain text key. Like you don’t store the git credentials in a file, but with libsecret.

I would prefer something that never ask for the password.

Things like the gnome-keyring or kwallet keep all the passwords in an encrypted file, they get decrypted and kept in ram using your login password when you log into gnome/KDE session and programs can ask for passwords using some API. Once you log out the passwords are removed from ram and no one can read them. My goal is to have something like this, so I’m never asked for a password, I just log into my session and everything is available