OpenSSH's server login component (the authorized_keys checking) can't properly respect XDG_CONFIG_HOME because it won't be set at the time it's reading the authorized_keys file. The user's home directory is stored in /etc/passwd but the XDG variables have a million different ways to set them, none of which are truly standardized. Best you could really do is hardcoding .config or the like, which you can do by changing the AuthorizedKeysFile in sshd_config.