There's nothing quite like discovering at 11PM that one of your servers is compromised. Great way to get the blood pumping
This is not the kind of process you want to see spiking CPU usage in htop.
See if you can somehow pinpoint when the attacker got in

It actually didn't take long to figure out how they got in - the culprit of course was me. This is a NixOS box and I had set the users `AuthorizedKeys` but had naively assumed that doing this would disable OpenSSH `PasswordAuthentication`. It does not.

In the logs I can see them logging in with the simple password I used for this user (because I have SSH keys setup, I'm safe right? I can use a dumb password...) and then overwriting my authorized keys file.

I honestly probably would have overlooked this in my investigation if they had just updated the key and not the comment but uhh this caught my eye

Side note: this is the public keys of the attacker. Is there any fun OSINT I can to see if they've been elsewhere or to get a hint of who they are?

@dan github and gitlab used to publicly expose the ssh keys of their users, for github i believe it was github.com/username.key

There were databases built by scraping all github users that allowed a reverse lookup to find the guthub account associated to a certain key.

Now this feature is behind an authenticated api endpoint, but I woudln't be surprised if some updated database is still around

@al if you come across it again, I’d be curious to check it out