Lemmy.world (and some others) were hacked

https://lemmy.world/post/1290412

Lemmy.world (and some others) were hacked - LemmyWorld

While I was asleep, apparently the site was hacked. Luckily, (big) part of the lemmy.world team is in US, and some early birds in EU also helped mitigate this. As I am told, this was the issue: - There is an vulnerability which was exploited - Several people had their JWT cookies leaked, including at least one admin - Attackers started changing site settings and posting fake announcements etc Our mitigations: - We removed the vulnerability - Deleted all comments and private messages that contained the exploit - Rotated JWT secret which invalidated all existing cookies The vulnerability will be fixed by the Lemmy devs. Details of the vulnerability are here [https://lemmy.world/post/1293336] Many thanks for all that helped, and sorry for any inconvenience caused! Update While we believe the admins accounts were what they were after, it could be that other users accounts were compromised. Your cookie could have been ‘stolen’ and the hacker could have had access to your account, creating posts and comments under your name, and accessing/changing your settings (which shows your e-mail). For this, you would have had to be using lemmy.world at that time, and load a page that had the vulnerability in it.

So what happened:

  • Someone posted a post.
  • The post contained some instruction to display custom emoji.
  • So far so good.
  • There is a bug in JavaScript (TypeScript) that runs on client’s machine (arbitrary code execution?).
  • The attacker leveraged the bug to grab victim’s JWT (cookie) when the victim visited the page with that post.
  • The attacker used the grabbed JWTs to log-in as victim (some of them were admins) and do bad stuff on the server.

Am I right?

I’m old-school developer/programmer and it seems that web is peace of sheet. Basic security stuff violated:

  • User provided content (post using custom emojis) caused havoc when processing (doesn’t matter if on server or on client). This is lack of sanitization of user-provided-data.
  • JavaScript (TypeScript) has access to cookies (and thus JWT). This should be handled by web browser, not JS. In case of log-in, in HTTPS POST request and in case of response of successful log-in, in HTTPS POST response. Then, in case of requesting web page, again, it should be handled in HTTPS GET request. This is lack of using least permissions as possible, JS should not have access to cookies.
  • How the attacker got those JWTs? JavaScript sent them to him? Web browser sent them to him when requesting resources form his server? This is lack of site isolation, one web page should not have access to other domains, requesting data form them or sending data to them.
  • The attacker logged-in as admin and caused havoc. Again, this should not be possible, admins should have normal level of access to the site, exactly the same as normal users do. Then, if they want to administer something, they should log-in using separate username + password into separate log-in form and display completely different web page, not allowing them to do the actions normal users can do. You know, separate UI/applications for users and for admins.

Am I right? Correct me if I’m wrong.

Again, web is peace of sheet. This would never happen in desktop/server application. Any of the bullet points above would prevent this from happening. Even if the previous bullet point failed to do its job. Am I too naïve? Maybe.

Marek.

Oh I forgot another line of defense / basic security mitigation. If a server produces an access token (such as JWT or any other old school cookie / session ID), pair it with an IP address. So in case of cookie theft, the attacker cannot use this cookie from his computer (IP address). If the IP changes (mobile / WiFi / ADSL / whatever), the legitimate user should log-in again, now storing two auth cookies. In case of another IP change, no problemo, one of the stored cookies will work. Of course limit validity of the cookie in time (lets, say, keep it valid only for a day or for a week or so).

pair it with an IP address.

I’m not totally sure how Lemmy uses these JWTs, but depending on the use case, pairing it with an IP address may not work.

In particular, mobile devices change IP addresses all the time, constantly switching between multiple WiFi and cell networks.

Developers should combine the token with a session ID generated by the server and stored in the cookie, and may want to also combine the token with some kind of device ID/fingerprint generated on the client. Both options are still spoofable, but maybe a bit more difficult to exfiltrate.

mobile devices change IP addresses all the time

I never noticed this. Yes, switch between mobile and WiFi, but this is only two addresses. In case of IPv4 this seems not problem. In case of IPv6, use /64 or /48 (or whatever is now recommended for residential end users) prefix instead of the entire 128bits. I’m not proposing to log-out the suer after IP change, I’m proposing multiple sessions to be accessible at the same time.

Mobile will often switch ip’s on tower handoffs. If you’re driving down the road or on a train, it’s nothing to change mobile ip addresses every 2 minutes.
Not in my experience. But OK, if this is the case, don’t use exact IPv4 address, lookup the routing database and use the sub-net. Or whatever. This is belt & suspenders style of defense in depth, just another layer of security if all others fail. Not core functionality.

I work for a mobile game company. Millions of clients. We deal with this a lot. You can’t even predict that they’ll stay in the same class A. I wouldn’t be surprised if they worked out a way to hand off ipv4 to 6 and vice-versa.

Then you have ISP’s and large work networks who send everyone out under the same NAT/PAT, 10’s of thousands of users all coming from one address.

IMO Providing a public service then trying to identify individuals by network without screwing someone over is a fools errand.

If you’re dipping logs and see one jackass doing something on x IP, you always have to go back and see how many ip’s that jackass is coming from and also how much viable traffic is coming from that ip.