LiteLLM Python package compromised by supply-chain attack
[LITELLM TEAM] - For updates from the team, please see: #24518 [Security]: CRITICAL: Malicious litellm_init.pth in litellm 1.82.8 PyPI package — credential stealer Summary The litellm==1.82.8 wheel...
Besides main issue here, and the owners account being possibly compromised as well, there's like 170+ low quality spam comments in there.
I would expect better spam detection system from GitHub. This is hardly acceptable.
They repeat only six sentences during 100+ comments:
Worked like a charm, much appreciated.
This was the answer I was looking for.
Thanks, that helped!
Thanks for the tip!
Great explanation, thanks for sharing.
This was the answer I was looking for.
Well, large companies/corporations don't care about Spam because they actually benefit from spam in a way as it boosts their engagement ratio
It just doesn't have to be spammed enough that advertisers leave the platform and I think that they sort of succeed in doing so.
Think about it, if Facebook shows you AI slop ragebait or any rage-inducing comment from multiple bots designed to farm attention/for malicious purposes in general, and you fall for it and show engagement to it on which it can show you ads, do you think it has incentive to take a stance against such form of spam
I just installed Harbor, and it instantly pegged my cpu.. i was lucky to see my processes before the system hard locked.
Basically it forkbombed `grep -r rpcuser\rpcpassword` processes trying to find cryptowallets or something. I saw that they spawned from harness, and killed it.
Got lucky, no backdoor installed here from what i could make out of the binary
This is tied to the TeamPCP activity over the last few weeks. I've been responding, and keeping an up to date timeline. I hope it might help folks catch up and contextualize this incident:
Would value your opinion on my project to isolate creds from the container:
https://github.com/calebfaruki/tightbeam
https://github.com/calebfaruki/airlock
This is literally the thing I'm trying to protect against.
So... I'm working on an open source technology to make a literal virtual machine shippable i.e. freezing everything inside it, isolated due to vm/hypervisor for sandboxing, with support for containers too since it's a real linux vm.
The problems you mentioned resonated a lot with me and why I'm building it, any interest in working to solve that together?: https://github.com/smol-machines/smolvm
Run locally on macs, much easier to install/use, and designed to be "portable" meaning you can package a VM to preserve statefulness and run it somewhere else.
worked in AWS and specifically with firecracker in the container space for 4 years - we had a very long onboarding doc to dev on firecracker for containers... So I made sure to focus on ease of use here.
Probably on the side of your project, but did you try SmolBSD? <https://smolbsd.org>
It's a meta-OS for microVMs that boots in 10–15 ms.
It can be dedicated to a single service (or a full OS), runs a real BSD kernel, and provides strong isolation.
Overall, it fits into the "VM is the new container" vision.
Disclaimer: I'm following iMil through his twitch streams (the developer of smolBSD and a contributor to NetBSD) and I truly love what he his doing. I haven't actually used smolBSD in production myself since I don't have a need for it (but I participated in his live streams by installing and running his previews), and my answer might be somewhat off-topic.
More here <https://hn.algolia.com/?q=smolbsd>
Thanks for the pointer! Love the premise project. Just a few notes:
- a security focused project should NOT default to train people installing by piping to bash. If i try previewing the install script in the browser it forces download instead of showing as plain text. The first thing i see is an argument
# --prefix DIR Install to DIR (default: ~/.smolvm)
that later in the script is rm -rf deleting a lib folder. So if i accidentally pick a folder with ANY lib folder this will be deleted.
- Im not sure what the comparison to colima with krunkit machines is except you don't use vm images but how this works or how it is better is not 100% clear
- Just a minor thing but people don't have much attention and i just saw aws and fly.io in the description and nearly closed the project. it needs to be simpler to see this is a local sandbox with libkrun NOT a wrapper for a remote sandbox like so many of the projects out there.
Will try reaching you on some channel, would love to collaborate especially on devX, i would be very interested in something more reliable and bit more lightweight in placce of colima when libkrun can fully replace vz
Love this feedback, agree with you completely on all of it - I'll be making those changes.
1. In comparison with colima with krunkit, I ship smolvm with custom built kernel + rootfs, with a focus on the virtual machine as opposed to running containers (though I enable running containers inside it).
The customizations are also opensource here: https://github.com/smol-machines/libkrunfw
2. Good call on that description!
I've reached out to you on linkedin
just sandbox the interpreter (in this case), package manager and binaries.
u can run in chroot jail and it wouldnt have accessed ssh keys outside of the jail...
theres many more similar technologies aleady existing, for decades.
doing it on a per language basis is not ideal. any new language would have to reinvent the wheel.
better to do it at system level. with the already existing tooling.
openbsd has plege/unveil, linux chroot, namespaces, cgroups, freebsd capsicum or w/e. theres many of these things.
(i am not sure how well they play within these scenarios, but just triggering on the sandboxing comment. theres plenty of ways to do it as far as i can tell...)
What if I wanted to write a program that uses untrusted libraries, but also does some very security sensitive stuff? You are probably going to suggest splitting the program into microservices. But that has a lot of problems and makes things slow.
The problem is that programs can be entire systems, so "doing it at the system level" still means that you'd have to build boundaries inside a program.
This assumes that we can get a locked down, secure, stable bedrock system and sandbox that basically never changes except for tiny security updates that can be carefully inspected by many independent parties.
Which sounds great, but the way things work now tend to be the exact opposite of that, so there will be no trustable platform to run the untrusted code in. If the sandbox, or the operating system the sandbox runs in, will get breaking changes and force everyone to always be on a recent release (or worse, track main branch) then that will still be a huge supply chain risk in itself.
I've been thinking the same thing. And it's somewhat parallel to what happened to meditation vs. drugs. In the old world the dangerous insights required so many years of discipline that you could sort of trust that the person getting the insight would be ok. But then any idiot can get the insight by just eating some shrooms and oops, that's a problem. Mostly self-harm problem in that case. But the dynamic is somewhat similar to what's happening now with LLMs and coding.
Software people could (mostly) trust each other's OSS contributions because we could trust the discipline it took in the first place. Not any more.
Except that LiteLLM probably got pwned because they used Trivy in CI. If Trivy ran in a proper sandbox, the compromised job could not publish a compromised package.
(Yes, they should better configure which CI job has which permissions, but this should be the default or it won't always happen)
LiteLLM maintainer here, this is still an evolving situation, but here's what we know so far:
1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy...
https://ramimac.me/trivy-teampcp/#phase-09
2. If you're on the proxy docker, you were not impacted. We pin our versions in the requirements.txt
3. The package is in quarantine on pypi - this blocks all downloads.
We are investigating the issue, and seeing how we can harden things. I'm sorry for this.
- Krrish
> - Krrish
Was your account completely compromised? (Judging from the commit made by TeamPCP on your accounts)
Are you in contacts with all the projects which use litellm downstream and if they are safe or not (I am assuming not)
I am unable to understand how it compromised your account itself from the exploit at trivvy being used in CI/CD as well.
1.82.7 doesn't have litellm_init.pth in the archive. You can download them from pypi to check.
EDIT: no, it's compromised, see proxy/proxy_server.py.
Are you sure you are pinned to a “known good” version?
No one initially knows how much is compromised
Update:
- Impacted versions (v1.82.7, v1.82.8) have been deleted from PyPI
- All maintainer accounts have been changed
- All keys for github, docker, circle ci, pip have been deleted
We are still scanning our project to see if there's any more gaps.
If you're a security expert and want to help, email me - [email protected]
theanonymousone
