So this was a whole shit show that as of now has cost me 2 days of investigation, clean-up, mitigation, will likely cost me more time & also feels extremely crappy all in all.
I'm just glad I didn't base any *really* important decisions on those manipulated numbers, that would have been REALLY bad.
Now I have to live with knowing that most of the growth I saw the past 2 years was likely faked, which feels quite terrible tbh. Feels like not doing a good job after all.
Well, the analysis and mitigation is ongoing, and in the process of that I found another ... thing. The verdict is still out on this being a case of a rampant running CI, or some weird VPN endpoint, or something evil. The traffic from the cloud IP I found was definitely organic, but still had some issues and also was too short lived per instance identifier.
Nuked everything from that source. Goodbye 100k instances 😢
Shitty week, really. And still not done with analysis and mitigation.
Well. FFS 😡 Even more manipulation. Spent the whole day analysing again, compiled a report, confronted Obico about it and they admitted it. Will post about that on the OctoPrint blog tomorrow. I have no energy left today.
https://obico.io/blog/2024/07/03/my-apologies-for-the-mistake/
I'm writing this post to apologize to Gina Häußge, one of the most important and respectable person in 3D-printing community, as well as to the OctoPrint community for the stupid and selfish mistake I made and the harm I have caused:
Just did a round on the punching bag, with Limp Bizkit's "Break Stuff" blasting from the sound system.
Doesn't solve this of course, but at least the adrenaline is out of my system again. I can seriously not recommend it enough to have a punching bag in your office... And that's not a joke. I literally have that.
@foosel you are showing an _enormous_ amount of civility and restraint. Kudos to that.
If I were to judge, these people owe you not only compensation for the cleanup time, but they also owe the community for the deception, something they will never be able to make up for in my opinion.
@foosel What is wrong with people? 😩
Sorry that you have to go through all this!
@foosel
What is it with those two plugins?
Their titles in the overviews are suspiciously similar ("Full Remote Access, AI Failure Detection") - are they forks of each other or just competitors? Looks like they have some kind of dick competition (and piss on everyone elses lawn as a sideeffect) O_o
(sorry if this is a stupid question, I'm not familiar with the Octoprint ecosystem. TBH I don't even own a 3D printer)
@foosel Yikes! Give them the entire boot. No mercy.
I can't fathom how someone could think that would ever be acceptable behavior.
@foosel Very disappointing to hear about this scummy behaviour.
Rest assured: Octoprint is awesome, and your work is appreciated.
@foosel I know from Ad click fraud and Boaty McBoatface that if you give people an inch they’ll take a mile- but sheesh, pumping a plug-in a few places up the stats page is… just why!?
I guess it’s at least a form of flattery that they found your project important enough to manipulate… 😬
But don’t sweat it too much- growth is a poor substitute to consistently putting out an awesome tool that people love and use! ❤️
@gadgetoid The top ten list was a requested feature by several plugin devs, and I totally get it, it's exciting to watch the movement on that list, especially if you have some skin in the game (and DON'T manipulate the data/have a fanboy manipulate the data... yes, I'm angry still).
But @do3cc just suggested to add some filtering functionality to be able to filter out commercial plugins, and I think that's a great idea that I'll look into implementing.
@foosel please take a little time to process and cleanse these feelings.
You are succeeding.
Your work is powering my practice, and many others too. I can hear my printer running now, and you are to thank.
This jank episode is a fruit of your success.
As your success grows, more haters and fakers and manipulators and capitalists and scammers, and every type of flim-flam artist will be trying to find an angle.
It may feel bad to spend efforts on defence, but valuable work needs protecting.
@jeroen94704 tbh, I'm just relying on clear evidence in my judgement here.
Their statement sounds plausible and there's no proof for or against it. I don't see a point in pushing this further, I'm pretty sure that whoever did this will not be doing it again given the outcome.
@do3cc What you already have since of a few weeks ago is "commercial", "cloud" and "free tier" tags on plugins.octoprint.org, and with 1.11.0 those will also be shown in the internal plugin repo browser.
But that's a good point in allowing to filter based on that info, I'll look into adding that.
@oliviersaraja Exactly.
And no bouldering session today (that would also already have been done), I got my StroVac booster on Wednesday and my left arm is still just a thing of agony whenever I try to heave it over 90 degrees 😅
@oliviersaraja Hopefully soon, for now it's keeping an eye on the reactions to this thing and waiting for some more updates from OctoEverywhere 😅
But at least my AC is working fine.
@foosel this may feel bad but you did a good job. I am using octo because it simply works, and others will agree. It's good though that you come clean with someone else's bad behavior, and that too is part of doing a good job.
Thank you for your work. I am sure you have the community's back. You rock. 💪🏽
@bear_lab Thank you for your words of encouragement! I'm still angry and sad that someone did this, but I am also not going to allow this thing to be the final straw.
Currently trying my best at distracting myself and doing stuff I enjoy (just finished the textblock of another book), come Monday I'll go back to some more investigation and prevention.
Andrew LeCody
Gina Häußge
🪬🍄🌈🎮💻🚲🥓🎃💀🏴🛻♓🧿
Harald now @ treehouse.systems
Uckermark MacGyver 
DJGummikuh
AxWax
Peter Mount
Daniel Gibson
Fritz Adalis
balloob
Jim P.
__Miguel_
tjhowse
gadgetoid
Ryan Makes, Dreamscaper
🇳🇱🇪🇺Jeroen🇺🇦🇺🇦🇺🇦🇺🇦
Patrick Gerken
Olivier Saraja ☕️📚🦖🛸👻
Pxl Phile
Mike Shaver 🤷🏼♂️
TeflonTrout 
he/him
Sandzwerg
Trevor Flowers
Nathan Byrd
Greg Saunier
Torsten