Mastodawn

Eva Jan 30, 2024

As someone in software for a while that has been the most surprising thing that Open Source did: It massively increased complexity even for small projects because "that's how Google/Amazon/etc do it".

Yes Amazon does Microservice architectures. They also have a few people for every service that knows the ins and outs. You have a team of 5 that now not only has to understand the problem but juggle dependency chains from here around the moon and back so your React App that should have been plain HTML doesn't fail while showing a basically static page.

You won't grow to Google/Amazon scale. It's fine. Just build a simple solution you can maintain.

Working on React/K8s or whatever is mostly you training yourself on your own dime and time to be a potential hire for some Big tech company that will fire you to juice the numbers at he end of the next quarter.

Markus Feilner (has moved)Jan 29, 2024

@tante Agreed, that's the old KISS principle of Unix. But on the other hand, isn't microservices architecture also exactly KISS?
Not asking for a friend, but dragging him in here... :-)
@dragotin

Florian Haas Jan 29, 2024

@mfeilner The acronym that matters here isn't KISS, it's YAGNI (Ya Ain't Gonna Need It).

Thomas Fricke (he/his)Jan 30, 2024

@mfeilner @tante @dragotin

For servers, Unix, multiuser logins, Bash, many programming languages, LAMP stack, Kernels and init systems are bloat. From a #microkernel perspective everything is bloat.

#kubernetes is for people who need to automate and scale. Teams, servers, languages, locations, applications.

If you don't need hundred servers, don't use it. Period.
Ten node clusters are a vaste of effort.

There are projects, who need scale, even outside the clouds.

Thomas Fricke (he/his)Jan 30, 2024

@mfeilner @tante @dragotin

Use #Erlang if you want simple but distributed systems. 1200 Bytes per thread is unbeaten.

Any object oriented language turned into a bloated mess. C++, Dotnet, Java and counting

Every software developer dreams of his or not very often her programming language. This is nonsense.

Stop this. Do the engineering.

AI will not save us.

Entertainment Porpoises Only Jan 29, 2024

@tante this, always this

wb x64 Jan 29, 2024

@pikesley @tante currently struggling to decide if it's actually easier and cheaper to pay for one t4.xlarge instead of three small docker containers and a database container... after almost ten years of trying my best to architect the system away from being a single t4.xlarge

LAMP on a single VM will outlive us all, it seems

Anthropy Jan 29, 2024

@pikesley @tante I honestly love these- but what if I just like microservices/libraries/etc as a form of separation of concern in my own head :P

not to say one should make things overly complex, but they don't have to be if you don't immediately try to make each of those parts as massive and complicated as the varieties at Big Cloudy.. though then perhaps we should call those Nanoservices or something along those lines instead to avoid giving the wrong idea to your co-developers/contributors? 🤔

Matti Järvinen Jan 30, 2024

@pikesley @tante dividing to smaller parts also brings separation of concerns and quicker deployments.

Ok, this isn't a problem of hobby scale applications at all.

Currently I'm wrestling with reducing deployment time of a 60k FTE scale product, it's a monolith and requires almost an hour to just deploy even for a smallest part of it. Its projects are interconnected on so many levels that it's almost impossible to separate partial deployments.

kfet Jan 29, 2024

@tante Monoliths and monorepos are really really bad for rapidly evolving software, where you want clear lines of responsibility separation.

Whether that means processes or modules is a choice, and usually depends on how it needs to scale. Yes, scale is a very real problem to solve, it is not imaginary at all.

TLDR; mono* just results in overpriced spaghetti

vriesk (Jan Srz)Jan 30, 2024

@kfet @tante there's no organizational difference between a monorepo split into subdirectories with separate OWNERS (or whatever else the ownership notion is supported with) and multiple repos.

The main difference is whether you want to drive everything from the same HEAD or live in neverending publishing hell for each of your common dependencies.

kfet Jan 30, 2024

@vriesk @tante It’s a headache, and only works with lots of custom tooling to fix the mess that is a shared source control, which was never intended that way, i.e. shared commit history and branches of entirely unrelated modules.

Single repo for a self-contained module is great of course.

vriesk (Jan Srz)Jan 30, 2024

@kfet @tante filtering history affecting only specific subdirectory is rather on the trivial side of that "custom tooling", while I agree, this model does not work very well with long-lived branches. Monorepo is for HEAD-based development.

@tante I know! Instead of react, I'll make a static page in node.js!

➴➴➴Æ🜔Ɲ.Ƈꭚ⍴𝔥єɼ👩🏻‍💻Jan 29, 2024

@tante Frequently leadership at orgs are composed of people that never really had to get their hands dirty.
So, they make decisions based on a cargo-cult mindset.

Arcaik Jan 29, 2024

@tante While I don't entirely disagree, for my homelab, it's way simpler to run stuff in k8s than if I had to set them up on VMs.

d@nny disc@ mc² Jan 29, 2024

@tante twitter only broke out into microservices after it became relevant enough to matter and the billing app for ad spend (the "moneyrail") remained until at least 2019 (i helped kill it as an intern project)

d@nny disc@ mc² Jan 29, 2024

@tante i personally like react as a framework and have used it as a frontend novice to make very very simple web pages implementing analysis tools for a lab i worked with but kubernetes is absolutely literally the thing you only use if you happen to already be extremely familiar with it and systems like it

d@nny disc@ mc² Jan 29, 2024

@tante i agree with you that react requiring precompilation can be a very centralizing force and that's why facebook supports it

MaineC Jan 29, 2024

@tante maybe that's what we get for letting Google scale companies themselves and VC backed startups targeting Google scale companies maintain open source

Wasn't there a time when smaller players used open source as a development model to join forces and solve the problems they had that vendors could not of would not solve?

Glyph Jan 29, 2024

@tante relevant: https://blog.glyph.im/2023/07/bilithification.html

Bilithification

Not sure how to do microservices? Split your monolith in half.

MistherFrenchFries Jan 29, 2024

@tante I remember all the software books in the software section of the bookstore. I remember ‘computers’ edging in between doctors and lawyers as the new way to escape poverty.

Pauline Oplossingen bestaan al Jan 29, 2024

@tante I don't agree. I look at it form an Ops perspective. Yes, If you just need a simple application or static page, than build , host and maintain that.
However one of the main reasons why people started using docker for example is that applications are easily shippable and deployable. It works the same for everyone.
Now If I need redundancy at application level you will soon need something like k8s. Or you will have to maintain several servers on which you run the containers with in front some load balancing service.
That is a lot of maintaining and engineering if you compare that to a ready to go k8s service with a lot of tooling and templates available.

Ps.
I work for local government as a freelancer and will never work for big tech.

@tante Not everyone needs Kubernetes

qsenn Jan 29, 2024

@tante Yes! Especially on the point of react being used in areas HTML + JS work fine. In my side projects I’ve practically sworn off React. The more I learn about options prebaked into JavaScript, the less I value frameworks for a majority of apps.
I can’t imagine how many web apps can be rebuilt using WebComponents and CustomEvents and simple custom state management library with less complexity. Worst case, just use Lit or Enhance.
#JavaScript #React #WebComponents

CassandraVert Jan 30, 2024

@tante
Unnecessary complexity seems like an immature stage of development in a lot of areas, from writing to art to computer programming. Maybe we're seeing younger developers on the front lines instead of at the bottom of a dev team.

Linh Pham Jan 30, 2024

@tante One of the projects that people had been betting on at work was architected to be microservice and micro-frontend to the max and that lead to so many problems.

That project was supposed to be the foundation for the next gen versions of several apps. Due to the issues and complexity, the project has been shifted into maintenance mode. 🤦🏻

Linh Pham Jan 30, 2024

@tante Every time there was a release, the minimum number of components that a planned release would have hovered around 20-25. Major changes would ramp that up to 40+.

Since we have to detail a bunch of stuff for various auditing and process requirements, it took 5-10 times longer to document the deployment, testing and rollback procedures than it would to actually do the deployment itself.

Deity forbid that you had to rollback or have the deployment pipeline fail in any manner.

Needlessly complicated. Funny thing is, the person who heavily contributed to the design also worked on a prior attempt at a next-gen platform that failed because it was also needlessly complicated.

That's saying something as the platform that they wanted to replace was an attempt at microservices and they made how the services would be called and authenticated absolutely bass-ackwards.

morgan Jan 30, 2024

@tante How is this something "open source did"?

The Long Tail Jan 30, 2024

@mw @tante it isn't, it is something Big Tech did, by weaponising open source.
Nobody in their right mind would have built K8s or React or Kafka themselves to help their small company - these are all projects dreamed up in Big Tech orgs to fix Big Tech problems which Big Tech caused.
If you aren't Big Tech, you don't need it.
Also, the public facing FOSS projects all lack the secret sauce, e.g. Goog don't use K8s internally, they use Borg.

Philip Brechler Jan 30, 2024

@tante I think this "need" for complexity and scalability is not just in the development part, it's also then spilling over into the adjacent functions like UX or Product. There is also the tendency to build more and more complexity

Mark T. Tomczak Jan 30, 2024

@tante > Working on React/K8s or whatever is mostly you training yourself on your own dime and time to be a potential hire for some Big tech company that will fire you to juice the numbers at he end of the next quarter.

I was already planning to learn them; you don't have to sell me on them any harder.

mhoye Jan 30, 2024

@tante I think the problem is worse than that, with ambitious people at smaller shops specifically choosing React/K8s/other bigcorp-tech _specifically so_ they get on the job training for their next gig at the expense of their current gig.

One of biggest underappreciated benefits of supporting (and paying for) employee learning/development programs is that they don't need to run up the tech debt in your shop that funds their next career move and leaves your org holding the bag.

tante Jan 30, 2024

@mhoye very good point

Trouble Jan 30, 2024

@tante Every architectural change is a tradeoff. Monolith or microservice, there's associated cost with every choice.

For example, if a dev can use anyone else's code, all public functions are now APIs and owners can't change them without breaking dependent's compiles. Language visibility isn't enough. "visibility" rules can help. APIs and microservices add stricter enforcement.

You can easily design yourself into a corner if you don't understand the tradeoffs at all levels.

Bartek Jan 30, 2024

@tante I only trust myself to be able to maintain a k8s deployment. Anything dealing with underlying infra is too low level, complex and costly.

Julianoë Jan 30, 2024

@tante I've been client-splained on some freelance project by a client that was paying my webdev services to make a website/webapp for a small non profit org and asked me to do this and that because "that's how YouTube does it, so that must be the way to do it". He did not want to understand that he was a medium website running on volunteers while YouTube was backed by a megacorp with thousands of employees

Julianoë Jan 30, 2024

@tante "look we want the comments section to act exactly like the comments on YouTube". "Can you change the share to social media section to be the same on YouTube"

Sandor Szücs Jan 30, 2024

@tante a bit too pessimistic view. Of course as startup it makes no sense to start an architecture that could run at enormous scale (let’s say micro services in kubernetes, because it is the regular bashing topic), if you don’t need it immediately.
I am pretty sure companies that are building SaaS with a lot of ingestion like logs or spans or profiling will say that they need it at day one.
If the company grows to a certain site like >100 engineers you will rebuild anyways.

trdebunked Jan 30, 2024

@tante and when you increase the complexity of projects a la g/a/f/a/m, you get projects that require g/a/f/a/m support to be sustained.

in other words, this is how free software turns into software controlled by monopolies- using mostly unpaid or low-paid, digital sweatshop labour.

not an attempt to conflate free software with proprietary software. the corporations already do that. there are still nuances. there are still exceptions.

trdebunked Jan 30, 2024

@tante also it should not be a surprise. this is exactly the strategy microsoft laid out in internal memos the same year the phrase "open source" gained widespread notoriety.

this was what "open source" was always going to become.

Niclas Hedhman Jan 31, 2024

A whole bunch of posts today with this same sentiment.

Am I killing myself, trying to set up k8s rather than "just" try to deploy on bare metal again (on VMs at the moment, and not happy with storage pricing).

kccqzy Jan 31, 2024

@tante @hyc A big part of the advantages of Google or Amazon is that they can afford to write more code to do less. A small shop shouldn't.

matt Jan 31, 2024

@tante scale goes in both directions. Can that platform you're considering scale _down_ as well as up?

Cody Krieger Jan 31, 2024

. @tante to add some nuance to this—i don't think this is the fault of open source in general, but i definitely agree that some of the big corporate-sponsored open source projects breed this kind of thinking and behavior.

tuxflo Jan 31, 2024

@tante well I get your point but I've also seen the other way round. Where small companies create everything from scratch to get features, which are included in the big frameworks/libraries and then become an unmaintainable nightmare. In one of my projects they built their own "kubernetes light" to get rolling updates and almost zero downtime deployments.
As you can guess, nobody understood the system and the onboarding process was very hard.

Eric Carroll Feb 6, 2024

@tante
If I had a penny for every time I was told by a dev "that's how Google does it" and then had to argue "I don't think so and it doesn't make sense for our scale point"...

Then I worked for my most recent boss, an ex-googler on Borg.

Learned a *ton*.

Great discussions on culture, process and flow.

Guess what?

Those absolutely certain, totally confident devs?

They were all dead wrong.

Alex Hart Feb 23, 2024

@tante I feel this every time I see a new "Clean Architecture Android Sample" blahblahblah pop up anywhere. So much unnecessary abstraction, so much boilerplate. People are treating patterns as gospel instead of trying to seek and understand the principles that drive them.