floofloofAmazon cloud boss echoes NVIDIA CEO on coding being dead in the water: "If you go forward 24 months from now, it's possible that most developers are not coding"
Everybody talks about AI killing programming jobs, but any developer who has had to use it knows it can do anything complex in programming. What it’s really going to replace is program managers, customer reps, makes most of HR obsolete, finance analysts, legal teams, and middle management. This people have very structured, rule based day to days. Getting an AI to write a very customized queuing system in Rust to suit your very specific business needs is nearly impossible. Getting AI to summarize Jira boards, analyze candidates experience, highlight key points of meetings (and obsolete most of them altogether), and gather data on outstanding patents is more in its wheelhouse.
I am starting to see a major uptick in recruiters reaching out to me because companies are starting to realize it was a mistake to stop hiring Software Engineers in the hopes that AI would replace them, but now my skills are going to come at a premium just like everyone else in Software Engineering with skills beyond “put a react app together”
Trouble is, you’re basing all that on now, not a year from now, or 6 months from now. It’s too easy to look at it’s weaknesses today and extrapolate. I think people need to get real about coding and AI. Coding is language and rules. Machines can learn that enormously faster and more accurately than humans. The ones who survive will be those who can wield it as a tool for creativity. But if you think it won’t be capable of all the things it’s currently weak at you’re just kidding yourself unfortunately. It’ll be like anything else - a tool for an operator. Middlemen will be wiped out of the process, of course, but those with money remain those without time or expertise, and there will always be a place for people willing to step in at that point. But they won’t be coding. They’ll be designing and solving problems.
An inherent flaw in transformer architecture (what all LLMs use under the hood) is the quadratic memory cost to context. The model needs 4 times as much memory to remember its last 1000 output tokens as it needed to remember the last 500. When coding anything complex, the amount of code one has to consider quickly grows beyond these limits. At least, if you want it to work.
This is a fundamental flaw with transformer - based LLMs, an inherent limit on the complexity of task they can ‘understand’. It isn’t feasible to just keep throwing memory at the problem, a fundamental change in the underlying model structure is required. This is a subject of intense research, but nothing has emerged yet.
Transformers themselves were old hat and well studied long before these models broke into the mainstream with DallE and ChatGPT.