My job as a senior developer with a team of juniors is to figure out what to write, sketch a PoC as guidance, and then delegate the actual implementation to them. I'm going to look at that, explain misunderstandings or poor style choices, and guide them into implementing something that meets our standards.
I don't think LLMs can do my job yet. But I think we're getting shockingly close to them being able to do the other part. And I'm worried how we're going to get more senior developers.
@glyph @mjg59
that's entirely fair, and they have been getting better, but what constitutes "worth using" is pretty individual. I'm curious if you have any examples of something you'd quantity that way.
Maybe some relatively complex feature or bugfix you already wrote that you'd like to use as a benchmark for capability? Alternatively, a couple of trivial features you'd like in a personal project but haven't gotten around to building?
At a more mundane level, I suspect they could reliably alleviate a significant amount of the drudgery associated with maintaining OSS - fixing tests when dependencies are updated, etc. Nothing you can't trivially do yourself, but also in my experience painful to try to get the ADHD brain to pay attention to.
1. Why do some people develop AI psychosis and others don't? Or does everyone eventually succumb and we just haven't used it longitudinally enough? Hormesis or linear-no-threshold?
2. How can one maintain a balance of failed-vs-successful prompts, to avoid time-wasting? Intuitive evaluation will always favor the successes.
3. If the tech *does* actually work, doesn't give me psychosis, and works more often than not with enough of an edge, de-skilling seems like a big problem.
@glyph @mjg59 I also worry about addiction. As Netflix learned, hours-go-up is likely a bad thing for your business to optimize in isolation, because its usually bad for your customers too. I know my team works hard to avoid that trap.
A lot of that concern is related to my earlier response to #1, but it can also be a great enabler of hyperfocus, which can be both very pleasurable and counterproductive.
All that said, you seem pretty convinced that using these things is like pulling the arm on a slot machine - sometimes you get a reward but a lot of the time you get garbage and have to try again a different way. They really truly are not like that these days in my experience, and have not been like that for a while. If you model them or their users that way in your reasoning, you will be making category errors.
As a user, (and maybe you'll say I have AI psychosis) the experience is more like working with a very fast, very precocious junior who has memorized half of wikipedia and is very quick with google, and who is getting better at writing code, but reasonably often needs detailed instruction or directional course correction. You don't cut their head off and ask the talent agency to send you a new one every time they give you an answer that doesn't quite match what you want, you clarify your request, or ask for a more achievable scope of work. Unlike searching google, your queries compound to vector the agent where you want it to go, conversationally, rather than standing alone individually.
@glyph @mjg59 for ai psychosis, my sense is that the observed outcomes are some combination of "already psychotic/already narcissist", people who are unusually susceptible to the same validation/reinforcement traps used in social media who discover the feedback loop can be instantaneous and permanently tilted in their favor, and an unfortunate subset of people who are prone to believe everything they read.
Which models they interact with, and how those are configured, makes a big difference. Some models are brokenly sycophantic, and that encourages this. Some models gladly engage in the kind of secret world government mind control "I discovered secrets the FBI needs to know about" kind of roleplay that draws susceptible people in. Training the model to refuse to go down these rabbit holes and keeping discussions factual is a hard problem, but one that modern models are much better at.
These dangers are one of the reasons that readily accessible open source model weights with near frontier capabilities worry me. I recognize that sounds hypocritical given my employer, but these systems are easier to misuse, and their snapshot-in-time nature can't benefit from ongoing safety work.
My belief is that occurrence is the product of underlying susceptibility, multiplied by unsafe model behavior. If those don't combine to meet a threshold level, people stay grounded in the real world. I don't see longitudinal use as an additional risk, although it obviously exacerbates symptoms for people who are above that threshold.
With modern models deployed with safety measures from the major providers, I think the risk is relatively low for most users.
@glyph @mjg59 re:3 - eh, maybe. I think I'm not convinced either way on that one yet.
On the one hand, these days I typically don't bother looking in detail at large chunks of the output where I can measure the results, and have used it for long enough to feel comfortable with the levels of reliability I can expect from it. I'm typically very comfortable asking it to read logs, write debug harnesses, and generally check its own work under my supervision. I've also used it enough that I have a pretty good ability to smell when it's going off the rails, or is likely to do so.
That, sense I think, is something that takes longer for someone who isn't already senior to develop, but at the same time, I've had some fascinating conversations with people who started from almost no ability to code and are now running (and debugging, and fixing) pretty significantly complex projects. It's neat to see the lowered barrier to entry allowing people to scratch their own itches, but I am absolutely certain that "using an LLM" doesn't preclude skills development or the ability to fix complex problems. You're just starting at a different abstraction layer, and now have a handy built-in tutor and contextual reference guideif you want to move up or down the stack to learn more.
The problem of "who will pay for the juniors for long enough that they become senior enough to be worth employing" is probably going to be a real problem. I'm not sure there is a good answer to that one aside from acknowledging that there will be disruption, and that our social contracts in America particularly are utterly broken.
Production of all hardware, building and operation of all data centres are huge environmental issues, and while human activity was certainly extremely polluting even before that, the whole content generating stuff comes on top of that.
This idiotically might not concern companies like your employer.
But content generation models shift power to those who own them.
This might also not concern your employer, but if it's a SW corp they're externalising their core product
@PaulM it's more a direct reply to Glyph then to you, so I think I want to headline Glyph? But I don't have a firm enough grasp on the workings of mastodon to be certain either way.
email / reddit style discussion trees have their merrit, I'd rather have that (and no character cap, but I digress)
@ari well then, I'm not sure why you're talking like that to glyph. He's an independent open source developer who does consulting and has a patreon and doesn't use LLMs and has contributed to a bunch of fundamental internet software you undoubtedly use.
So you kinda sound like an asshole here.
@mjg59 I dunno. Sure we'll get a lot of (more) terrible apps, services, etc in the short term. "AI" is sort of accelerating the "everything is unreliable slop-ware" trend that's been infecting software development for the past decade plus.
At some point I suspect there may (once again) be a market for software that's not utter garbage.
I feel bad for everyone stuck working for these awful companies (increasingly *all* companies), while the industry destroys its capability to write software.
@mjg59 @greg I agree wholeheartedly with the junior pipeline problem, though I suspect that we end up with junior devs who are good at piloting the models, and learn to debug even hard problems within that context.
We didn't stop being able to computer when people stopped learning assembly or c, I hope we have a similar outcome here.
Matthew Garrett
Glyph
gunstick
Mike Roach
Paul McMillan
ari
Wouter Verhelst
Brian Swetland
Val Packett 🧉
viq
Jason Haar 
Jess👾
Greg
Nemo
Chris-Evelyn
Alan Coopersmith