cm0002Tradeoffs
RustyNovaAnd of course the ai put rail signals in the middle.
Chain in, rail out. Always
!Factorio/Create mod reference if anyone is interested !<
mercSpoilerYour spoiler didn’t work.
orcaAnd then 12 hours spent debugging and pulling it apart.
heavydustAnd if you need anything else, you have to use a new prompt which will generate a brand new application, it’s fun!
That’s not really how agentic ai programming works anymore. Tools like cursor automatically pick files as “context”, and you can manually add them or the whole ckdebase as well. That obviously uses way more tokens though.
We’re in trouble when it Larsen’s to debug.
But then, as now, it won’t understand what it’s supposed to do, and will merely attempt to apply stolen code - ahem - training data in random permutations until it roughly matches what it interprets the end goal to be.
We’ve moved beyond a thousand monkeys with typewriters and a thousand years to write Shakespeare, and have moved into several million monkeys with copy and paste and only a few milliseconds to write “Hello, SEGFAULT”
And it still doesn’t work. Just “mostly works”.
A bunch of superfluous code that you find does nothing.
AllNewTypeFaceYou can instantly get whatever you want, only it’s made from 100% technical debt
That estimate seems a little low to me. It’s at least 115%.
even more. The first 100% of the tech debt is just understanding “your own” code.
Just start again from scratch for every feature 🤣
mesaIm looking forward in the next 2 years when AI apps are in the wild and I get to fix them lol.
As a SR dev, the wheel just keeps turning.
I’m being pretty resistant about AI code Gen. I assume we’re not too far away from “Our software product is a handcrafted bespoke solution to your B2B needs that will enable synergies without exposing your entire database to the open web”.
without exposing your entire database to the open web until well after you check to us has cleared, so it’s fine.
Lol.
It has its uses. For templeting and/or getting a small project off the ground its useful. It can get you 90% of the way there.
But the meme is SOOO correct. AI does not understand what it is doing, even with context. The things JR devs are giving me really make me laugh. I legit asked why they were throwing a very old version of react on the front end of a new project and they stated they "just did what chatgpt told them" and that it "works". Thats just last month or so.
The AI that is out there is all based on old posts and isnt keeping up with new stuff. So you get a lot of the same-ish looking projects that have some very strange/old decisions to get around limitations that no longer exist.
WrittenInRed [She/They]Yeah, I think personally LLMs are fine for like writing a single function, or to rubber duck with for debugging or thinking through some details of your implementation, but I’d never use one to write a whole file or project. They have their uses, and I do occasionally use something like ollama to talk through a problem and get some code snippets as a starting point for something. Trying to do too much more than that is asking for problems though. It makes it way harder to debug because it becomes reading code you haven’t written, it can make the code style inconsistent, and a non-insignifigant amount of the time even in short code segments it will hallucinate a non existent function or implement something incorrectly, so using it to write massive amounts of code makes that way more likely.
The CursoAI debugging is the best experience ever.
It’s so much easier than googling don’t stack trace and then browsing GitHub issues and stack overflow.
The AI also enabled some very bad practices.
It does not refactor and it makes writing repetitive code so easy you miss opportunities to abstract. In a week when you go to refactor you’re going to spend twice as long on that task.
As long as you know what you’re doing and guide it accordingly, it’s a good tool.
Holdup! You’ve got actual, employed, working, graduated juniors who are handing in code that they don’t even understand?
Our gluten-free code is handcrafted with all-natural intelligence.
Give it time, eventually every project looks like the right.
I mean, not quite every project. Some of my projects have been turned off for not being useful enough before they had time to get that bad. Lol.
penquinIf you know what you’re doing, AI is actually a massive help. You can make it do all the repetitive shit for you. You can also have it write the code and you either clean it or take the pieces that works for you. It saves soooooo much time and I freaking love it.
Shhhh! You’re not supposed to rock the AI hate boat.
Lmao. I don’t give a shit. I’ve been saving a ton of time ever since I started using it. It gobbles up CSS, HTML and JS like hotcakes, and I’m very much ok with that.
I hate the ethics of it, especially the image models.
But frankly it’s here, and lawyers were supposed to have figured out the ethics of it.
I use hosted Deepseek as an FU to OpenAI and GitHub for stealing my code.
That’s the thing, it’s a useful assistant for an expert who will be able to verify any answers.
It’s a disaster for anyone who’s ignorant of the domain.
Tell me about it. I teach a python class. Super basic, super easy. Students are sometimes idiots, but if they follow the steps, most of them should be fine.
Sometimes I get one who thinks they can just do everything with chatgpt. They’ll be working on their final assignment and they’ll ask me what a for loop is for. Than I look at their code and it looks like Sanscrit. They probably haven’t written a single line of code in those weeks.
It’s taken me a while to learn how to use it and where it works best but I’m coming around to where it fits.
Just today i was doing a new project, i wrote a couple lines about what i needed and asked for a database schema. It looked about 80% right. Then asked for all the models for the ORM i wanted and it did that. Probably saved an hour of tedious typing.
I’m telling you. It’s fantastic for the boring and repetitive garbage. Databases? Oh hell yeah, it does really well on that, too. You have no idea how much I hate working with SQL. The ONLY thing it still struggles with so far is negative tests. For some reason, every single AI I’ve ever tried did good on positive tests, but just plain bad in the negative ones.
I assume we’re talking about software testing? I’d like to know more about:
-
The meaning of negative and positive tests in this context
-
Good examples of badly done negative tests by LLMs
Yes, software, and specifically C# unit tests in my case.
Positive unit tests check if the code works as expected when given valid inputs. They confirm that the function or module behaves correctly under normal conditions.
Negative unit tests check how the code handles invalid or unexpected inputs. They ensure that errors are properly caught, exceptions are handled, and the system doesn’t break when things go wrong.
As for examples, it’s just the LLMs I have tried never wrote negative tests that actually worked. If you use Visual Studio, you’re probably familiar with those check marks that it has on unit tests. Those become green check marks when the test is valid, red X when it is invalid (isn’t correct). The negative tests from LLMs always have red X’s. Hope this makes sense.
As for examples, it’s just the LLMs I have tried never wrote negative tests that actually worked. If you use Visual Studio, you’re probably familiar with those check marks that it has on unit tests. Those become green check marks when the test is valid, red X when it is invalid (isn’t correct). The negative tests from LLMs always have red X’s. Hope this makes sense.
Thank you for thoroughly explaining this. Your explanations make good sense to me.
I’ve been trying to use aider for this, it seems really cool but my machine and wallet cannot handle the sheer volume of tokens it consumes.
I don’t even know what aider is. Lol. There are so many assistants out there. My company created a wrapper for chatgpt and gave us unlimited number of tokens and told us to go ham.
Aider is an LLM agent type app that has a programming assistant and an architect assistant.
You tell the architect what you want and it scans the structure of your code base to generate the boilerplate. Then the coder fills it in. It has command prompt access to then compile and run etc.
I haven’t really figured it out yet.
Damn, sounds like it could do some wonders.
2deckIf you’re having to do repetitive shit, you might reconsider your approach.
Depending on the situation, repetitive shit might be unavoidable
Usually you can solve the issue by using regex, but regex can be difficult to work with as well
Skill issue…
Nah, I’m good the way I do things. I have a good pace that has been working out very well for me :)
stilgar [he/him] I’ve tried this, to convert a large json file to simplified yaml. It was riddled with hallucinations and mistakes even for this simple, deterministic, verifiable task.
I turned on copilot in VSCode for the first time this week. The results so far have been less than stellar. It’s batting about .100 in terms of completing code the way I intended. Now, people tell me it needs to learn your ways, so I’m going to give it a chance. But one thing it has done is replaced the normal auto-completion which showed you what sort of arguments a function takes with something that is sometimes dead wrong. Like the code will not even compile with the suggested args.
It also has a knack for making me forget what I was trying to do. It will show me something like the left side picture with a nice rail stretching off into the distance when I had intended it to turn, and then I can’t remember whether I wanted to go left or right? I guess it’s just something you need to adjust to. Like you need to have a thought fairly firmly in your mind before you begin typing so that you can react to the AI code in a reasonable way? It may occasionally be better than what you have it mind, but you need to keep the original idea in your head for comparison purposes. I’m not good at that yet.
I don’t mess with any of those in-IDE assistance. I find then very intrusive and they make me less efficient. So many suggestions pop up and I don’t like that, and like you said, I get confused. The only time I thought one of them (codium) was somewhat useful is when I asked it to make tests for the file I was on. It did get all the positive tests correct, but all the negative ones wrong. Lol. So, I naturally default to the AI in the browser.
Thanks, it makes me feel relieved to hear I’m not the only one finding it a little overwhelming! Previously, I had been using chatgpt and the like where I would be hunting for the answer to a particularly esoteric programming question. I’ve had a fair amount of success with that, though occasionally I would catch it in the act of contradicting itself, so I’ve learned you have to follow up on it a bit.
Oh yeah, of course. You can’t just trust it 100%. One time Claude gave me a piece of code that was a nasty bug that could have caused some serious issues. It was a one liner that deleted an employee from database by mere searching said employee with their name. Thankfully I caught it in the dev environment before it got into prod (assuming AQ missed it, too) and started deleting people. lol.
ikiddTry Roocode or Cline with the Claude3.7 model. It’s pretty slick, way better than Copilot. Turn on Memory Bank for larger projects to reduce the cost of tokens.
I haven’t personally used it, but my coworker said using Cursor with the newest Claude model is a gamechanger and he can’t go back anymore 🤷♂️ he hasn’t liked anything outside of cursor yet
Thanks, I’ll give that a shot.
I knocked off an android app in Flutter/Dart/Supabase in about a week of evenings with Claude. I have never used Flutter before, but I know enough coding to fix things and give good instructions about what I want.
It would even debug my android test environment for me and wrote automated tests to debug the application, as well as spit out the compose files I needed to set up the Supabase docker container and SQL queries to prep the database and authentication backend.
That was using 3.5Sonnet, and from what I’ve seen of 3.7, it’s way better. I think it cost me about $20 in tokens. I’ve never used AI to code anything before, this was my first attempt. Pretty cool.
I used 3.7 on a project yesterday (refactoring to use a different library). I provided the documentation and examples in the initial context and it re-factored the code correctly. It took the agent about 20 minutes to complete the re-write and it took me about 2 hours to review the changes. It would have taken me the entire day to do the changes manually. The cost was about $10.
It was less successful when I attempted to YOLO the rest of my API credits by giving it a large project (using langchain to create an input device that uses local AI to dictate as if it were a keyboard). Some parts of the codes are correct, the langchain stuff is setup as I would expect. Other parts are simply incorrect and unworkable. It’s assuming that it can bind global hotkeys in Wayland, configuration required editing python files instead of pulling from a configuration file, it created install scripts instead of PKGBUILDs, etcetc.
I liken it to having an eager newbie. It doesn’t know much, makes simple mistakes, but it can handle some busy work provided that it is supervised.
I’m less worried about AI taking my job then my job turning into being a middle-manager for AI teams.
I think the further you get out in to esoteric or new things, the less they have to draw on. I’ve had a bit of the same issue building Lora telemetry on ESP32 with specific radio modules because there might be a couple of realworld examples out there of using those libraries.
I feel this pain.
I’ve been trying to get simple telemetry working over lora on a ESP32-C6, LLMs are largely worthless in this. We gotta fall back to old school RTFM models
That’s pretty awesome.
OTOH humans did design the tracks in both images.
I gave it a harder software dev task a few weeks ago… Something that is not answered on the internet… It was as clueless as me, but compared to me, it made up shit that could never work.
I tried to give it a piece of ~200 lines of JS I was positive there was an error in, and tried to gaslight me into thinking there wasn’t any… I tried everything, pointed it specifically to suspicious bits, asked for breakdowns, assertions, test cases… which it then promptly copy-pasted to me straight from my own code… Took me a few hours to find, but there was, in fact, a rookie mistake in it, just hard to spot at a glance.
It depends. AI can help writing good code. Or it can write bad code. It depends on the developer’s goals.