Jesse Skinner

Asking #ChatGPT to write code that I know how to write feels similar to working with a junior developer. It takes a lot of revisions and feedback to get things just right, and it feels like it'd be faster for me to do it myself.

Unfortunately, unlike a human junior developer, ChatGPT can't learn from me over time, due to the limited context window.

Where ChatGPT shines is writing code that I don't know how to write myself, saving me the learning curve of figuring it out.

Mar 29, 2023 at 2:59pmWeb
Show threadJesse SkinnerMar 30, 2023
OK I take it back. The API ChatGPT saved me from learning turned out not to exist. So I had to learn it anyway to figure that ChatGPT was full of shit. Probably took me longer in the end. Sigh.
Show threadJesse SkinnerMar 30, 2023
"Apologies for the confusion, it seems that I am about as useful as a junior developer that is overdosing on LSD."
Show threadJesse SkinnerMar 30, 2023

I just told ChatGPT: "Answer everything with humility and make it clear that you're guessing and not sure if you're right." and now it's at least reminding me that it's making everything up when it answers my questions.

It still hallucinated an API endpoint, but this time when I pointed it out, it had a much more useful response.. (cont..)

Show threadJesse SkinnerMar 30, 2023

"I apologize for the confusion. It seems that the endpoint is not available in the API, which is likely the reason for the "Not Found" error. In this case, it might be best to contact the support team to help you find the necessary endpoint.

Again, I apologize for any confusion and appreciate your understanding of my limitations as an AI language model."

Show threadJesse SkinnerMar 30, 2023

When answering the question initially, it added "However, please note that I might not be entirely accurate, and you should always consult the official documentation or seek expert advice."

IT SHOULD ALWAYS SAY THAT!

Show threadJesse SkinnerMar 31, 2023

Finding a couple use cases with #ChatGPT that are super useful and don't have me pulling my hair out after.

1. Paste in a database schema and get it to write queries based on my plain english descriptions. Protip: ask for "oneliner" to make it easier to paste into the #mysql cli.

2. Paste in a whole module and ask for a code review.

Both of these seem to have a low chance of hallucinations and higher value with low effort.

Show threadAlex Crocker | Domain InvestorMar 30, 2023
@JesseSkinner I have better results with GitHub CoPilot personally. It's not meant to write full programs, but I use it more as a tool in my toolbox. It's VERY helpful for getting up to speed with a new framework, library, or even programming language that you aren't as familiar with without having to reference the docs as much.
Show threadJesse SkinnerMar 30, 2023
@croc yep, I'm a big fan. Maybe it's because I have lower expectations, and copilot suggestions seem more like guesses and less like the confident explanations of an expert. I should look at ChatGPT as guessing more often. Perhaps they should make it say "maybe" and "perhaps" and "I'm not sure but..."
Show threadAlice GGMar 30, 2023
@JesseSkinner Had the same experience 5 mins ago. I was checking if chatgpt could help me with pandas. It manage to point me to the right functions and give code examples that made sense at first, but as the conversation went on it started randomly hallucinating parameters
Show threadBrian HoggMar 29, 2023
@JesseSkinner But then aren't you forced to learn it anyway to get it just right?
Show threadJesse SkinnerMar 29, 2023
@brianhogg I wouldn't have it any other way. I need to understand what is happening so I can make sure it's not doing something wrong, insecure, dangerous, etc.
Show threadBrian HoggMar 29, 2023

@JesseSkinner you had said it saved you the time of learning it, but you’d still need to know it to know if the solution it’s providing is accurate, though. It sounded like you were saying you could sidestep the learning part for things you don’t already know how to do.

(I use copilot, but it’s primarily helpful for me to speed up things I already know how to do.)

Show threadJesse SkinnerMar 29, 2023
@brianhogg ah right, what I meant is that it flattened the learning curve for me, making it a lot faster to learn how to do something instead of searching and reading through docs.
Show threadBrian HoggMar 29, 2023
@JesseSkinner ah, fair! That makes sense.
Show threadAlan Houser ☻␆Mar 29, 2023
@JesseSkinner
I played with that for fun:
https://twitter.com/pixelmelter/status/1639342569485131793
Alan Houser on Twitter

“HOLY #ChatGPT Umm... "Please write HTML for a website that has the following H1 heading: "Welcome to our website!" then an H2 subheading that reads "We're the best in snow removal!"... See request in the video😳 HTML: https://t.co/yLpXUD6gKq @whale @round @zeldman @brad_frost”

Twitter
Show threadJesse SkinnerMar 29, 2023
@squareflair Yeah it's definitely good at that boilerplate kind of stuff too.
Show threadAlan Houser ☻␆Mar 29, 2023

@JesseSkinner I was hoping for OCR and/or photo identification, which would be amazing.

But not yet I guess.
(linked an image of a headset...) #AI #ChatGPT #chatgtp4 #ocr

Show threadJesse SkinnerMar 29, 2023
@squareflair yeah it still can't load URLs so it just guesses stuff from the URL itself. Apparently it will be able to handle images soon, but that is still in beta.
Show threadAlan Houser ☻␆Mar 30, 2023
@JesseSkinner I want to work there. It seems like the equivalent of Google's early days of search.
Show threadI use Mar 29, 2023
@JesseSkinner I tried asking ChatGPT to write my some date time code in JavaScript and it just couldn’t do it. It made my native API functions and everything. I also tried using Bing AI to help me solve some issues I’m working through with module federation (sharing MUI theme context) and it just copy pasted the top Stack overflow post which just said to abstract the theme into a separate module. I told it that I couldn’t do this and it responded with the same answer
Show threadI use Mar 29, 2023
@JesseSkinner the day that AI can solve date time issues or how to take into account the context for which something needs to be built is really when it will become a tool that I will use on a daily basis.
Show threadJesse SkinnerMar 29, 2023

@asteroidrainfall Yep. Somebody needs to keep an eye on the AI to make sure it's doing what it's supposed to, for a long time to come, probably forever.

I think the most useful thing it can do is act as a kind of broad search engine, where you can describe your vague use case and get some higher level ideas about what to do. It has suggested libraries for me I didn't know existed.

Show threadn3wjackMar 29, 2023
@JesseSkinner @asteroidrainfall that's it basically, a new way to search information. But it doesn't understand any of it, or reasons about it.
It just strings stuff together it "read" about in coherent language that seem to fit together.
But sometimes it doesn't fit together at all.
Show threadJesse SkinnerMar 29, 2023
@n3wjack @asteroidrainfall yeah that's right. I know some humans that are good at that too ;)