Refactoring and (A)I
Reading Time: 3 minutesFor two days I have been migrating my blog from Hugo to Eleventy via the markdown pages, and the photos via Ghost Export for WordPress. In the process I achieved a goal, and then AI broke things, so I achieved them again, and then AI broke them again.
The favourite thing for Ai, and Gemini, in particular to break was the logic that took the markdown titles, matched them to a json file, and then helped 11ty marry the photo with the correct post. When it worked it was brilliant, but when it broke I would spend half a day trying to fix it. By the third time this happened I decided to ask Gemini to help me write a script to hard code the image path straight into markdown pages.
That step was a resounding success. The logic is simple. AI is great for a simple question, but as soon as you start a russian doll/inception of steps the context window, or scope window, to use a programming term gets confused.
Scope Windows and AI
I saw the term context window used, but I'd like to experiment with scope and AI. Specifically I'd like to document how it behaved when I saw that the eleventy.js file was getting too long, and I asked Gemini to refactor it. The first mistake was to accept that Gemini could do this in just two steps. Step 1, filters.js, and then step 2 for another set of logic.
Now, in an ideal world, AI would know what it did, see the code, determine what each part does, and tidy everything up into neat refactored niches. It didn't do that. Remember that mention of the image logic being lost three times? The third time was due to refactoring.
To be specific, Gemini took the complex logic for matching thumbnails to posts and replaced it with two lines of irrelevant code, that I noticed eventually, as I tried to debug why the refactoring had failed.
Gemini and Git
When I noticed that Gemini has issues with keeping within the scope of a question I decided to use git. Every time Gemini provided me with a solution that worked I'd commit it. The logic is that if it breaks something I can tell it "This is how things were, and this is how they are now. I'd like to fix things".
Why Entrust Git rather than AI?
In my experience AI is fantastic as spitting out code, and muddling contexts, but very bad at answering "Can you remind me which command we used to do that", whereas git is fantastic at that, if you're efficient about git commit comments that mean something to you.
The AI Negative Feedback Loop
If AI makes a mistake, either because your instructions were not clear or because it got confused, asking another question will muddle it even further. Imagine an AI chat as a sand pit by the sea. The more you chat, the deeper the hole you dig. Eventually you make a mistake, and the tidal waters begin to flow into your sandpit. Within seconds that chat is swamped and unusable. With AI it's good to have a bug out bag ready. "Now that this context is hallucinating let's move to a new one".
Git and a Fresh Chat
If you're working on an eleventy blog and you're adding functionality, one step at a time, it's good to add bookmarks in the form of git commits. You use Gemini, or your favourite LLM to write the function, and once it works you bookmark it. You then write the next, and bookmark that. Eventually, when the LLM, or you, makes a mistake, and the LLM hallucinates or starts muddling scopes, you just start with a clean slate, helped by git, and continue moving forward, with a minimum of hassle.
AI driven Refactoring Lessons Learned
Next time I refactor I won't ask "can you refactor this long eleventy.js file?" I will break into stages. I like that Gemini refactored the functions into two files based on function type. I made the mistake of not checking that it preserves the logic of the function while refactoring. Be explicit.
If Logic Works, Embed It
With the thumbnails and featured images logic I assumed that once the logic worked, it would be permanent. It isn't. It can easily change. When I told Gemini to use the logic, and move the image paths directly into the markdown files I simplified my life. Complex logic could be removed from eleventy, and posts could go with a standardised approach to displaying images. Simplification works.
And Finally
Use git to help you keep track of changes. AI is not a search engine, so don't rely on it to remember a solution it provided for you. It will come up with a new one. Be as concise as possible and keep the context window small. The larger it is, the more room there is for mistakes.
And finally, AI might be logical and rational but it doesn't have an understanding of what it is doing. To use the Short Circuit analogy "A machine doesn't have emotions, or feel, it only runs as programmed". AI is Not Number five. AI is a tool. That's why it's good to practice with using it properly.
#11ty #AI #context #eleventy #javascript #js #refactoring
Marcel
Dev Leader
Clare Sudbery
InfoQ
Leanpub
Jack Linke 🦄
richard