Florian Haas17h ago

Those of you who work in an organisation that makes heavy use of GenAI coding assistants: how did you educate your developers that if they automate sweeping changes to user-facing code, they MUST also automate the corresponding sweeping changes to user-facing documentation?

Or have you identified a strong correlation between embracing GenAI coding assistants and not giving a flying fuck about user-facing docs?

Show threadChristoph Petrausch17h ago
@xahteiwi Using Docs to tell them to update User Facing docs (agent.md or development.md). The only thing GenAI Agent do better than the average human developer is reading markdown docs and following it.
Show threadFlorian Haas17h ago
@hikhvar Okay, this approach works when we're talking about docs that (a) live in the same repo as the code being modified, and (b) use Markdown because that's the least error-prone format for the LLM to ingest. Fair summary?
Show threadChristoph Petrausch17h ago

@xahteiwi It works better for markdown, but from what I see, the agent I'm working mostly with (claude) is also capable of editing other docs format that are not Office formats like DocX. If your docs live in a wiki like e.g. Confluence or MediaWiki the agent could do it if there is a sufficient capable MCP. Also at least claude is able to work with multiple git repositories.

The funny thing is, the better documented and accessible the processes are the better the agents are.

Show threadFlorian Haas17h ago
@hikhvar Okay. How well is that working for you with GUIs and screenshots?
Show threadChristoph Petrausch16h ago
@xahteiwi Ok, I have not done this in practice. In theory the agent could ask a the human in the loop for an updated screenshot.
Show threadFlorian Haas15h ago

@hikhvar Consider that screenshots are typically cropped, or include highlights, or blur parts to hide information that is private or uninteresting, etc. So, automating such screenshot updates is an "interesting" task.

Consider further that you may have hundreds of screenshots that are all affected by a change of one line of CSS, rendering your "let's ask a human" approach impractical.

And sadly, not thinking about this sort of thing is exactly what I mean by not giving a fuck about docs.

Show threadChristoph Petrausch
@xahteiwi was that different when people created the code? If changing CSS is a problem in the process, you must instruct everyone to not change it. In the best case you have verified this via a CI test in your pipeline. If so, the agent will see this signal similarly to your new hire will learn about it.
9:52amWeb
Show threadFlorian Haas14h ago
@hikhvar Well of course it was the same when people created the code, but they typically wouldn't make nearly as sweeping and impactful changes.
Show threadChristoph Petrausch13h ago

@xahteiwi well depends on the humans you are working with. I have seen both experienced (I'm an expert I know what I'm doing) and inexperienced (How bad can a small CSS change be?) do sweeping and impactful changes. I have been both.

With Coding agents you need better documentation or testing do guide what changes are acceptable, and what changes are unacceptable. The irony is, that will help humans as well.