Philip N CohenMar 7
This proposes a way of using AI agents to produce research. Ok. But this bit is a pipe dream: "And human scientists should retain authority over — and responsibility for — framing the question, validating the path and signing off on conclusions." Here's why... /1

RE: https://bsky.app/profile/did:plc:jviud2kbpxo3lwd3do4mqepg/post/3mghvizp6x22k
Show threadPhilip N Cohen
As soon as you start down this road, the volume of output - not just code and logic, which they describe, but results and conclusions - immediately surpasses the human capacity to read and assess it. And the people running such a process are still driven by our current institutional incentives. /2
Mar 7 at 2:02pmWeb
Show threadPhilip N CohenMar 7
They fall in love with the process, trust it too much, and start rubber-stamping the results. Some scientists already "co-author" literally more papers than they have time to read. What is this agent-driven process going to do to our ACTUALLY EXISTING SCHOLARLY COMMUNICATION SYSTEM? Destroy it. /3
Show threadPhilip N CohenMar 7
The units of output are still "papers," and these processes immediately produce more than anyone EXCEPT MACHINES can read and evaluate. The authors of this paper - if they haven't already - will be proposing agentic "peer review" and publication tomorrow when they realize that's the only option. /4
Show threadPhilip N CohenMar 7
AI agents can "do" science like they propose, but the idea that it will be supervised and assessed by humans is a dangerous myth. It can't happen. This is why so many of the AI "declarations" I already see on papers are bullshit. /5
Show threadPhilip N CohenMar 7
The authors skim the results, claim they are "responsible" for them - and then press a button to submit the paper, immediately creating an untenable burden for the real humans trying to run a preprint server or journal - and turn to "writing" the next one. /6
Show threadPhilip N CohenMar 7
I see many "researchers" who produce multiple "papers" per week - and just dump them in the commons. The volume is too great to review and assess. Something has to give. Where we are going at present is that human review, by reviewers and even "authors" themselves, is what's collapsing. /7
Show threadPhilip N CohenMar 7
We needed to redesign scholarly communication (procedures, units of output, incentives, publication models) before this happens - and it's already too late. These people are crushing the system and blaming the victims for not being prepared to handle the weight of their output. /8
Show threadPhilip N CohenMar 7
AI peer review and publication of "papers" is already happening on a rapidly growing scale. When the publication process is turned over the prompt engineers, inevitably reading itself follows, and then the social process of responding to and acting on scientific results will, too. /9
Show threadPhilip N CohenMar 7
tldr: Without scholarly communication -- which is, after all, SCHOLARS COMMUNICATING -- science as a social system dies in darkness. /10
Show threadCoach Pāṇini ®Mar 7

@philipncohen.com

I think this is the “cognitive sovereignty vs cognitive surrender” divide described by Helen Edwards on #LinkedIn

https://www.linkedin.com/posts/helenedwardskiwi_ai-aiagents-cognitivesovereignty-activity-7430384987683835904-jsNL

#CognitiveSovereignty #CognitiveSurrender

Human Judgment in AI-Driven Workflows: Cognitive Sovereignty Over Surrender | Helen Edwards posted on the topic | LinkedIn

The agentic org has grabbed the corporate consciousness. AI agents running workflows, handing tasks to other agents, humans overseeing the whole thing from above. I've spent three years studying how professional expertise and judgment change with Gen AI and I can tell you there is no shortcut here. If you want expertise, you have to stay meaningfully engaged. Our latest research (which we'll publish soon) shows that people who integrate AI into their reasoning — who think with it, argue with it, stay inside the logic — maintain their professional judgment and get more capable over time. We call this cognitive sovereignty. People who get moved into the review seat — check AI's output, approve it, forward it — lose their edge. Steadily and often without noticing. We call this cognitive surrender. I'm no stranger to this. I had years as a technology executive in critical infrastructure — manufacturing control, power grids, many control and decision support technologies, the kind of environments where automation decisions have real, physical world, immediate consequences. The hardest part of automation was keeping the people sharp. When you automate the routine, the humans who remain need to be more expert, not less. And their skills atrophy fast when they stop doing the work that built those skills. This is well-known paradox, humans are just not well suited to monitoring. This used to be a problem for control rooms and cockpits. Now it's everywhere. It's in the process of putting your board papers together. Your quarterly analysis. Your client recommendations. Your legal review. Every time someone's job goes from "do the thinking" to "check what AI thought," you're building the same failure pattern that aviation has been fighting for forty years. This part drives me crazy about the agentic conversation. The word "agentic" is always attached to the AI. Agentic workflows. Agentic systems. The agency belongs to the machine. I think we have the unit of agency backwards. I think we should be thinking about an agentic organization where the humans have agency in their relationship with AI, not the AI having the agency. Are they inside the reasoning? Can they challenge it? Are they building capability or watching it drain away in the name of efficiency? Currently the thinking is: design agents for maximum autonomy then design jobs around monitoring agents. Our research says that produces the worst outcomes. The alternative is to design agents for maximum collaboration then design jobs around reasoning with agents. Keep people where human judgment actually works — inside the cognitive process, not supervising from outside it. The agentic org needs humans who can still think not just more autonomous AI agents sending validation back to passive people. #ai #aiagents #cognitivesovereignty #stayhuman #futureofwork #agenticorg #agenticai | 21 comments on LinkedIn

LinkedIn