1/
Commentary on #GenAI / #SyntheticTextExtrudingMachines use within #scientific reviews [1]
Relevance:
"The stakes are high.
#SystematicReviews and other types of evidence synthesis inform clinical practice, public-health guidance and policy decisions that affect entire populations. Errors could give false hope to patients or lead health systems to waste money on ineffective or unsafe interventions"
Among the points:
- privately-owned tools vs independence from industry
- tools miss context
2/
The commentary on the first point:
"most of the tools available were developed by #private companies. This is problematic for #reviews that evaluate drugs and medical devices, because these need to be #independent of industry. What’s more, few AI models are open source, with most relying on opaque, proprietary ‘#BlackBox’ processes. This means there’s no way to examine whether a tool might disproportionately include trials with results favourable to one drug company"
3/
On the second point (to me, perhaps even more fundamental, as truly understanding #context is a prerequisite for intelligence):
"conducting #SystematicReviews is not a purely computational task. Human specialists are needed to define meaningful review questions, evaluate relevance, interpret results and understand clinical or policy implications. Context and subjective nuance are seldom well-represented in AI models’ training data, and the" #GenAI models do "fabricate information" [1]
4/
(Note: the vastly general term "#AI" is here used, as often, instead of the quite more specific intended meaning: #GenAI / #SyntheticTextExtrudingMachines)
[1] Sarkar, R., 2026. Why AI can’t be trusted to write scientific reviews. Nature 653 (8116), 983–983. https://doi.org/10.1038/d41586-026-01616-3
Daniele de Rigo