Dan Goodman
If you take a paper and get an LLM to review it, then get the LLM to rewrite the paper and write a reply to the reviewer, and then repeat, what happens? Convergence to something better? Cycles of arbitrary change that never converge? Descent into meaningless drivel?
Mar 9 at 9:07pmWeb
Show threadAlbert CardonaMar 9

@neuralreckoning

Loss of what the point of the paper was.

Show threadDan GoodmanMar 9
@albertcardona just like real peer review then? 😉
Show threadAlbert CardonaMar 9

@neuralreckoning

Snark aside, I find peer review helpful, to a point. Human peer review, which is about ideas, vision, concepts, and suffiency of methods and data, not about pretty turns of sentence.

Show threadDan GoodmanMar 10
@albertcardona you're right I'm sorry I just couldn't help myself. The goal was wide open.
Show threadAlbert CardonaMar 10

@neuralreckoning

How's PCI neuro going? I'm interested in the model; a pity though that there are reject decisions, unnecessarily so in my opinion.

I see @brembs and Koen Vervaeke in the board as well.

Show threadStephen RoyleMar 9

@neuralreckoning my prediction is meaningless drivel. Using this as evidence:

https://www.reddit.com/r/ChatGPT/comments/1kbj71z/i_tried_the_create_the_exact_replica_of_this/

Show threadDan GoodmanMar 9
@steveroyle that's magnificent!
Show threadElio CampitelliMar 9
@neuralreckoning If my experiments asking LLMs to fix grammar and typos are anything to go by, I'd go for 2, constant meaningless and arbitrary changes that never converge. Maybe if the prompts included specific instructions to accept the paper in the reviewer step, it would randomly stop at some point if by chance the LLM chose the stop token. If allowed to continue, I'd imagine that the text would slowly drift via partial synonyms into something completely different.,