I used ChatGPT to help me write a peer review. It didn't help at all. There is a big difference between being really cool and being a useful tool. This experience provides some lessons for how we should evaluate LLMs and other new AI. https://freedom-to-tinker.com/2023/03/08/can-chatgpt-and-its-successors-go-from-cool-to-tool/
Can ChatGPT—and its successors—go from cool to tool?

Anyone reading Freedom to Tinker has seen examples of ChatGPT doing cool things.  One of my favorites is its amazing answer to this prompt: “write a

Freedom to Tinker
@msalganik Are you concerned about ChatGPT using the unpublished paper in future responses? I've heard concerns of, e.g., doctors putting patient info in to get it to write up reports. I don't know if there's a privacy/IP issue here.