I used ChatGPT to help me write a peer review. It didn't help at all. There is a big difference between being really cool and being a useful tool. This experience provides some lessons for how we should evaluate LLMs and other new AI. https://freedom-to-tinker.com/2023/03/08/can-chatgpt-and-its-successors-go-from-cool-to-tool/
at Apple University. Plugged in at UC Berkeley's AFOG and Social Science Matrix.