Colin Carter 
Daniel BuschekMy first research thread shared here is a recap of our recent #UIST paper. We asked: How can #AI support #writing without trying to correct us or writing for us? This led us to go "Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries". ✍️🤖
We took the writing strategy of "reverse outlining" as a starting point, to explore a new interaction model and AI role: The AI neither replaces the writer, nor "checks" or "corrects" them. It rather generates parallel content to empower users in their self-evaluation. 📝🧐
Key findings from 12 people writing with our prototype: Summaries give an "external" perspective on your writing, helpful to revise paragraph content and scope. People also used them to gain an overview and developed strategies to integrate insights from summaries into the text.
Shiwali Mohan | शिवाली मोहन@DBuschek Interesting use of language models. I have a few questions -
1. Did the language models behave sensibly? I.e., there is some work now that shows that language models don't understand compositionality or abstractions. So I wonder if you saw that in your study.
2. Are their concerns that people would start writing for the AI - i.e, text that AI understands?
@shiwali
Thanks!
1. We didn't evaluate summarization quality much (a short attempt is in the appendix). Practically speaking, it worked well in observations and people's own impressions. I don't remember that people mentioned specific aspects being better/worse in the summaries, such as the ones you mentioned. I think, as long as summaries are not too bad or outright wrong, the key result - providing a kind of second perspective that triggers writers to reflect - probably holds here.
2. We saw some behaviour in this regard, like editing the text to see how it impacts the summary. This might partly be just exploration in the study setting but it could also be an interesting way of testing e.g. if a desired main point "comes across" as such by being picked up in the AI summary. If used this way I'd say it shifts the (user's assumed) role of the AI further towards a proxy reader, not merely a "mirror" of my written content.
This is one of the best usecases of LLMs that I have seen. Thank you!
The utility of the LLM is to drive more introspection and analysis in the human writer. Beautiful!
Thank you for your questions and the kind words! 😊
Will share with my coauthors who did most of the work and are not (yet) on Mastodon.
noah bubenhofer@DBuschek it‘s really amazing that Word is still in use. There have been a lot of better ideas for word processing software in computer history, no chances to get a significant market share. Here is the next brilliant idea… I really hope that we will write differently on our computers in the near future…