Matthew Facciani

@matthewfacciani
750 Followers
462 Following
230 Posts
Social scientist. Postdoc at University of Notre Dame. Studies polarization, misinformation, & media literacy. #SciComm #BiInSci šŸ³ļøā€šŸŒˆā€Øā€ØTo get summaries of my research and be notified when my book is available, sign up for my newsletter:
http://www.matthewfacciani.com

Citations and Trust in LLM Generated Responses

Yifan Ding, Matthew Facciani, Amrit Poudel, Ellen Joyce, Salvador Aguinaga, Balaji Veeramani, Sanmitra Bhattacharya, Tim Weninger
https://arxiv.org/abs/2501.01303 https://arxiv.org/pdf/2501.01303 https://arxiv.org/html/2501.01303

arXiv:2501.01303v1 Announce Type: new
Abstract: Question answering systems are rapidly advancing, but their opaque nature may impact user trust. We explored trust through an anti-monitoring framework, where trust is predicted to be correlated with presence of citations and inversely related to checking citations. We tested this hypothesis with a live question-answering experiment that presented text responses generated using a commercial Chatbot along with varying citations (zero, one, or five), both relevant and random, and recorded if participants checked the citations and their self-reported trust in the generated responses. We found a significant increase in trust when citations were present, a result that held true even when the citations were random; we also found a significant decrease in trust when participants checked the citations. These results highlight the importance of citations in enhancing trust in AI-generated content.

Citations and Trust in LLM Generated Responses

Question answering systems are rapidly advancing, but their opaque nature may impact user trust. We explored trust through an anti-monitoring framework, where trust is predicted to be correlated with presence of citations and inversely related to checking citations. We tested this hypothesis with a live question-answering experiment that presented text responses generated using a commercial Chatbot along with varying citations (zero, one, or five), both relevant and random, and recorded if participants checked the citations and their self-reported trust in the generated responses. We found a significant increase in trust when citations were present, a result that held true even when the citations were random; we also found a significant decrease in trust when participants checked the citations. These results highlight the importance of citations in enhancing trust in AI-generated content.

arXiv.org
Here is a quick political knowledge test where you can compare your score to the average American. Only 12% of Americans get all six questions correct!
https://www.pewresearch.org/politics/quiz/what-do-you-know-about-the-u-s-government/
What do you know about the U.S. government?

Test your civics knowledge by taking our short 6-question quiz.

Pew Research Center
A new study found that in the year following Elon Musk's acquisition of Twitter (now X), users reported their feeds becoming more negative and featuring less reliable content. They also said they were less likely to use Twitter/X.
https://osf.io/preprints/psyarxiv/acbwg
OSF

New study finds that fact-checks framed as "confirmations" (e.g., "It is TRUE that...") lead to higher engagement compared to "refutation" frames (e.g., "It is FALSE that..."). This pattern was consistent across four countries and the "confirmation" fact-checks also reduced self-reported negative emotions related to polarization.
https://www.nature.com/articles/s41598-024-53337-0?fromPaywallRec=true #MisinfoResearch
Framing fact-checks as a ā€œconfirmationā€ increases engagement with corrections of misinformation: a four-country study - Scientific Reports

Previous research has extensively investigated why users spread misinformation online, while less attention has been given to the motivations behind sharing fact-checks. This article reports a four-country survey experiment assessing the influence of confirmation and refutation frames on engagement with online fact-checks. Respondents randomly received semantically identical content, either affirming accurate information (ā€œIt is TRUE that pā€) or refuting misinformation (ā€œIt is FALSE that not pā€). Despite semantic equivalence, confirmation frames elicit higher engagement rates than refutation frames. Additionally, confirmation frames reduce self-reported negative emotions related to polarization. These findings are crucial for designing policy interventions aiming to amplify fact-check exposure and reduce affective polarization, particularly in critical areas such as health-related misinformation and harmful speech.

Nature
People who played our 5-minute online game significantly increased their ability to detect misinformation! Our media literacy game, Gali Fakta, was designed for an Indonesian audience. Our open-access article is available here:
https://misinforeview.hks.harvard.edu/article/playing-gali-fakta-inoculates-indonesian-participants-against-false-information/
#MisinfoResearch
Playing Gali Fakta inoculates Indonesian participants against false information | HKS Misinformation Review

Although prebunking games have shown promise in Western and English-speaking contexts, there is a notable lack of research on such interventions in countries of the Global South. In response to this gap, we developed Gali Fakta, a new kind of media literacy game specifically tailored for an Indonesian audience. Our findings indicate that participants who

Misinformation Review

🚨 New Publication Alert! 🚨
Our study found that people with homogenous networks (all friends & family share the same political views) are more likely to believe and share fake news and political rumors. We also found lower cognitive reflection scores predicted susceptibility to fake news, but not rumors! #MisinfoResearch #science

https://sciendo.com/article/10.21307/connections-2019.044

Personal Network Composition and Cognitive Reflection Predict Susceptibility to Different Types of Misinformation

Despite a rapid increase in research on the underpinnings of misinformation susceptibility, scholars still disagree about the relative impacts of social context and individual cognitive factors. We argue that cognitive reflection and identity-based network homogeneity may have unique influences on different types of misinformation. Specifically, identity-based network homogeneity predicts bias that is related to any type of identity-based information (i.e., political rumors), and cognitive reflection is more tailored toward truth discernment (i.e., fake news headlines). We conducted our study using an online sample (N = 214) split evenly between Democrats and Republicans and collected data on personal network composition, cognitive reflection, as well as susceptibility, sentiments, and sharing behavior in relation to political rumors and misinformation, respectively. Results demonstrate that where network homogeneity predicts belief and sharing in both political rumors and fake news headlines, cognitive reflection only predicts belief and sharing of fake news headlines. Social vs. cognitive factors for predicting different types of misinformation are discussed.

Sciendo

"Gigafact is a nonprofit that equips newsrooms to counter misinformation and protect the democratic process."

And we're proud to be partnering with #Gigafact, another organization focused on filling a particular gap. Here, layout-friendly drop-in debunking content, consistently treated and conveniently to hand for editors and reporters.

We're sharing our expertise on #ClimateMyths with Gigafact.

The tight 150-word budget makes each a puzzle for authors to solve.

https://skepticalscience.com/fact-brief-breath.html?utm-source=mastodon&utm-campaign=socialnetworks&utm-term=sks

Fact Brief - Does breathing contribute to CO2 buildup in the atmosphere?

Skeptical Science is partnering with Gigafact to produce fact briefs — bite-sized fact checks of trending claims. This fact brief was written by Sue Bin Park in collaboration with members from our Skeptical Science team. You can submit claims you think need checking via the tipline.

Skeptical Science

I’ll be more active on my Threads account where I’ll continue to share scientific articles about misinformation and media literacy. You can follow me on Mastodon here via the Fediverse @[email protected]

#MisinfoResearch #science #socialmedia #medialiteracy #misinformation #scicomm

Urban farming is a favorite #sustainability tactic – but research shows it’s 6x more carbon intensive than farming on a farm. How to reduce the carbon footprint?
1. Use recycled materials
2. Grow carbon-intensive crops
3. Build for the long-term

https://theconversation.com/urban-agriculture-isnt-as-climate-friendly-as-it-seems-but-these-best-practices-can-transform-gardens-and-city-farms-221537

Urban agriculture isn’t as climate-friendly as it seems, but these best practices can transform gardens and city farms

A study of dozens of city gardens and urban farms across the US and Europe found several ways to boost their benefits, not just for their neighborhoods, but for the planet.

The Conversation

My edited book with meatspacepress is out!

This book considers how market (and state) power in the tech sector is rebuilding around the internet’s material infrastructure - with chapters by @Mer__edith Joan Donovan @mikarv fiekejansen nielstenoever gurshabadgrover & many others