I know quote tweets aren't a thing here but I feel strongly about this, so here goes!
It makes me sad to see the below take pop up so often. Because they're trained directly on data and adopt the perspective of those data, the epistemology of ML/AI/LLM is, IMO, perfectly aligned with the situated knowledges perspective. Feminists, this is actually *our* moment to shine! We can do so much with these methods! We can absolutely use these methods in a way that aligns with a #feminist epistemology.
@LauraNelson okay so getting back to your point I both agree with the goal of a true epistemic pluralism in how we handle large scale information management, let's call it a kind of cyborg epistemology, but I also kinda agree with folks like @alex and @emilymbender that I don't think LLMs are going to be good tools for the job by their structure
I don't even think LLMs are useless here, but I do think that the model of "ask a question, get an answer" is probably a dead-end in terms of what the tools we need actually should look like
@LauraNelson @alex @emilymbender and to elaborate slightly further I guess what I kind of mean is that I think whatever tools are needed to build the knowledge base for intentionally situated LLMs are probably going to be more useful than an LLM trained on that corpus itself, y'know?
I think the final interface to this corpus can be far more interactive and illustrative and show us connections better than a chatgpt interface could, if you feel me?
@LauraNelson it's cool to know that we're closer to the same vision here than not
I really don't know practically where to start, though, beyond vague pictures in my head of queries returning big knowledge graphs you can interact with somehow
something kind of like "proof objects" in automated theorem proving, where the proof can be calculated algorithmically but you can inspect the steps that lead to the conclusion and how the final artifact is built
Question answering models can use rich knowledge sources -- up to one hundred retrieved passages and parametric knowledge in the large-scale language model (LM). Prior work assumes information in such knowledge sources is consistent with each other, paying little attention to how models blend information stored in their LM parameters with that from retrieved evidence documents. In this paper, we simulate knowledge conflicts (i.e., where parametric knowledge suggests one answer and different passages suggest different answers) and examine model behaviors. We find retrieval performance heavily impacts which sources models rely on, and current models mostly rely on non-parametric knowledge in their best-performing settings. We discover a troubling trend that contradictions among knowledge sources affect model confidence only marginally. To address this issue, we present a new calibration study, where models are discouraged from presenting any single answer when presented with multiple conflicting answer candidates in retrieved evidences.
@maria_antoniak @left_adjoint This is an interesting solution: "To address this issue, we present a new calibration study, where models are discouraged from presenting any single answer when presented with multiple conflicting answer candidates in retrieved evidences."
Providing multiple answers, and explaining why. I hope more people are exploring that option.
Laura K. Nelson
lynn / clarissa
Maria Antoniak