I know quote tweets aren't a thing here but I feel strongly about this, so here goes!
It makes me sad to see the below take pop up so often. Because they're trained directly on data and adopt the perspective of those data, the epistemology of ML/AI/LLM is, IMO, perfectly aligned with the situated knowledges perspective. Feminists, this is actually *our* moment to shine! We can do so much with these methods! We can absolutely use these methods in a way that aligns with a #feminist epistemology.
@LauraNelson I don't think LLMs are able to do this, though, because they require so much data to train from scratch? Maybe there's a way to fine-tune towards the situated knowledges perspective, though.
The big question is if there's a way that pre-training also has embedded biases, which does seem to be the case.
@LauraNelson @TedUnderwood @emilymbender If we just try to point to the Common Crawl or The Pile, the only way people have characterized this is through the lens of bias. While that gives us some information, I don't think it's enough to call it a "viewpoint."
Also, cf. part I of our paper https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4217148
@alex @LauraNelson @TedUnderwood My take on the OP isn't that LLMs can legitimately be said to have a "view from nowhere", but rather that the people who think they have knowledge at all* are the types who think that "view from nowhere" is possible and that surely massive scale would deliver it.
*as opposed to information about word form distribution in some specific dataset, which is what they have.
@alex @LauraNelson @TedUnderwood @emilymbender and to be clear in the sequel I talked about how I think viewing LLMs as having knowledge is just going to exacerbate cultural and epistemological hegemony problems we already have with large scale information management
it takes the problem of the texas school board deciding what goes in high school textbooks across the country into a more global and more insidious issue, basically
Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, B...
@LauraNelson okay so getting back to your point I both agree with the goal of a true epistemic pluralism in how we handle large scale information management, let's call it a kind of cyborg epistemology, but I also kinda agree with folks like @alex and @emilymbender that I don't think LLMs are going to be good tools for the job by their structure
I don't even think LLMs are useless here, but I do think that the model of "ask a question, get an answer" is probably a dead-end in terms of what the tools we need actually should look like
@LauraNelson @alex @emilymbender and to elaborate slightly further I guess what I kind of mean is that I think whatever tools are needed to build the knowledge base for intentionally situated LLMs are probably going to be more useful than an LLM trained on that corpus itself, y'know?
I think the final interface to this corpus can be far more interactive and illustrative and show us connections better than a chatgpt interface could, if you feel me?
@LauraNelson it's cool to know that we're closer to the same vision here than not
I really don't know practically where to start, though, beyond vague pictures in my head of queries returning big knowledge graphs you can interact with somehow
something kind of like "proof objects" in automated theorem proving, where the proof can be calculated algorithmically but you can inspect the steps that lead to the conclusion and how the final artifact is built
Question answering models can use rich knowledge sources -- up to one hundred retrieved passages and parametric knowledge in the large-scale language model (LM). Prior work assumes information in such knowledge sources is consistent with each other, paying little attention to how models blend information stored in their LM parameters with that from retrieved evidence documents. In this paper, we simulate knowledge conflicts (i.e., where parametric knowledge suggests one answer and different passages suggest different answers) and examine model behaviors. We find retrieval performance heavily impacts which sources models rely on, and current models mostly rely on non-parametric knowledge in their best-performing settings. We discover a troubling trend that contradictions among knowledge sources affect model confidence only marginally. To address this issue, we present a new calibration study, where models are discouraged from presenting any single answer when presented with multiple conflicting answer candidates in retrieved evidences.
@maria_antoniak @left_adjoint This is an interesting solution: "To address this issue, we present a new calibration study, where models are discouraged from presenting any single answer when presented with multiple conflicting answer candidates in retrieved evidences."
Providing multiple answers, and explaining why. I hope more people are exploring that option.
@LauraNelson e.g., here's my phd advisor's paper on linguistic change in online communities -- really, beer-review forums. https://www.cs.cornell.edu/~cristian/Linguistic_change_files/linguistic_change_lifecycle.pdf
here, language models are bespoke and bigram-level, used to model month-by-month patterns of language used by this group of people who reviewed craft beers on the internet. baked into the research question is the intuition that these patterns change over time.
@LauraNelson the cool finding is that as you write more beer reviews, you might start to settle into your habitual ways of writing these reviews, even as the community moves on without you.
(side note, i feel maybe a variant or reversal of this about my current mastodon instance? anyway, not important.)
but maybe there's something here? i.e., in the vein of, "here we map the gradual ossification of bureaucratic-speak in this organization, showing the development of linguistic devices that help it shirk responsibility for the various crises it's implicated in."
@LauraNelson second, the "LMs have knowledge" trend is totally bizarre to me, given the "community norm" interpretation that i was socialized into. maybe that's why people make distinctions between NLP and comp-soc-sci/text-as-data.
(i mentioned both interpretations to an anthro professor recently, who seemed a bit weirded out by the idea of writing down a unified interpretation of "what it means to find patterns in text", so...that too.)
Laura K. Nelson
Alex Hanna
Prof. Emily M. Bender(she/her)
Ted Underwood
lynn / clarissa
Nabeel Siddiqui
katy ilonka gero
Maria Antoniak
THIS ACCOUNT HAS MOVED
kavi 🚱🌉
justine