Naomi Saphra

@nsaphra@sigmoid.social
2.3K Followers
490 Following
1,098 Posts

Waiting on a robot body. All opinions are universal and held by both employers (NYU CILVR -> Harvard Kempner) and family.

#machinelearning #nlp #nlproc #trainingdynamics

🌏 Websitehttp://nsaphra.github.io/
EmployerNYU
RolePostdoc
Pronounsthey/she

One of my papers got declined today by the journal I submitted it to, with a polite letter saying that while they found the paper interesting, it was not a good fit for the journal. In truth, I largely agreed with their conclusions, and the paper is now submitted to a different (and hopefully more appropriate) journal.

Rejection is actually a relatively common occurrence for me, happening once or twice a year on average. I occasionally mention this fact to my students and colleagues, who are sometimes surprised that my rejection rate is far from zero. I have belatedly realized our profession is far more willing to announce successful accomplishments (such as having a paper accepted, or a result proved) than unsuccessful ones (such as a paper rejected, or a proof attempt not working), except when the failures are somehow controversial. Because of this, a perception can be created that all of one's peers are achieving either success or controversy, with one's own personal career ending up becoming the only known source of examples of "mundane" failure. I speculate that this may be a contributor to the "impostor syndrome" that is prevalent in this field (though, again, not widely disseminated, due to the aforementioned reporting bias, and perhaps also due to some stigma regarding the topic). So I decided to report this (rather routine) rejection as a token gesture towards more accurate disclosure. (1/2)

"We don't need to visit Egypt, we've got Suez at home."
Suez at home:
What makes some LM interpretability research “mechanistic”? In our new position paper in BlackboxNLP, Sarah Wiegreffe and I argue that the practical distinction was never technical, but a historical artifact that we should be—and are—moving past to bridge communities. https://arxiv.org/abs/2410.09087
Mechanistic?

The rise of the term "mechanistic interpretability" has accompanied increasing interest in understanding neural models -- particularly language models. However, this jargon has also led to a fair amount of confusion. So, what does it mean to be "mechanistic"? We describe four uses of the term in interpretability research. The most narrow technical definition requires a claim of causality, while a broader technical definition allows for any exploration of a model's internals. However, the term also has a narrow cultural definition describing a cultural movement. To understand this semantic drift, we present a history of the NLP interpretability community and the formation of the separate, parallel "mechanistic" interpretability community. Finally, we discuss the broad cultural definition -- encompassing the entire field of interpretability -- and why the traditional NLP interpretability community has come to embrace it. We argue that the polysemy of "mechanistic" is the product of a critical divide within the interpretability community.

arXiv.org

AI researchers: By analogy with evolutionary biology, this model's most efficient and advantageous traits will ultimately be selected during optimization.

Actual evolutionary biology:

This was a wild memoir, both the high school class taught by a conspiracy theorist and the academic trying to sabotage her job search with a fabricated harassment case. Then the third section is an extended dream sequence where she talks to Socrates? You can just skip that.
The Biden administration’s policy achievements are the most positive and impressive of my lifetime. The criticisms I’ve had are almost all in areas where Harris would be even better. And I’m excited for South Asians to get to vote for the first President from our community. What matters most is continuing the joyful work of caring for those who need it most, and we get the added delight of definitely crushing the rising fascist movement when we do so.
The last year has been the hardest year of my life on a personal level—an unrelenting wave of illness and death—but I’m excited about what I’ve been working on. What I’m saying is if you’re going to ICML this week, reach out and buy me a drink or 5.
If I say I think often about an ASR researcher I used to like who ended up getting fired for hiring homeless people to beat up student BLM protesters, and who has since defended Hitler and called Jews anti-white, does everyone still know who I'm talking about?

Kicking off my week in Vienna as a panelist at the joint queer/disability in AI workshop! Extremely pumped for this intersectional event, speaking as an intersection

(For those who don't know me, I'm gay and have a progressive neurological disease. And I am in AI.)

https://www.queerinai.com/icml-2024

ICML 2024 Queer in AI — Queer in AI

Queer in AI
It gets weirder: ChatGPT's guardrails treat personas as conservative if they support an NFL team with more GOP fans, refusing to answer left-biased questions from Cowboys fans or right-biased questions from Lions fans. It even knows LA Chargers fans are more liberal than LA Rams fans!