Marco 🌳 ZoccaOct 12, 2024

Interested in interpretable ML, particularly for LLMs?

eg "causal" interpretability, as in the "OthelloGPT" paper [1]?

Let's connect!

1. https://arxiv.org/abs/2210.13382

#ai #machinelearning #interpretability #interpretableml #mechanisticinterpretability

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see? We investigate this question by applying a variant of the GPT model to the task of predicting legal moves in a simple board game, Othello. Although the network has no a priori knowledge of the game or its rules, we uncover evidence of an emergent nonlinear internal representation of the board state. Interventional experiments indicate this representation can be used to control the output of the network and create "latent saliency maps" that can help explain predictions in human terms.

arXiv.org
Show threadCosima MeyerOct 1, 2024

And while this is true for any modeling approach, I think it's especially relevant as we move to more complex models. In my experience, interpretability is important not only in making models accessible to the end user but also in the development process as we strive to build fair and reliable models that produce the best results.

#interpretableml #xai #datascience #python

Cosima MeyerOct 1, 2024

This book is golden! (and I think I'll have to read it again because it's so full of information!)

For anyone trying to understand their models, Serg Masís' book "Interpretable Machine Learning with #Python" provides the right mix of theory and practical approaches. It has both a high-level and applied perspective, which I really enjoyed, and gives both practitioners and those new to the field a good and illustrative starting point.

#interpretableml

José OramasMar 6, 2024
Great news, this year #AIMLAI will be held in conjunction with #ECMLPKDD 2024. Looking forward to meeting you in Vilnius! #xai #interpretableML #explainability #interpretability #ai #ml @ECMLPKDD @IDLabResearch @imecVlaanderen @UAntwerpen
UKRI AI for Healthcare CentresJul 28, 2023

On Friday 28 July and Sat 29 July
@icmlconf

come and follow presentations by
@PFestor

@alj_jenkins
and
@JoshSouthern13

#interpretableML for health #GenerativeAI #GNN

Ulrich JunkerApr 27, 2023

Interview of #CynthiaRudin about #InterpretableML. She is advocating #interpretable models for high-stake #decisions, which merit the extra training effort. Those #whiteboxes (or should we say #lightboxes?) give us hope in times where business leaders feel tempted to surrender to obscure #darkboxes (sounds more adequate than #blackboxes, doesn’t it?), which escape #accountability and #adjustability!

“Cynthia Rudin Builds AI That Humans Can Understand” | Quanta Magazine

https://flip.it/pmlCdg

Cynthia Rudin Builds AI That Humans Can Understand | Quanta Magazine

Cynthia Rudin wants machine learning models, responsible for increasingly important decisions, to show their work.

Quanta Magazine
Conor O'SullivanJan 23, 2023

My new article in @towardsdatascience

One of the points is that IML bring DS closer to goal of science - understanding our natural world.

This is one of the reasons I’m so attracted to the field. Trying to understand how a model works is 1000x more interesting than simply evaluating it.

#DataScience #MachineLearning #InterpretableML #XAI

No paywall link:
https://towardsdatascience.com/data-science-is-not-science-bb95d783697a?source=friends_link&sk=990dff05efbd4c9369c667f7977a23a7

Data Science Is Not Science - Towards Data Science

How data science differs from science. Bringing it closer to the scientific process can provide more reliable results. For example, by using IML or XAI.

Towards Data Science
José OramasJan 11, 2023

Our #WACV23 paper on the topic of evaluation for model interpretation methods is available online https://openaccess.thecvf.com/content/WACV2023/html/Behzadi-Khormouji_A_Protocol_for_Evaluating_Model_Interpretation_Methods_From_Visual_Explanations_WACV_2023_paper.html

Congrats to @behzadikhormuji for the good work.
#XAI #Interpretability #interpretableml #UAntwerp #imec

WACV 2023 Open Access Repository