Erik-JanToday in our Data Science reading group @utrechtuniversity we talked about this paper by Jessica Hullman et al.: https://arxiv.org/abs/2203.06498v6
It's really great & well-written; as @daob mentioned in the meeting, an incredible amount of work went into Table 1, which compares pitfalls of the scientific process in the social psychology and machine learning fields:
The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning
Recent arguments that machine learning (ML) is facing a reproducibility and replication crisis suggest that some published claims in ML research cannot be taken at face value. These concerns inspire analogies to the replication crisis affecting the social and medical sciences. They also inspire calls for greater integration of statistical approaches to causal inference and predictive modeling. A deeper understanding of what reproducibility concerns in research in supervised ML have in common with the replication crisis in experimental science can put the new concerns in perspective, and help researchers avoid "the worst of both worlds," where ML researchers begin borrowing methodologies from explanatory modeling without understanding their limitations and vice versa. We contribute a comparative analysis of concerns about inductive learning that arise in causal attribution as exemplified in psychology versus predictive modeling as exemplified in ML. We identify themes that re-occur in reform discussions, like overreliance on asymptotic theory and non-credible beliefs about real-world data generating processes. We argue that in both fields, claims from learning are implied to generalize outside the specific environment studied (e.g., the input dataset or subject sample, modeling implementation, etc.) but are often impossible to refute due to forms of underspecification. In particular, many errors being acknowledged in ML expose cracks in long-held beliefs that optimizing predictive accuracy using huge datasets absolves one from having to make assumptions about the underlying data generating process. We discuss risks and opportunities that arise as both fields attempt to resolve concerns about methods.
Dgar
Wladimir Mufty
Casper Albers ✅
Kit Yates