Winter 2025 - Researcher positions in AI and machine learning — FCAI

Lessons for Theory from Scientific Domains Where Evidence is Sparse or Indirect - Computational Brain & Behavior

In many scientific fields, sparseness and indirectness of empirical evidence pose fundamental challenges to theory development. Theories of the evolution of human cognition provide a guiding example, where the targets of study are evolutionary processes that occurred in the ancestors of present-day humans. In many cases, the evidence is both very sparse and very indirect (e.g., archaeological findings regarding anatomical changes that might be related to the evolution of language capabilities); in other cases, the evidence is less sparse but still very indirect (e.g., data on cultural transmission in groups of contemporary humans and non-human primates). From examples of theoretical and empirical work in this domain, we distill five virtuous practices that scientists could aim to satisfy when evidence is sparse or indirect: (i) making assumptions explicit, (ii) making alternative theories explicit, (iii) pursuing computational and formal modelling, (iv) seeking external consistency with theories of related phenomena, and (v) triangulating across different forms and sources of evidence. Thus, rather than inhibiting theory development, sparseness or indirectness of evidence can catalyze it. To the extent that there are continua of sparseness and indirectness that vary across domains and that the principles identified here always apply to some degree, the solutions and advantages proposed here may generalise to other scientific domains.

Posterior accuracy and calibration under misspecification in Bayesian generalized linear models

Generalized linear models (GLMs) are popular for data-analysis in almost all quantitative sciences, but the choice of likelihood family and link function is often difficult. This motivates the search for likelihoods and links that minimize the impact of potential misspecification. We perform a large-scale simulation study on double-bounded and lower-bounded response data where we systematically vary both true and assumed likelihoods and links. In contrast to previous studies, we also study posterior calibration and uncertainty metrics in addition to point-estimate accuracy. Our results indicate that certain likelihoods and links can be remarkably robust to misspecification, performing almost on par with their respective true counterparts. Additionally, normal likelihood models with identity link (i.e., linear regression) often achieve calibration comparable to the more structurally faithful alternatives, at least in the studied scenarios. On the basis of our findings, we provide practical suggestions for robust likelihood and link choices in GLMs.