Significance tests, p-values, and falsificationism

This thread takes its inspiration from the recent discussions in social science and statistics about significance tests, what they’re good for, whether p-values should be banned, and what all of that has to do with general scientific methodology, particularly the Popperian one called falsificationism. See this Twitter thread for a random place to jump into the discussion. That place, however, was where @f2harrell suggested we create a thread over here. I’ll start this thread off, if I may, with...

Время выполнения заданий в немодерируемых UX тестах: как измерить и проверить?

Каждый исследователь на старте, когда впервые сталкивается со статистикой Всем привет! Меня зовут Ульяна Айкович и я исследователь БКС Мир Инвестиций. Сегодня я расскажу, как мы можем глубже...

Всё, что вы хотели знать о t-тесте, но боялись спросить

Всем привет! Меня зовут Александр Щукин, я аналитик в СКБ «Контур». В этой статье я расскажу о базовых основах применения t‑критерия Стьюдента. Для начала предлагаю Вам поучаствовать в...

Second-generation p-values: Improved rigor, reproducibility, & transparency in statistical analyses

Verifying that a statistically significant result is scientifically meaningful is not only good scientific practice, it is a natural way to control the Type I error rate. Here we introduce a novel extension of the p-value—a second-generation p-value (pδ)–that formally accounts for scientific relevance and leverages this natural Type I Error control. The approach relies on a pre-specified interval null hypothesis that represents the collection of effect sizes that are scientifically uninteresting or are practically null. The second-generation p-value is the proportion of data-supported hypotheses that are also null hypotheses. As such, second-generation p-values indicate when the data are compatible with null hypotheses (pδ = 1), or with alternative hypotheses (pδ = 0), or when the data are inconclusive (0 < pδ < 1). Moreover, second-generation p-values provide a proper scientific adjustment for multiple comparisons and reduce false discovery rates. This is an advance for environments rich in data, where traditional p-value adjustments are needlessly punitive. Second-generation p-values promote transparency, rigor and reproducibility of scientific results by a priori specifying which candidate hypotheses are practically meaningful and by providing a more reliable statistical summary of when the data are compatible with alternative or null hypotheses.

Selective Inference: The Silent Killer of Replicability

Experience Statistics

Experience Statistics

Experience Statistics