Measures of Central Tendency for an Asymmetric Distribution, and Confidence Intervals – Statistical Thinking

There are three widely applicable measures of central tendency for general continuous distributions: the mean, median, and pseudomedian (the mode is useful for describing smooth theoretical distributions but not so useful when attempting to estimate the mode empirically). Each measure has its own advantages and disadvantages, and the usual confidence intervals for the mean may be very inaccurate when the distribution is very asymmetric. The central limit theorem may be of no help. In this article I discuss tradeoffs of the three location measures and describe why the pseudomedian is perhaps the overall winner due to its combination of robustness, efficiency, and having an accurate confidence interval. I study CI coverage of 18 procedures for the mean, one exact and one approximate procedure for the median, and five procedures for the pseudomedian, for samples of size \(n=200\) drawn from a lognormal distribution. Various bootstrap procedures are included in the study. The goal of the confidence interval procedures is to achieve non-coverage probabilities that are close to the nominal 0.025 level in both tails. The usual standard deviation-based central limit theorem approach failed in both tails. The BCa bootstrap method was the most accurate for computing confidence limits for the mean, but the upper limit was too small with \(n=200\), having non-coverage probability of 0.086 for the right tail instead of the nominal 0.025. Three types of intervals for the more robust pseudomedian were extremely accurate, giving more reasons to use the pseudomedian as a primary location measure, whether or not the distribution is symmetric.

How to interpret “confidence intervals” in observational studies

This question complements the one in the thread Random sampling versus random allocation/randomization- implications for p-value interpretation. Given that observational studies involve neither random sampling nor random allocation, why are they riddled with “95% confidence intervals”?

Sampling Error

Part 1c of Sampling and Confidence Intervals by Dr. Alvin Ang

Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation | Annual Reviews

This review examines recent advances in sample size planning, not only from the perspective of an individual researcher, but also with regard to the goal of developing cumulative knowledge. Psychologists have traditionally thought of sample size planning in terms of power analysis. Although we review recent advances in power analysis, our main focus is the desirability of achieving accurate parameter estimates, either instead of or in addition to obtaining sufficient power. Accuracy in parameter estimation (AIPE) has taken on increasing importance in light of recent emphasis on effect size estimation and formation of confidence intervals. The review provides an overview of the logic behind sample size planning for AIPE and summarizes recent advances in implementing this approach in designs commonly used in psychological research.

Uncertainty Estimation with Conformal Prediction – Michael Clark

More options for uncertainty estimation

What fraction of repeat experiments will have an effect size within the 95% confidence interval of the first experiment?

Let's stick to an ideal situation with random sampling, Gaussian populations, equal variances, no P-hacking, etc. Step 1. You run an experiment say comparing two sample means, and compute a 95%

Understanding Precision-Based Sample Size Calculations | UVA Library