Measuring the Unmeasurable? Systematic Evidence on Scale Transformations in Subjective #SurveyData https://d.repec.org/n?u=RePEc:iza:izadps:dp18029&r=&r=hap
"The relationship between the ‘cost’ of deviating from #linearity and the risk of sign reversal is, as one might expect, concave. Approximately 20% of results published in leading economic journals are reversed with some transformation that has a plausible cost. Restricting ourselves to interpreting wellbeing data as merely ordinal (i.e. allowing for any departure from linear scale use), increases this share to about 60%.
… Turning to relative magnitudes, we focus on unemployment and income as key determinants studied across multiple papers in our database. While coefficient signs for these are fairly robust, their relative magnitudes are highly sensitive to scale use assumptions. Marginal rates of substitution between unemployment and income can vary by an order of magnitude under plausible deviations from linearity.
… #statisticalInference and estimates of relative effect magnitudes become unreliable, even under modest departures from linearity. This is especially problematic for policy applications. We show that these concerns generalise to many other widely used survey-based constructs."
#ordinalMeasures #LikertScale #economics