“Transit Makes you Short”: On Health Impact Assessment of Transportation and the Built Environment
How the p-value and coefficient changes with observation size Recently published:
Ermagun, Alireza, and Levinson, D. (2015) “Transit Makes you Short”: On Health Impact Assessment of Transportation and the Built Environment. Journal of Transport and Health. (online first) [doi]
Abstract
This study conducts an in-depth analysis to alert policymakers and practitioners to erroneous results in the positive impacts of transit use on health measures. We explore the correlation of transit use and accessibility by transit and walking with self-reported general health, Body Mass Index (BMI), and height. We develop a series of linear regression and binary logit models. We also depict the coefficient-p-value-sample-size chart, and conduct the effect size analysis to scrutinize the practically significant impacts of transit use and accessibility on health measures. The results indicate transit use and accessibility by transit and walking are significantly associated with general health and BMI. However, they are practically insignificant, and the power of the large sample in our particular case causes the statistically insignificant variable to become significant. At a deeper level, a 1% increase in transit use at the county level diminishes the BMI by only 0.0037% on average. The elasticity of transit use also demonstrates that every 1% increase in transit use would escalate the chance of having excellent or very good general health by 0.0003%. We show there is a thin line between false positive and true negative results. We alert both researchers and practitioners to the dangerous pitfalls deriving from the power of large samples and the weakness of p-values. Building the results on just statistical significance and sign of the parameter of interest is worthless, unless the magnitude of effect size is carefully quantified post analysis.