Transportist: Dr. Yadi Wang
Transportist: Dr. Yadi Wang
A Benefit-Cost Analysis of Benefit-Cost Analysis
Congratulations to Dr. Yadi Wang for “satisfying the requirements for the award of the degree of Doctor of Philosophy at the University of Sydney.”
Thesis Title: "A Benefit-Cost Analysis of Benefit-Cost Analysis”
Lead Supervisor: Professor David Levinson.
Abstract: Benefit/Cost Analysis (BCA) has a long history of being used as the primary tool aiding public decision-making. Two primary questions motivate this dissertation:
• Do the results output by BCA add value and impact decision-making? and
• Is BCA trustworthy enough that decision-makers should use it?
Grounded on more than 100 transport projects completed in Asian developing countries, the US, and Australia, this dissertation aims to understand and examine the benefits (if any) created by using BCA via five dimensions: accuracy, appropriateness & consistency, fiscal sustainability, transparency & replicability, and comprehensive.
The dissertation draws the following conclusions:
First, given the findings that the accuracy of BCA for road projects in Asian developing countries is better than that reported in other international evidence and that transport projects in the United States and Australia persistently underperform the original estimates, the accuracy of BCA varies depending on the context.
Second, the shadow price adopted by the road projects in Asian developing countries when calculating user benefits is determined fairly and adjusted realistically, and applying a uniform 12% social discount rate ensures the comparability of BCA results and secures the overall return on investment. In the US, the ‘do-nothing’ option is separated from the ‘do-minimum’ option, and appointing the latter as the baseline for evaluating other ‘do-something’ alternatives secures the realism of the appraisal of alternatives. All these findings corroborate that the social discount rate, non-market valuation method, and baseline case are appropriate and consistent to report and evaluate project performance.
Third, the lower-than-estimated ridership reported in the US and Australia imply the unlikelihood that revenues generated by the use of transport services, like tolls and fares, are capable of justifying project financial viability and sustaining future spending, negating that the benefit assessment approach embedded in the existing BCA process contributes to fiscal sustainability.
Fourth, various challenges encountered when attempting to replicate BCA suggest that the current BCA processes are neither sufficiently transparent nor replicable to defend the justifiability of decisions made upon BCA.
Last, the findings based on the alternative history of US light rail project show that the travel-time-based user benefit assessment is unable to fully differentiate the worthiness of candidate alternatives. One particular mode is always preferred over other competing alternatives in the US and Australia. These phenomena cast doubt on whether the present BCA is free from manipulation or bias.
The findings that in the US and Australia the ultimate preferred alternative persistently underperforms the ‘do-minimum’ and second-best ‘do-something’ option remind us of the opportunity costs. More projects could have served more people with the same budget, and prospective demand could have been managed by more economical courses of action. More importantly, this restricts the BCA from contributing to ex ante decision-making because options with a higher rate of return were generally declined.
These findings corroborated some practical issues (i.e. measurement and valuation problems) underlying the application of BCA. These practical issues are classified into three categories: deficiencies in the inputs to BCA, the technique and empirical basis of BCA itself, as well as the limited role of BCA in decision-making. The poor quality of inputs to BCA, including various estimates and options under investigation, would likely result in poor quality results, compromising the investment decisions made upon the results. Using NMV to monetize various user benefits, as one of the core processes underpinning the conventional BCA, is also subject to various practical difficulties. The findings that in the US and Australia the ultimate preferred alternative persistently underperforms the ‘do-minimum’ and second-best ‘do-something’ options remind us of the opportunity costs. More projects could have served more people with the same budget, and prospective demand could have been managed by more economical courses of action. More importantly, this reveals that BCA carries limited weight in decision-making because options with a higher rate of return were generally declined.
Indeed, the wide investment decision-making process embraces many factors, and choices between alternatives are the responsibility of decision-makers, not BCA. BCA as an investment assurance tool typically assesses the relative advantages of alternative options (or projects) from the economic perspective. As such, BCA can inform but cannot direct decision-makers, particularly when objectives are maximizing aggregate social welfare (accounting for social equity and environmental sustainability) rather than cost-effectiveness. Discounting BCA in decision-making may not necessarily be navigating politicians to the wrong decisions. Multilateral financial institutions like Asian Development Bank (ADB) demonstrate sound value of using BCA to assist project decision making.
However, the problematic mobility-oriented and travel-time-based benefit assessment methodology embedded in the conventional transport BCA, as demonstrated by the empirical studies in the US and Australia, cannot leverage the value of the concepts underpinning BCA. Abandoning BCA and switching to other decision-making tools or frameworks are not risk-free solutions because the flaws of input quality and the limited role in decision-making potentially apply to all frameworks and tools. The feasibility of incorporating changes in land use and real estate value into project-specific incremental benefit assessment via access measures, as corroborated in the New York Second Avenue Subway project (in chapter 10) and many other empirical studies conducted in major cities across the globe (in chapter 4), could potentially mitigate some measurement and valuation problems. Using the access-based method potentially outsources the valuation issues related to NMV to the real estate market, as the unit dollar value of access changes is informed by property/land value uplift.
As a result, it should be noted that the research and findings in this thesis are not challenging the theoretical basis for BCA. Rather, this thesis proves that there are gaps between the theory and the practice. In theory, theory should have precisely captured the truly additional benefits ascribed to transport investment. However in practice, as demonstrated by numerous findings presented in this thesis, the empirical implementation of the theoretical practices, alongside the idealized assumptions, confronted many challenges. In a nutshell, as the quote says, “In theory, theory and practice are the same. But in practice, they are different”.