(… continued from Let’s be courageous, part 4, posted on 24 Aug 2016).
I finished my previous posting by suggesting that Department for Transport (DfT) analysts may be guilty of being over-optimistic when assessing project benefit cost ratio (BCR) values: some evidence that this may well be the case can be found in the BCR predictions that were made for the HS1 project.
As recently as January 2009, using the analysis methodology favoured by the DfT, the BCR for HS1 was rated at 1.76, including wider economic benefits, indicating, it was claimed, “strong value for money” (see footnote 1). Despite this, work commissioned by the Government only three years later concludes that the analysis at that time indicated that the BCR had fallen to 0.64, including wider economic benefits (WEI), or 0.53 if these benefits are excluded (see footnote 2). This reappraisal puts HS1 firmly into the low value for money category using the DfT’s own evaluation criteria.
In a report published in the wake of the HS1 BCR rework finally becoming available in October 2015, the House of Commons Committee of Public Accounts (PAC) expresses concern that despite “evidence [that] suggests that its methodology is inadequate for some types of transport projects” the DfT “continues to use it to assess new projects such as HS2” (see footnote 3).
Earlier in 2015, the Economics Affairs Committee of the House of Lords (EAC) levelled its own criticism at the BCR methodology and the way that it had been employed to evaluate the value for money offered by the HS2 project. The EAC accepts that “cost-benefit analysis is an important discipline” but, somewhat stating the obvious, the EAC also opines that “the reliability of the method for quantifying the benefits of a project depends upon the quality of the evidence used in the analysis” (see footnote 4). This is an important qualification, however, as the EAC is also of the opinion that the “cost-benefit analysis for HS2 relies on evidence that is out-of-date and unconvincing” (see footnote 5).
The DfT appears to be sensitive to the criticisms and, as the HS2 project has developed, has sought to accommodate its detractors. So changes to assumptions and parameters have been made, and the value of the BCR has fluctuated a little. However, whenever a change looks like it might diminish the BCR beyond what might be considered a politically-acceptable level, some balancing adjustment appears to have found to avoid this necessity, and embarrassment has been miraculously evaded. As I put it in my blog They haven’t been twiddling their thumbs, part 2 (posted 10 Dec 2013):
“It appears that, whenever a breach in the Government’s case is discovered, it compensates by constructing some new pretence.”
The DfT makes the point that “as with all business cases, the underlying economic case for HS2 will change over time” and that the assessment of the business case “will continue to develop in the months and years ahead” (see footnote 6). This is perfectly proper, of course, but I think that we have the right to expect that each new assessment will be approached with an appropriate degree of realism, and that dispassionate and reasoned logic will be at the fore, rather than any tendency towards window dressing.
The DfT also concedes that “there are challenges to appraising the potential benefits and costs of a transformational transport scheme such as HS2”, and that there is a degree of uncertainty with any predictions (see footnote 7). The DfT response to this uncertainty is to employ a combination of risk analysis and sensitivity testing to arrive at the “range of benefit cost ratios that could result from combinations of different assumptions” (see footnote 8). Some results of analysis of this type has been reported by the National Audit Office showing for example, that Phase 1 returns a low value BCR rating for 23 per cent of modelled scenarios (see footnote 9).
Whilst there is the obvious caveat that the value of this type of analysis depends substantially upon the credibility of the ranges of scenarios that are chosen, it is surely valuable to have some idea of the possible uncertainty associated with a BCR assessment. Unfortunately, it appears to be almost invariably the practice of HS2 advocates and detractors alike to quote point values of BCR without giving any consideration to the values over which the BCR can reasonably be expected to range.
The uncertainties in analysing the business case for a single specified project can be eliminated to an extent if the same set, or largely the same set, of assumptions can be utilised to compare the results for different projects; either in order to find the best of a number of strategic alternatives, or merely the most worthy candidates for funding. In this respect, the EAC describes cost-benefit analysis as “the best tool for comparing several projects to see which provides the best value for money” (see footnote 10). Work of this type was carried out for the DfT early on in the lifetime of the HS2 project and the BCRs that were determined for three strategic alternatives to HS2 (see footnote 11) are quoted, under the subheading The Benefit-Cost Ratio, by Policy Analyst Harry Fairhead in Section Two of his Taxpayer’s Alliance briefing paper, Rich man’s toy: The case for scrapping HS2. These BCRs, which all include WEI, range between 3.11 and 6.06, all comfortably better value for money than the HS2 solution.
Whilst the BCR calculations of the alternatives to HS2 were carried out prior to some significant changes in methodology being introduced by the DfT in the past couple of years, I think that it a safe assumption that there are alternatives to HS2 that represent much better value for money for the taxpayer, and any review of the HS2 project should bear this in mind.
(To be continued …)
- See paragraph 6.1.3 of the report Economic Impact of High Speed 1, Final Report, London and Continental Railways, January 2009.
- See the final paragraph on page 140 (Section 6.4.1) of the report First Interim Evaluation of the Impacts of High Speed 1, Final Report, ATKINS, AECOM and Frontier Economics, October 2015. The publication of this report was originally promised for summer 2013, and its delay was criticised by the PAC.
- See paragraph 23 in the report The Sale of Eurostar, House of Commons Committee of Public Accounts Sixteenth Report of Session 2015-16, January 2016.
- See paragraph 366 in the report The Economics of High Speed 2, House of Lords Economic Affairs Committee, 1st Report of Session 2014-15, March 2015.
- See paragraph 10 in The Economics of High Speed 2.
- See paragraph 2.8 of the report HS2 Outline Business Case Economic Case, Department for Transport, March 2014.
- See paragraph 2.6 of HS2 Outline Business Case Economic Case.
- See paragraphs 5.4 to 5.7 of HS2 Outline Business Case Economic Case.
- See Figure 12 in the report Progress with preparations for High Speed 2, National Audit Office, June 2016.
- See paragraph 418 in The Economics of High Speed 2.
- The source of Mr Fairhead’s figures is Table 5.8 in the report High Speed Rail Strategic Alternatives Study Update Following Consultation, Atkins/Department for Transport, January 2012.