An apology

My apologies to those of you itching with anticipation for part 10 of Let’s be courageous, but I have not been able to prepare this post due to other pressing calls on my time this past week.

I hope to have part 10 ready by the next but one four-day slot, that is Sunday 25th September.

Update 25th September – Events have conspired against me this past week and I’m afraid that I have not been able to work on part 10, so I am now working towards posting it on Monday 3rd October.

Let’s be courageous, part 9

(… continued from Let’s be courageous, part 8, posted on 9 Sep 2016).

Taxpayer’s Alliance Policy Analyst Harry Fairhead devotes one whole subsection of the section that deals with the business case (Section Two) of his briefing paper, Rich man’s toy: The case for scrapping HS2, to a complaint about the way that net present value has been determined in the HS2 benefit cost ratio (BCR) calculations: under the subheading Inappropriate discount rate he argues that the application of an “inappropriate discount rate may mean that the present value of the benefits and costs have been misrepresented and that the BCR analysis is flawed”.

I must confess that, in my reading about HS2, I have not encountered very much comment about this particular topic – perhaps this is because we are straying into the nuts and bolts of investment economic theory here, of little interest to the average reader – but I was stimulated sufficiently by Mr Fairhead’s charge of misrepresentation to investigate the subject.

The application of a discount rate in business case assessments takes account of “social time preference”, which is a principle that may be summarised by the old adage “a bird in the hand is worth two in the bush”. It takes account of the general preference of people to receive goods and services now rather than later, and is typified by the expectation that interest will be paid on money loaned. This means that £1 received, or spent, in the future will be worth less than £1 net present value and so needs to be “discounted” by a percentage to work out that net present value (see footnote 1). For the HS2 business case the discount rates that have been applied, stated to be in accord with the recommendations of WebTAG Unit 3.5.4 (August 2012) are 3.5 per cent up to 2043, 3.0 per cent between 2044 and 2088 and 2.5 per cent thereafter (see footnote 2).

So £100 to be received in a year’s time, with a discount rate of 3.5 per cent applying, has a net present value of £96.66 (100/103.5). If it is to be received in two years’ time the net present value is £93.35 (96.66×100/103.5), and so on (see footnote 3).

This leads to two important observations about the way that the discount rate affects net present value:

  • For any given vale of the discount rate, the net present value decreases as the time to the money being received or spent increases
  • For the same time period, the net present value is decreased as the discount rate is increased

For projects like HS2 the costs tend to be front heavy, as the costs of construction normally dominate, and the benefits accrue later, since the project must normally be operational before it can deliver benefits. This characteristic, together with the operation of the properties noted in the above two bullet points, leads to the BCR being better when the discount rate is lower. This effect is not insignificant. In its June 2011 report on HS2, commissioned by the Government, the consultancy company Oxera demonstrated that this was the case. It recalculated the BCR for the full network, as it was at the time, replacing the discount rate profile that had been used (3.5 per cent falling to 3 per cent after thirty years) by the same profile but with the rates reduced by 0.5 per cent (i.e. 3.0 per cent falling to 2.5 per cent after thirty years). The result was that the BCR increased by 1.0, without wider economic impacts (WEI), and by 1.2 if WEI were taken into account (see footnote 4).

Mr Fairhead bases his charge that an inappropriate value for the discount rate has been applied for the assessment of the HS2 business case upon the analysis in a March 2016 paper, Smoke and mirrors: how the cost of systematic risk has vanished from public sector appraisals and evaluations, published by consultancy company Europe Economics and authored by Stephen Topping, Managing Consultant for the firm.

In his paper, Mr Topping points out that the H M Treasury recommended discount rate was, prior to the 2003 revision of the Treasury Green Book, much higher than the currently recommended range of 3.5 per cent declining in steps to 1.0 per cent over longer terms; it was then 6 per cent. One of the drivers behind this reduction was, as Mr Topping expresses it, “the removal from the discount rate of the risk premium that was previously incorporated to reflect systematic risk”: Mr Topping deduces that this risk premium was 1.5 per cent, or one-quarter of the total discount rate.

Systematic risks are identified in the Treasury Green Book as “appraisal risks that are systematically related to the overall performance of the economy” resulting from “random factors unforeseen at the time of the appraisal”, and I guess that the stock market crash of 2008 would be a relatively recent example of such an event (see footnote 5).

The Treasury Green Book describes the revision made to the discount rate in the 2003 edition as one which “unbundles” it (see footnote 6). A memorandum submitted to the Commons Public Accounts Committee by H M Treasury clarifies the significance of this description: it explains that “it is now essential that other appraisal issues, which could otherwise be accounted for implicitly by using a higher rate, be dealt with separately and explicitly” and that the purpose of the unbundling is to “ensure that separate elements of an appraisal are now properly accounted for” (see footnote 7).

Despite this advice from H M Treasury, Mr Topping notes that a separate adjustment “to take account of the cost associated with systematic risk is not being made in practice” by those responsible for preparing the business cases for Government projects, “despite the fact that systematic risk is a key driver of the cost of capital in the private sector”. He adds that he “has never seen a cost-benefit analysis which included such a cost”: certainly, no account of it has made in the HS2 business case.

Mr Topping attributes this failure to a statement in the Treasury Green Book, that (see footnote 8):

“Given the size of national income relative to the scale of most individual projects, the cost of variability for projects that benefit the community as a whole is usually negligible.”

He deems this judgement to be “erroneous”, arguing that “the absolute cost of variability may still be significant” in the context of a project “given that the national income is very large compared to the scale of most individual projects”. Mr Topping concludes that the “implication is that public expenditure projects and policy proposals may appear to pass a cost-benefit test when they are not in fact worth doing”.

As I remarked above, I have not been able to find much in the way of commentary on this subject from other quarters, but I have identified one public figure who has picked it up. In an opinion piece in the Daily Telegraph, Archie Norman of ASDA and ITV fame and a former Conservative MP, ventures the opinion that the “Government’s case for HS2 assumes an absurdly low and declining discount rate” that “is not risk-adjusted”.

(To be continued …)

Footnotes:

  1. A fuller explanation of the principle may be found in paragraphs 5.48 to 5.54 and Annex 6 of the publication The Green Book Appraisal and Evaluation in Central Government, H M Treasury, July 2011.
  2. See paragraph 3.4.15 in the publication The Economic Case for HS2 PFM v4.3: Assumptions report, HS2 Ltd/Department for Transport, October 2013.
  3. A table allowing net present values to be calculated for different delays in payment and various discount rates may be found on page 100 of The Green Book Appraisal and Evaluation in Central Government.
  4. See Table A2.2 in the report Review of the Government’s case for a High Speed Rail programme, Oxera for the House of Commons Transport Select Committee, June 2011.
  5. See paragraphs 34 and 35 in Annex 4 to The Green Book Appraisal and Evaluation in Central Government.
  6. See the Preface to The Green Book Appraisal and Evaluation in Central Government.
  7. See under the subheading Associated changes in the new edition of the Green Book in the document Further supplementary memorandum submitted by the HM Treasury Discount rate, H M Treasury written evidence to the House of Commons Public Accounts Committee, June 2003.
  8. See paragraph 38 in Annex 4 to The Green Book Appraisal and Evaluation in Central Government.

Let’s be courageous, part 8

(… continued from Let’s be courageous, part 7, posted on 5 Sep 2016).

The year 2013 saw the Department for Transport (DfT) publishing a draft revision to its WebTAG guidelines. Of the alternatives that had been set out in the report commissioned from the Institute for Transport Studies at the University of Leeds, the DfT chose to employ the “do minimum” approach (Option 2). This meant that the WebTAG guidance retained the cost saving approach determination for the valuation of saved travelling time, but reviewed the evidence and the per-passenger hour valuations derived from that evidence. As a result the WebTAG valuation of a business rail passenger’s time was reduced from £47.18 to £31.96 (see footnote 1) and also the update saw “the introduction of a range around the values [of saved time] reflecting the quality of the available data” (see footnote 2).

It would appear that, in preparing the October 2013 revision to the HS2 business case, the DfT was anxious to avoid the criticisms about the high value of each hour’s business time saved that had been assumed for the original BCR calculation, because the reduced WebTAG guideline value was employed for the October 2013 revision, despite the guidelines being only a draft: my understanding of DfT practice is that revised guidelines are not normally adopted for business cases until they are deemed “definitive” (see footnote 3). Also, in an obvious nod towards accepting that employing a willingness to pay approach could deflect much of the criticism about not taking account of the possibility of using time spent travelling by rail productively, the DfT compiled a “comparison of values resulting from different approaches”, which was the basis for the DfT’s conclusion that the updated WebTAG travel time savings values “are a suitable representation of businesses’ willingness-to-pay” (see footnote 4).

The House of Lords Economic Affairs Committee (EAC), it appears, was not convinced that the “proxy”, as it referred to it, value of £31.96 really did serve to quell the criticism. According to the EAC “the values of working time savings for rail do not take account of the fact that time on a train can be used productively” (see footnote 5).

The EAC’s verdict is quite damning (see footnote 6):

“We find it difficult to have any faith in benefits that have been estimated on the basis of these values, particularly as the Department for Transport has recently concluded that fresh evidence is required and has commissioned further research.”

On the basis of the October 2013 guideline value for business traveller time saved, the value used for the original HS2 business case was about one-half too high. All other things being equal, this would have had a big impact upon the BCR, as business traveller savings form a large proportion of the total benefits claimed for HS2. In some work carried out in September 2012, and therefore before the WebTAG values were revised, the HS2 Action Alliance calculated that, had the original value of business time been just one-third too high, then this would reduce the BCR for the full network by around 0.4 (see footnote 7).

I say that it “would have had a big impact upon the BCR” because, by an incredibly fortunate stroke of serendipity, the DfT discovered that it had underestimated the proportion that HS2 business travellers would contribute to the overall total of passengers. This proportion is analysed separately for four different HS2 journeys, between: London and Birmingham, London and Leeds, London and Manchester, and London and Sheffield. In the original business case the proportion of the total number journeys over each of these routes attributed to travellers on business ranged from 23 per cent to 28 per cent (see footnote 8). For the October 2013 revision of the business case this range of proportions was increased to between 56 per cent and 65 per cent (see footnote 9).

Since business travel hours saved are rated by WebTAG at more than fourfold the value of travel for other purposes, this increase in the business travel proportion delivered a net increase in the value of user benefits of £11.5bn (see footnote 10), recovering a sizeable proportion of the benefits lost from the reduction in the hourly rate.

The DfT provided a written explanation to the EAC of how this windfall had arisen, attributing it to a change in the way that data upon which the travel purpose proportions are estimated had been collected (see footnote 11). Demonstrating the thoroughness that was characteristic of the way that the EAC approached its inquiry into the economic case for HS2, the evidence behind the DfT’s upward revision to the estimate of proportion of HS2 passengers who will be travelling for business purposes is reviewed in the EAC’s report (see footnote 12). The EAC appears to have been far from convinced by the DfT, concluding that “the substantial increase in forecast business travel in the latest economic case is questionable” (see footnote 13).

Overall, the EAC gives a distinct thumbs down to the DfT’s efforts to monetarise the benefits from travel time savings, opining (see footnote 14):

“84 per cent of the estimated benefits of HS2 rely on the values allocated to travel time savings and demand forecasts. The evidence behind both of these is inconsistent and out-of-date.”

The “further research” into the value of travel time savings that I refer to above has been going on since the October 2013 WebTAG revisions, and the DfT published a progress report in December 2014. We are promised that the Department is intending to “go beyond” using the cost saving approach determination by collecting evidence of willingness to pay from stated preference surveys of employers and employees (see footnote 15). The aim is to use this new evidence of willingness to pay to publish new guideline values of travel time savings in an update to WebTAG in 2016 (see footnote 16). No doubt, a corresponding reworking of the HS2 business case will follow this.

(To be continued …)

Footnotes:

  1. See paragraph 391 in the report The Economics of High Speed 2, House of Lords Economic Affairs Committee, 1st Report of Session 2014-15, March 2015.
  2. See paragraph 4.16 in the publication Understanding and Valuing the Impacts of Transport Investment, Department for Transport, October 2013.
  3. See paragraph 6.1.7 of the publication The Economic Case for HS2, HS2 Ltd/Department for Transport, October 2013. This paragraph advises that the revised values “will shortly be mandated for use on all transport schemes”, implying that the updated values were not definitive when the revised HS2 business case was published.
  4. See the sidebar below paragraph 6.1.7 on page 44 of The Economic Case for HS2.
  5. See paragraph 395 in The Economics of High Speed 2.
  6. See paragraph 396 in The Economics of High Speed 2.
  7. See the subsection Impact of the overestimation of the £70k/a on the BCR in the paper Overestimation of value of business travellers time –the facts, HS2 Action Alliance, September 2012.
  8. See paragraph 406 in The Economics of High Speed 2.
  9. See Table 27 in The Economics of High Speed 2.
  10. Footnote 496 on page 116 of The Economics of High Speed 2 derives this net value from an increase to benefits to business travellers of £14.5bn less a decrease in benefits to commuter/leisure travellers of £3.1bn.
  11. See paragraphs 9 to 12 pages 401 and 402 of the publication The Economic Case for HS2 Oral and Written Evidence, House of Lords Economic Affairs Committee, October 2014.
  12. See paragraphs 397 to 415 in The Economics of High Speed 2.
  13. See paragraph 416 in The Economics of High Speed 2.
  14. See paragraph 417 in The Economics of High Speed 2.
  15. See paragraph 5.13 in the publication Understanding and Valuing the Impacts of Transport Investment Progress Report 2014, Department for Transport, December 2014.
  16. See paragraph 5.28 in Understanding and Valuing the Impacts of Transport Investment Progress Report 2014. The “release point” for the new version of WebTAG Unit A1.3 is currently advised as November 2016, according to the DfT document WebTAG: Forthcoming Changes.

 

 

Let’s be courageous, part 7

(… continued from Let’s be courageous, part 6, posted on 1 Sep 2016).

My previous posting promised that I would delve into the assumptions behind the value set for the per-passenger hourly benefit gained by business travellers from a shorter journey time that has been utilised in the benefit cost ratio (BCR) calculation– this value being a choice that has attracted much criticism. It should be appreciated however that this question goes beyond the HS2 project since, although the estimate of the number of passenger hours travel time that HS2 will save is a project-specific calculation, the value of each passenger hour is standardised across all transportation projects by the WebTAG guidelines.

This standard rate has been worked out using the “cost saving approach”, a process that the House of Lords Economic Affairs Committee explain (see footnote 1):

“It calculates the value of working travel time by adding the gross wage to non-wage labour costs. The gross wage rate is calculated for rail passengers using evidence from the National Travel Survey. A percentage increase is then applied to reflect non-wage labour costs such as national insurance and pensions contributions.”

The WebTAG guideline value for an hour saved of businessman’s time that was utilised for the original (2011) presentation of the HS2 business case was £47.18, based upon 2010 values (see footnote 2). One critic has calculated that, if the 24.1% mark-up that has been applied to account for non-wage costs is excluded, this equates to an average annual salary level for business rail travellers of about £70,000 in 2009 values (see footnote 3). That critic remarked that an average of £70k annual salary “does appear high” and I think that most observers at the time thought this to be the case. The problem seems to have been one of failing to keep up with the reducing elitism of business travel. The value was derived from “2002 earnings data applied to 1999-2001 National Travel Survey data” (see footnote 4) and, accordingly, reflected the situation as it had been more than ten years previously. Our critic opined that “it is unlikely that this high average income will be sustained in real terms over time” – indeed it is unlikely that it applied even in 2011 when the HS2 business case was first published – and that “if the market [for rail business travel] is to grow at the rate forecast, the composition of rail business travellers will be diluted by more people at the lower end of the income range and the average income of the group will gradually move towards mean income levels for business travellers by all modes” (see footnote 5).

This putative overvaluing of business rail users’ time, added to the overestimating of the productive value of the time saved due to questionable assumptions I identified in part 6, that journey time saved is used in the workplace, rather than for leisure, and that business passengers are not productive while travelling, combine to make the original calculation of business user benefits extremely dubious, to say the least. Confirmation that the DfT acknowledged the criticisms that had been levied at its methodology was provided in 2012, but the Department justified its assumptions as “a necessary simplification in the absence of robust evidence to underpin a more sophisticated approach” (see footnote 6). An indication that the DfT was at least considering an alternative approach came in 2013, with the publication of a report commissioned by the Department from the Institute for Transport Studies (ITS) at the University of Leeds reviewing the evidence for valuing working time travel savings.

The authors of the ITS report make clear that it is not their purpose “to provide a recommended approach to valuation but rather to provide an authoritative evidence base to allow the Department to make informed judgements as to the best way forward in the valuation of travel time savings for business travellers” (see footnote 7).

The report compares the pros and cons of the cost saving approach that had been employed for the HS2 business case analysis with two alternative methods: the Hensher approach, which the ITS characterises as providing “a conceptual framework for capturing a range of additional factors that might influence the value of business travel time savings to employers and employees”; and the willingness to pay (WTP) approach, which the ITS advises “could either be based on observations of actual behaviour (a ‘revealed-preference’ approach) or hypothetical choices (a ‘stated preference’ approach)”. The ITS cautions that the Hensher approach “has been widely examined but seldom implemented due to the difficulties and uncertainties involved”, but regards WTP as “relatively straightforward to implement” (see footnote 8).

Helpfully, the ITS report identifies nine different options from which the DfT was invited to select a way forward. The first two of these options, “do nothing” and “do minimum”, both retain the cost saving approach, whereas the other seven options all require a change to one of the two alternative approaches, or to a combination of approaches (see footnote 9).

An examination of what the DfT actually chose to do when it reviewed the HS2 business case in 2013 will be the subject of my next posting.

(To be continued …)

Footnotes:

  1. See paragraph 380 in the report The Economics of High Speed 2, House of Lords Economic Affairs Committee, 1st Report of Session 2014-15, March 2015.
  2. See Table 3 on page 43 of the publication The Economic Case for HS2, HS2 Ltd/Department for Transport, October 2013.
  3. See the second paragraph of the subsection The value of working time on trains on page 34 of the report Review of the Economic Case for HS2, Castles C and Parish D, RAC Foundation, November 2011.
  4. See paragraph 3.6 in the report Review of the Government’s case for a High Speed Rail programme, Oxera for the House of Commons Transport Select Committee, June 2011.
  5. See the third paragraph of the subsection The value of working time on trains on page 34 of Review of the Economic Case for HS2.
  6. See page 8 of the paper Review of the value of time assumptions for business travellers on HS2, Department for Transport Strategy Unit, April 2012.
  7. See subsection 1.1 in the report Valuation of travel time savings for business travellers, Wardman M, et al, Institute for Transport Studies, April 2013.
  8. The quotes from the ITS authors are all to be found on page 7 of Valuation of travel time savings for business travellers. Overviews of the three approaches may be found in subsection 2.4.1 of the ITS report.
  9. For an explanation and discussion of the nine options suggested refer to Chapter 5 of Valuation of travel time savings for business travellers.

Let’s be courageous, part 6

(… continued from Let’s be courageous, part 5, posted on 28 Aug 2016).

In my previous posting I reported that Policy Analyst Harry Fairhead had, under the subheading The Benefit-Cost Ratio, in Section Two of his Taxpayer’s Alliance briefing paper, Rich man’s toy: The case for scrapping HS2, made the comparison between the latest BCR estimate for HS2 Phase 1 of 1.7, including wider economic benefits (WEI), and similar estimates for three strategic alternatives to HS2, which were all in the BCR range 3 to 6 (about), including WEI.

In its March 2015 report on HS2, the House of Lords Economic Affairs Committee (EAC) attempted to place the value for money expectations for HS2 in the wider context of government spending on transportation projects. The EAC quotes one of the witnesses that gave it evidence, Professor Henry Overman Professor of Economic Geography at the London School of Economics, as finding it “surprising that we refer to projects as medium value for money when they are probably in the bottom 10%, say, of transport projects that we have on the books as doable” (see footnote 1).

The truth of the professor’s observation is demonstrated by data provided to the EAC by the DfT, which is summarised in a table provided in the EAC’s report and reproduced below.

Source: Lords Economic Affaits Committee (see footnote 2)

Source: Lords Economic Affaits Committee (see footnote 2)

It appears from this table that none of the DfT’s spend in the years 2011 and 2012 went on projects rated lower than high value for money, and in 2013 a mere 6% of spend was for projects rated medium value for money. The figures in the tabulation give a strong indication that projects like HS2 that hover, at very best, around a BCR of 2 to 2.5 will rate within the lowest decile of value for money across the range of projects that have received DfT funding in recent years.

In a letter to the EAC, David Prout, Director-General of HS2 Group at the DfT, chose to place a different slant upon the data in the tabulation, claiming that the value for money category into which HS2 falls is the same as for “the majority of transport spending by DfT over the past three years”.

Since the table gives some support to both Professor Overman’s and Mr Prout’s interpretations, the EAC asked the Secretary of State for a further breakdown of the “high” category into the two value for money ranges 2.0-3.0 and 3.0-4.0. In its report, the EAC expresses surprise that the SoS was unable to furnish this information, commenting that “while the [DfT] is able to provide a breakdown between projects of low value for money (1.0–1.5) and medium value for money (1.5–2.0), it is apparently unable to distinguish between projects with a value for money of 2.0–3.0 and 3.0–4.0”. Apparently feeling that there may be a degree of convenience associated with this inability to satisfy its request, the EAC adds the speculation that the inability, or perhaps unwillingness, to break the high value for money category down further may be “because the majority of projects in the 2.0–4.0 range (80 per cent of all projects in 2013) were, unlike HS2, in the 3.0–4.0 range” (see footnote 3).

Under the subheading Criticism of government BCR assumptions Mr Fairhead examines the familiar ground of the criticisms that have been levied against the values utilised by the DfT to ascribe a monetary value to transport user benefits. These have been calculated on the basis of a set amount for each passenger hour saved by using HS2. Three different values of time have been employed: one for user travel undertaken during the course of work (business travel); one for workers travelling to and from their normal employment location (commuters); and, one for those travelling for leisure purposes.

Mr Fairhead concentrates his attention on the benefits that have been calculated as accruing from time savings for business travellers, which is justified by his claim that around one half of the total transport user benefits result from this single mechanism (see footnote 4). As I mentioned in part 4 of this blog series, Mr Fairhead feels that the monetary value of the benefit to business travellers from the travel time savings that HS2 will offer has been overvalued and cites two “inherent flaws” in the assumptions that have been made:

  • That it is “a questionable assumption that passengers are not productive while travelling”
  • That the DfT has assumed that “journey time saved is time that would now be spent in the workplace”, whereas it is likely that workers will utilise time saved for leisure.

Similar criticisms of the assumptions behind the value utilised for the per-passenger hourly benefit gained by business travellers from a shorter journey time have come from a number of quarters and, over the life of the HS2 project, the DfT has moved some way towards addressing these, but this topic will have to wait until my next posting.

(To be continued …)

Footnotes:

  1. See paragraph 352 in the report The Economics of High Speed 2, House of Lords Economic Affairs Committee, 1st Report of Session 2014-15, March 2015. The original comment is recorded under Q49 in the Revised transcript of evidence taken before the Select Committee on Economic Affairs Inquiry on The Economic Case for HS2, Tuesday 28thOctober 2014.
  2. The tabulation is Table 22 in The Economics of High Speed 2.
  3. See paragraph 355 in The Economics of High Speed 2.
  4. Mr Fairhead’s source is Table 15 in Appendix 6 to the publication The Economic Case for HS2, HS2 Ltd/Department for Transport, October 2013. Examination of this tabulation indicates that total transport user benefits are predicted to be £59.8bn, of which £40.5bn accrues from savings by business travellers. This indicates that Mr Fairhead has erroneously underestimated the importance of business travel to the HS2 business case, as it actually accounts for around two-thirds of the claimed benefits.

 

 

Let’s be courageous, part 5

(… continued from Let’s be courageous, part 4, posted on 24 Aug 2016).

I finished my previous posting by suggesting that Department for Transport (DfT) analysts may be guilty of being over-optimistic when assessing project benefit cost ratio (BCR) values: some evidence that this may well be the case can be found in the BCR predictions that were made for the HS1 project.

As recently as January 2009, using the analysis methodology favoured by the DfT, the BCR for HS1 was rated at 1.76, including wider economic benefits, indicating, it was claimed, “strong value for money” (see footnote 1). Despite this, work commissioned by the Government only three years later concludes that the analysis at that time indicated that the BCR had fallen to 0.64, including wider economic benefits (WEI), or 0.53 if these benefits are excluded (see footnote 2). This reappraisal puts HS1 firmly into the low value for money category using the DfT’s own evaluation criteria.

In a report published in the wake of the HS1 BCR rework finally becoming available in October 2015, the House of Commons Committee of Public Accounts (PAC) expresses concern that despite “evidence [that] suggests that its methodology is inadequate for some types of transport projects” the DfT “continues to use it to assess new projects such as HS2” (see footnote 3).

Earlier in 2015, the Economics Affairs Committee of the House of Lords (EAC) levelled its own criticism at the BCR methodology and the way that it had been employed to evaluate the value for money offered by the HS2 project. The EAC accepts that “cost-benefit analysis is an important discipline” but, somewhat stating the obvious, the EAC also opines that “the reliability of the method for quantifying the benefits of a project depends upon the quality of the evidence used in the analysis” (see footnote 4). This is an important qualification, however, as the EAC is also of the opinion that the “cost-benefit analysis for HS2 relies on evidence that is out-of-date and unconvincing” (see footnote 5).

The DfT appears to be sensitive to the criticisms and, as the HS2 project has developed, has sought to accommodate its detractors. So changes to assumptions and parameters have been made, and the value of the BCR has fluctuated a little. However, whenever a change looks like it might diminish the BCR beyond what might be considered a politically-acceptable level, some balancing adjustment appears to have found to avoid this necessity, and embarrassment has been miraculously evaded. As I put it in my blog They haven’t been twiddling their thumbs, part 2 (posted 10 Dec 2013):

“It appears that, whenever a breach in the Government’s case is discovered, it compensates by constructing some new pretence.”

The DfT makes the point that “as with all business cases, the underlying economic case for HS2 will change over time” and that the assessment of the business case “will continue to develop in the months and years ahead” (see footnote 6). This is perfectly proper, of course, but I think that we have the right to expect that each new assessment will be approached with an appropriate degree of realism, and that dispassionate and reasoned logic will be at the fore, rather than any tendency towards window dressing.

The DfT also concedes that “there are challenges to appraising the potential benefits and costs of a transformational transport scheme such as HS2”, and that there is a degree of uncertainty with any predictions (see footnote 7). The DfT response to this uncertainty is to employ a combination of risk analysis and sensitivity testing to arrive at the “range of benefit cost ratios that could result from combinations of different assumptions” (see footnote 8). Some results of analysis of this type has been reported by the National Audit Office showing for example, that Phase 1 returns a low value BCR rating for 23 per cent of modelled scenarios (see footnote 9).

Whilst there is the obvious caveat that the value of this type of analysis depends substantially upon the credibility of the ranges of scenarios that are chosen, it is surely valuable to have some idea of the possible uncertainty associated with a BCR assessment. Unfortunately, it appears to be almost invariably the practice of HS2 advocates and detractors alike to quote point values of BCR without giving any consideration to the values over which the BCR can reasonably be expected to range.

The uncertainties in analysing the business case for a single specified project can be eliminated to an extent if the same set, or largely the same set, of assumptions can be utilised to compare the results for different projects; either in order to find the best of a number of strategic alternatives, or merely the most worthy candidates for funding. In this respect, the EAC describes cost-benefit analysis as “the best tool for comparing several projects to see which provides the best value for money” (see footnote 10). Work of this type was carried out for the DfT early on in the lifetime of the HS2 project and the BCRs that were determined for three strategic alternatives to HS2 (see footnote 11) are quoted, under the subheading The Benefit-Cost Ratio, by Policy Analyst Harry Fairhead in Section Two of his Taxpayer’s Alliance briefing paper, Rich man’s toy: The case for scrapping HS2. These BCRs, which all include WEI, range between 3.11 and 6.06, all comfortably better value for money than the HS2 solution.

Whilst the BCR calculations of the alternatives to HS2 were carried out prior to some significant changes in methodology being introduced by the DfT in the past couple of years, I think that it a safe assumption that there are alternatives to HS2 that represent much better value for money for the taxpayer, and any review of the HS2 project should bear this in mind.

(To be continued …)

Footnotes:

  1. See paragraph 6.1.3 of the report Economic Impact of High Speed 1, Final Report, London and Continental Railways, January 2009.
  2. See the final paragraph on page 140 (Section 6.4.1) of the report First Interim Evaluation of the Impacts of High Speed 1, Final Report, ATKINS, AECOM and Frontier Economics, October 2015. The publication of this report was originally promised for summer 2013, and its delay was criticised by the PAC.
  3. See paragraph 23 in the report The Sale of Eurostar, House of Commons Committee of Public Accounts Sixteenth Report of Session 2015-16, January 2016.
  4. See paragraph 366 in the report The Economics of High Speed 2, House of Lords Economic Affairs Committee, 1st Report of Session 2014-15, March 2015.
  5. See paragraph 10 in The Economics of High Speed 2.
  6. See paragraph 2.8 of the report HS2 Outline Business Case Economic Case, Department for Transport, March 2014.
  7. See paragraph 2.6 of HS2 Outline Business Case Economic Case.
  8. See paragraphs 5.4 to 5.7 of HS2 Outline Business Case Economic Case.
  9. See Figure 12 in the report Progress with preparations for High Speed 2, National Audit Office, June 2016.
  10. See paragraph 418 in The Economics of High Speed 2.
  11. The source of Mr Fairhead’s figures is Table 5.8 in the report High Speed Rail Strategic Alternatives Study Update Following Consultation, Atkins/Department for Transport, January 2012.

 

 

Let’s be courageous, part 4

(… continued from Let’s be courageous, part 3, posted on 20 Aug 2016).

In Section Two of his Taxpayer’s Alliance briefing paper, Rich man’s toy: The case for scrapping HS2, Policy Analyst Harry Fairhead examines the way that the business case for HS2 has been set out by the Department for Transport (DfT).

The aim of the work on the business case that has been undertaken by HS2 Ltd, on behalf of the DfT, is to “consider whether all of the collective impacts (including on existing transport networks) delivered by [the HS2 project] represent good value for taxpayers’ money” (see footnote 1). We are told that the work “adheres to the general guidance on evaluating proposals published by H M Treasury in the Green Book” and also follows “the more detailed advice” in the DfT’s own WebTAG guidelines that give instruction on “how to apply Green Book principles to transport investments” (see footnote 2).

This method “aims to capture all of the impacts – positive and negative – as well as the associated risks and uncertainty” associated with the project proposal and, “where possible”, to express these impacts “in units of money”. The output of this calculation for a particular set of assumed circumstances is a single figure, termed the benefit cost ratio, or BCR (see footnote 3).

Many of you will be familiar, possibly very familiar, with the concept of BCR, but just in case you are not, I will attempt a summary.

The analysis is undertaken for the whole life of HS2, which is assumed to be sixty years after service opens although it seems reasonable to expect, perhaps, that HS2, like its Victorian counterparts, will well exceed this life expectancy. The net present value of the monetary equivalent of direct benefits is totalled and the same is done for the net costs, this being the sum of capital and operating costs less the operating revenue: the BCR is the ratio of the value of benefits to costs. The method offers the option of adding the assessed value of “wider economic impacts” (WEI) to the benefits to modify the BCR: although, in theory, the WEI can be negative taking them into account will usually, as is the case with HS2, improve the BCR (see footnote 4).

In the early stages of the HS2 project the BCR was almost invariably quoted for both excluding and including WEIs, although the practice in recent times appears to favour using the higher value that results from taking WEIs into account. According to the National Audit Office (see footnote 5), the latest appraisals yield a BCR of 1.7 (1.4 excluding WEI) for Phase 1 and 2.2 (1.8 excluding WEI) for the full network (incorporating Phase 2a). According to WebTAG, this puts Phase 1 alone in the low to medium value for money category and the full network in the medium to high value for money bracket (see footnote 6).

According to the DfT (see footnote 7):

“WebTAG has been developed over many years and has benefited greatly from the UK’s long tradition of applying cost benefit analysis to transport infrastructure investment proposals. Comparisons show that the UK appraisal system compares very well with those in other countries and the UK has led the world in setting out its guidance on analysing the impact of proposals in an open and transparent way.”

In contrast, Mr Fairhead reflects that the HS2 business case has “received significant criticism”, that the methodology “has been accused of being simplistic” and that “there are inherent flaws in the assumptions” that have been made. He notes that an early critique of the business case “has suggested that the BCR for Phase 1 of HS2 could be as low as just 0.5” (see footnote 8).

There are two important features of the BCR methodology that has been employed for the HS2 business case that render the result wide open to criticism. In the first place, the calculation requires forecasting for a period that extends up to seventy years from today; this is just an impossible task, and puts the process well into fairyland. No matter how conservative an approach appears to have been taken, what Harold Macmillan is supposed to have described as “events, dear boy, events” (see footnote 9) will inevitably come along to undermine the best of forecasts. Who, for example, would have, five years ago, foreseen the UK leaving the European Union, and the consequent impacts upon the economic prospects for our country? Who, even today, is sure what those impacts will be?

The second ground for scepticism is that the process of assessing the impacts and evaluating them in monetary terms is, despite being presented as a scientific procedure, very much a matter of subjectivity. Mr Fairhead particularly identifies the monetary benefit attributed to journey time savings by business travellers, which he reports account for around one-half of the total transport user benefits, assessed at “just over £40 billion”. He claims that the DfT’s analysis is deficient in assuming that journey time saved is used in the workplace, rather than for leisure, and that business passengers are not productive while travelling: and he is far from alone in making these criticisms.

I fear that what we may have witnessed as the HS2 business plan has been developed and refined is a phenomenon that is the benefit equivalent of the cost optimism bias effect that I discussed in part 2 of this blog series, where those promoting HS2 may be, with the intention of defending the project, regarding the contribution to the BCR by some identified benefits as greater than a truly independent assessor would rate them: in my blog A work of fiction (posted 8 Sep 2012) I referred to this as producing “a more congenial result”.

(To be continued …)

Footnotes:

  1. See paragraph 2.4 of the report HS2 Outline Business Case Economic Case, Department for Transport, March 2014.
  2. See paragraph 1.1 of HS2 Outline Business Case Economic Case.
  3. See paragraph 1.5 of HS2 Outline Business Case Economic Case.
  4. For an explanation of WEIs see section 2 of the paper, Using the impacts of active traffic management rollout project to discuss wider economic benefits in transport appraisal, Bose R, Kohli S and van Vuren T, European Transport Conference 2008, Noordwijkerhout, October 2008. This paper defines WEIs as “benefits that are from accessibility improvements in the transport markets and accrue in form of productivity gains due to agglomeration effects, increased outputs in markets with imperfect competition and improvements in labour supply”.
  5. See Figure 11 in the report Progress with preparations for High Speed 2, National Audit Office, June 2016.
  6. See Table 17 in HS2 Outline Business Case Economic Case.
  7. See paragraph 1.3 of HS2 Outline Business Case Economic Case.
  8. Mr Fairhead’s comments can all be found in Section 2 of Rich man’s toy: The case for scrapping HS2. The business case critique to which he refers is Review of the Economic Case for HS2 Economic evaluation London – West Midlands link, Castles C and Parish D, RAC Foundation, November 2011. The revised Phase 1 BCR may be found in Table 1.
  9. According to the Daily Telegraph article, As Macmillan never said: that’s enough quotations, it has never been reliably established when he said this and to whom, assuming, of course, that he ever actually said it.