(… continued from Let’s be courageous, part 13, posted on 15 Oct 2016).
The problem with employing computer models is that we tend to regard them, quite unjustifiably, as authoritative, and the results that they produce as definitive, so it is probably prudent to exercise a degree of scepticism about the passenger demand predictions for HS2 that have been produced by the PLANET Framework Model (PFM). To do this is not to deny that the PFM is a powerful tool that is the result of much research and considerable development – a fact that even the most cursory of glances at the tome that describes the model will verify (see footnote 1) – but it does acknowledge the potential flaw in any such modelling that is summarised in an acronym that has been in use in the computer industry since the 1950s, “GIGO” (see footnote 2).
Another problem that I have with the HS2 demand modelling is getting to grips with assessing whether the underlying assumptions are reasonable. This is not because a veil of secrecy has been drawn over the workings at the heart of the PFM – after all pages and pages of description have been published – but there is, I find, a mist of unfathomability, which is the result of essential information either being missing (see footnote 3) or buried within an impenetrable overabundance of text. A further complicating factor is the near impossibility of predicting the future, in the light of the complex interaction of factors that is our economy and the rapid, and escalating, pace of technological development.
So it is, I think, difficult to claim definitively that HS2 Ltd is wrong about particular aspects of the demand prediction: I appear to share this predicament with the Economic Affairs Committee of the House of Lords (EAC), who found it “difficult to assess the plausibility” of the predictions (see footnote 4). One possible way around this stumbling block is to concentrate on the outputs, rather than the inputs, and assess how they compare with the actuality. Unfortunately, we can’t do this with HS2, as it hasn’t been built yet, but we can, perhaps, look at some high-speed railways that are in operation and see how well the analysts did in predicting demand for them. There are, of course, no currently-operational, high-speed city-to-city links in the UK, so it is necessary to look overseas, and this is precisely what the EAC did.
The EAC report choses France, a country with a wealth of experience of high-speed train services, to test just how reliable the original demand predictions made there have proved to be for lines that have been in operation for at least twenty years. The authoritative source that the EAC employed is authored by the Cour des Comptes, which, the EAC informs us is “the French equivalent of our National Audit Office”. This source advises that, of six routes that were assessed, the “traffic was on average 24 per cent lower than predicted” (see footnote 5) and that only the Paris to Lyon route was enjoying passenger demand that was higher than had originally been predicted. Traffic levels on the remaining five lines were all lower than predicted with the worst case, LGV Nord, only reaching “half the forecast level of traffic” (see footnote 6).
So the French forecasts proved, on the whole, to be over-optimistic, but is it fair to infer that this implies that UK forecasters suffer from a similar sanguinity? The only high-speed railway currently in operation in the UK is, of course, HS1, and this is somewhat different in its market to HS2, being international and commuter traffic rather than long-distance intercity. Putting aside this difference, the actual passenger demand for HS1 indicates that there could well be a similar optimistic streak at work amongst UK transport analysts. In 2012 the National Audit Office reported that actual passenger numbers “have proved to be two-thirds” of forecasts by the Department for Transport (DfT) in 1998. These forecasts were a downgrading of the original 1995 forecast made for the purposes of the PFI bid by London and Continental Railways (LCR): this original bid estimated demand at approximately double the DfT’s downgraded forecast, so was three times what has actually been achieved (see footnote 7).
In oral evidence given to the Public Accounts Committee of the House of Commons (see footnote 8) the Permanent Secretary of the DfT, Philip Rutnam, admitted that “demand was overestimated back in the 1990s” but attributed this to a lack of knowledge about the market for “what was a wholly new form of travel” in the form of international rail travel. He reassured the Committee that, in the case of domestic rail travel that is the market for HS2, his department has “literally tens of thousands, if not millions of data points, and a sophisticated system of demand forecasting that has developed over several decades into industry standards”. I believe that, as a top Whitehall mandarin, Mr Putnam is entrusted with acting in the very best interests of the taxpayer, of which I am one. I would therefore like to take this assurance at face value to assuage my doubts. However, one other thing that he said on that occasion rings alarm bells: he gave as one reason for the overestimation of the demand for HS1 services as “the impact of low-cost aviation and the impact that that has had on market structure”, which, by implication, was not taken into account in the demand modelling. In part 13 I observed that “the HS2 business model appears to have put HS2 in a bubble where it will not be subject to market forces” – it looks, perhaps, that history may be repeating itself!
(To be continued …)
- The “tome” referred to is PLANET framework model (PFM v4.3) Model description, HS2 Ltd/Department for Transport, October 2013.
- Garbage in, garbage out: a pithy reminder of the reliance on models of utilising accurate, or realistic, input data and assumptions in order to produce reliable predictions. The undue deference that we have for computer models, and unwarranted reliance that we place on their outputs, is summed up by the variance on the expansion of the GIGO acronym, “garbage in, gospel out”.
- An example being, as I reported in part 13, that we have not been advised the number of passengers the model assumes will use HS2.
- See paragraph 109 in the report The Economics of High Speed 2, House of Lords Economic Affairs Committee, 1st Report of Session 2014-15, March 2015.
- This 24 per cent over-estimation is quoted by Harry Fairhead as evidence that HS2 underperforming its predicted passenger demand is “a very real possibility” in the subsection Demand for HS2 on page 8 of his briefing paper Rich man’s toy: The case for scrapping HS2, Taxpayer’s Alliance, July 2016.
- The original French language comments may be found on page 95 of the report La Grande Vitesse Ferroviare: Un modèle porté au-delà de sa pertinence (High Speed Railways: A model carried beyond its relevance), Cour des Comptes, October 2014. My report is based upon the English translations in paragraph 117 of The Economics of High Speed 2. The four French TGV links that are not mentioned by name are: Atlantique (Paris-Western France), Interconnexion Ile de France, Rhône Alpes and Méditerrané.
- See paragraph 1.15 in the report The completion and sale of High Speed 1, National Audit Office, March 2012. The failure to meet the inflated passenger demand estimates was the reason that LCR ran into major financial difficulties in 1998.
- See Q88 in the transcript Oral evidence: The sale of Eurostar, HC 564, House of Commons Public Accounts Committee, Wednesday 18thNovember 2015.