It would be useful to review the success of past forecasts - what's the distribution of errors around forecasts of GDP and GDP change, 1 year ahead and 10 years ahead? The same for unemployment and other key indicators used in public policy. Unlike weather, the economy is not forecast frequently enough for there to be a great deal of data to go on. But perhaps it's already been done, enough to know whether the uncertainty is sufficiently stable to use to judge new forecasts.



But a claim like 'Brexit with a bilateral agreement between UK and EU would reduce GDP by £4,300 per household' is a conditional forecast. The success of past forecasts would only show the uncertainty around things not determined by the Brexit condition. The assumptions made about what Brexit would involve are fixed by the forecaster and would take further unpicking: argument over their validity would add more uncertainty, perhaps a lot more.



There may have been a lot of work done on this already. If anyone knows it and could consider turning it into practical advice for evaluating claims, it would make a good RadStats article - or a chapter in the RadStats books.



From my experience in population forecasts, there are key needs where uncertainty matters a lot (school rolls; town planning; adult care), and there are quite a lot of data in the UK. But very few evaluations. 



Ludi



------------------------------



Date:    Wed, 20 Apr 2016 17:10:56 +0100

From:    John Bibby <[log in to unmask]>

Subject: Fwd: Letter: "All forecasts are wrong. But some are more wrong than others" (Derek Jerram,"Forecasting is art, not science", Letters in "i", 2016april20, p.14.



Please see email below - all comments welcome!  JOHN BIBBY



---------------------------------------------------------------------------



You are invited to the following events that I am co-organising in York:



   - "Yorkshire, World War 1 & the Middle East

   <http://www.eventbrite.co.uk/e/yorkshire-world-war-1-the-middle-east-community-history-workshops-tickets-24106292570?aff=ehomesaved>"

   (*26 April* - Community History *FREE *Events)

   - " 'Bonjour M. Sykes!', 'Ca va, M. Picot' " (*16 May* - Sykes-Picot

   1916-2016)

   - *Borders and Beyond in the Middle East since 1914

   <http://www.yorksj.ac.uk/news---events/news---events-home/events/borders-and-beyond-in-the-midd.aspx>

   (*"BABITME", *17-18 June) *: WW1, Yorkshire family connections & others,

   implications for today, migration, nationalisms, religion, etc., etc..:

   www.tinyURL.com/BABITME <http://www.tinyurl.com/BABITME>





---------- Forwarded message ----------

From: John Bibby <[log in to unmask]>

Date: 20 April 2016 at 17:09





All forecasts are wrong, but some are more wrong than others. Forecasting should not lead to a "precise" number or "point forecast" - like the recent Treasury £4300 in their research on leaving the EU. Instead, it should lead to a "50% confidence interval forecast", such as £4000-£5000 - the interval being calibrated so that the researcher is 50% confident that the forecast is correct. This has the advantage of falsifiability - if an accurate forecasting method is used, exactly 50% of forecasts will be true; the other 50% will be false. (Alternatively, forecasters may choose a wider interval if they require a higher level of confidence e.g. £3000-£6000 with 90% confidence.)



My forecast - most forecasters will not listen to this, because it gives us a way of distinguishing the fraudulent forecasters from the genuine ones.



JOHN BIBBY  (01904-330334)

1 Straylands Grove

York YO31 1EB





******************************************************

Please note that if you press the 'Reply' button your

message will go only to the sender of this message.

If you want to reply to the whole list, use your mailer's

'Reply-to-All' button to send your message automatically

to [log in to unmask]

Disclaimer: The messages sent to this list are the views of the sender and cannot be assumed to be representative of the range of views held by subscribers to the Radical Statistics Group. To find out more about Radical Statistics and its aims and activities and read current and past issues of our newsletter you are invited to visit our web site www.radstats.org.uk.

*******************************************************