Hi,
No references, sorry, just a question. If one were to ask metereologists to present evidence that they produced correct forecasts, what would be acceptable? How would that differ from what we might consider acceptable for economic or policy forecasting?
At a recent talk by a metereologist I was struck by a) the fact that everyone from the military to someone walking kids to school relies on their forecasts but that b) it's as complex a system as you could hope for. One way they deal with this is through producing a broad range of thumbnail predictions given slightly different starting parameters - the job of interpretation of these is then a human one.
I think it's a useful comparison: no-one expects metereology to give anything but broad brushstroke predictions, because of the nature of the system they model. After about 7 days out, all bets are off - though some boundary conditions can be predicted for long-range forecasts. (and with e.g. climate change.) The question then is: what do are we willing to accept as a useful set of predictions (rather than strictly correct.)
Clearly, what's valid in metereology probably won't be for other forms of modelling. Interestingly, though, the person giving the talk told me they don't even like to talk about validation. I'd like to find out more about that, given how obsessively everyone in agent-based modelling is pursuing this enigmatic beast.
Anyway, my point was: one could ask metereologists to hand over examples of correct forecasts, and they could do this - since forecasts are right at least some of the time. But the more appropriate question is whether they did better than pulling forecasts out of a hat.
Got some more thoughts on this but gotta go...
Cheers,
Dan
|