Hi Ned, that is still a remarkable track record.
Are you using only 'extent' as your predictor ?
And did the fact that 2017 'extent' closely followed the "2010's average" since May play a role in the remarkable accuracy of your "predict-o-matic" ?
Also, next year, do you want to enter your prediction for the SIPN ?
https://www.arcus.org/sipn
Thanks for the note, Rob.
The name "predict-o-matic" was kind of a joke, but I decided I liked it. It is basically an overcomplicated way of doing something very simple. People keep asking about it, so I guess I should explain.
The goal is to make predictions about extent during the melting season. The core methods that it uses for this are ridiculously straightforward and other people here use them too, but in the predict-o-matic they're wrapped in a very sophisticated and fancy interface.
There are two versions. Both are based on the historical extent data from 2007-present (one could use pre-2007 data but I vaguely feel like some kind of radical threshold was crossed in 2007 so I start there).
Boring details inside the quote box, with more interesting discussion below...
V1: For any day t0, compute the extent change dE(year,t0,min) for every year from day t0 to that year's minimum, regardless of on what day that minimum occurs. Calculate the mean, standard deviation, etc. of those extent changes dE. Subtract the mean change from the current year's extent on day t0 to predict the minimum, and use the standard deviation to calculate various probabilities (see below).
V2: For any day t0, compute the average extent change dE(year,t0,tn) to day t1, t2, t3, ... tn in each previous year, and calculate the mean and standard deviation of dE on each day. Subtract the mean change for each day t1 ... tn from the current year's extent on day t0 to predict the evolution of extent over time, and use the standard deviation to calculate various probabilities.
Basically, version 1 gives the average of the minimums, while version 2 gives the minimum of the averages. Those are different. Version 1 always gives a slightly lower minimum (e.g., 4.14 vs 4.21) but version 2 forecasts the day-by-day evolution of extent, which version 1 doesn't do.
A not fully implemented version 3 combines the previous two, to give both the evolution over time plus the actual expected value and date of the minimum.
I know other people here do the same kind of calculations (hello, gerontocrat!). The nice thing about the predict-o-matic is that it's fully automated and generates a large number of outputs, including:
* Expected value of the minimum, plus a 95% confidence interval
* Expected value on every day of the season, plus a 95% confidence interval
* Projections of the minimum if the current year followed the trajectory of each previous year
* Probability values for each of the "bins" in the monthly JAXA extent polls
* Probability values for "ranks" relative to previous years
* Live-updating graphs for all of the above
* Day-by-day accuracy statistics of the initial ("static") forecast from the start of the season
The live updating graphs are really helpful to me in visualizing the probabilities. I've posted examples of the graphs here; scroll up-thread to see what they look like.
So the fundamental process is embarrassingly simple but the interface is nice.
You're absolutely 100% correct that the "success" (
) of the predict-o-matic this year is entirely due to the bizarrely average evolution of extent over the course of this melt season. I wish I could claim credit for continuous, uninterrupted 95+ percent accuracy over the past nine weeks, but I can't.
Another secret is that it's all part of an amateur psychology experiment. We on the ASIF have a really poor record of prediction on the polls (see
this thread) -- like an iceberg, the center of mass of our predictions tends to be way below the waterline. Time after time our monthly polls on daily or monthly extent or area end up much much too low. I thought that providing a bunch of posts with eye-catching graphics explaining the probabilities, during the time when people are making decisions on the polls, might help bump the distribution of poll responses towards more realistic numbers. Dunno if it had any effect.
Looking ahead, I have a bunch of ideas to improve the extent predict-o-matic, by incorporating other sources of information. But my first priority is something Jim Hunt asked about at the top of this page -- a daily-updating PIOMAS volume predict-o-matic. That would be much more useful, and I've got a plan for how to do it.
Here's one of the output graphs I haven't posted lately -- it shows how the V1 predictions (the "lower" one) have evolved over the past ten weeks:
The predicted minimum has ranged from a low of 4.02 to a high of 4.41 (that's today's prediction -- yes, it's the highest one of the season).