How accurate should the predictions be?
By far the worst prediction was the first year, 2013, when the median was 3.32 and the melt season ended at 5.21 so off by 1.89.
Was the forum particularly alarmist in 2013? Or did it instead have something to do with the trend of preceding seasons, dropping from 4.56 in 2011 to 3.57 in 2012 - a drop of 0.99.
So the median prediction, after a drop of 0.99 in 2012, was to drop only a further 0.25 in 2013.
Imo that was not so unreasonable given the information available - and yet it was the worst prediction. Imo that was Nature's fault more than the fault of the participants in the poll.
If someone wants to make a counter-argument that it was the participants fault then they would do well to explain how the 5.21 result could have been known in advance in 2013, or at least a good approximation to that. Good luck!
So imo it's not a case of having been chastened out of alarmist tendencies. Instead, the main reason we didn't, this year, pick as low as in 2013 is the additional physical information available to us from the intervening years.
We still might anyway be off by a lot this year - Nature simply isn't that easy to predict here!
Whether being out by 1.89 is a bad result or not, depends in the confidence limits put on the prediction. If you had confidence limits of 0.1, then its really awful to be out by 1.89, and if you had confidence limits of 2.0 it isn't.
I don't think the median is a good measure for judging poll accuracy. I think there are three different populations represented in the poll, and only one of them is interested in how close their guess is (the others being only concerned about whether its a record or not, and whether its a blue ocean or not). This shows up as a multi-modal distribution and I'd take the highest mode as the indicator of where those who are interested in accuracy are making predictions.
In the case of 2013 this mode is 7 bins away from the outcome, which my challenge would score at -10, -6, -4, -2 and 0, depending on expressed confidence.
In the case of 2014 and 2017 this mode is 5 bins away from the outcome, for scores of -10, -6, -4 , -1 and 0.
In 2015 and 2016 its just 2 bins out for scores of -10, -2, 1, 1, 0.
Note that I am taking the mode by eye off Steven's post above and several of those years might well be judged a bin closer if the original data was used and it was in the same format as this years polls.
Overall, I score having very high confidence in the forum poll -50, high confidence -22, medium confidence -10, low confidence -2 and no confidence 0. It would seem Neven's lack of confidence in his entry has something to recommend it.
The 2013 polling was heavily influenced by looking at an exponential trend. Those that voted based on extrapolating an exponential trend should have been aware that these extrapolations have very high error margins and put a wide margin of confidence on their prediction. I think 3.5 was tenable in the June 2013 poll, but ruling out 5.2 wasn't, and 3.5 was no longer tenable by July 2013.
Poll