Please support this Forum and Neven's Blog

Author Topic: Accuracy of poll predictions  (Read 1466 times)

Ned W

  • ASIF Citizen
  • Posts: 151
    • View Profile
Accuracy of poll predictions
« on: June 15, 2017, 09:27:31 PM »
In the thread for the June poll for the IJIS/JAXA daily minimum, there has been some discussion about the "accuracy" of our collective predictions.  I went back and looked at all of the June/July/August polls for the NSIDC monthly and CT or IJIS daily minimums, from 2013-2016 ... that's 24 polls.

First up, here are the distributions of responses by how they ended up (too high, correct, or too low). 

 



In 23 of the 24 cases, a majority of poll responses were on the "too-low" side.  The one exception is the August 2014 poll for CT Area daily minimum. 

Clearly recent events can have a big effect.  In 2013, the first year after the big 2012 record, every single participant in the poll erred on the too-low side, for four of the six polls that year.

Here are graphs comparing the median prediction from each poll to the final (observed) value for that year:





Overall, we do better in August than in June (duh!).  For June polls, our best results were in 2015.

And there's a very strong bias towards under-predicting ice extent.

Neven

  • Administrator
  • ASIF Governor
  • *****
  • Posts: 3847
    • View Profile
    • Arctic Sea Ice Blog
Re: Accuracy of poll predictions
« Reply #1 on: June 15, 2017, 10:55:53 PM »
Thanks for this, Ned.

It'd be interesting if we could give the bias a number, and then use that to predict the minimum. ASIF prediction + ASIF average bias = X.  ;D
Il faut cultiver notre jardin

Richard Rathbone

  • ASIF Citizen
  • Posts: 340
    • View Profile
Re: Accuracy of poll predictions
« Reply #2 on: June 16, 2017, 12:49:25 AM »
Thanks for this, Ned.

It'd be interesting if we could give the bias a number, and then use that to predict the minimum. ASIF prediction + ASIF average bias = X.  ;D

Post 2013 I use it as a sanity check. If my prediction isn't rather higher than the forum, I need to doublecheck my reasoning and the indicators I am using.

slow wing

  • ASIF Middle Class
  • Posts: 551
    • View Profile
Re: Accuracy of poll predictions
« Reply #3 on: June 16, 2017, 01:42:49 AM »
X would be an estimate of the bias only and probably not a very good one as it's only 4 years of data and Mother Nature has had a big say.

  The actual bias could be defined as the difference between what the forum votes and what some idealised observer with essentially perfect knowledge of the physics would vote.

  The trouble is, the weather fluctuates away from the climate mean so the idealised observer also won't predict the exact answer each year. (It's a chaotic system so the fluctuation is not predictable even with essentially perfect knowledge of the physics.)

  2013 was a rebound year. Was that mostly physics or fluctuation? Given the resumption of the downward trend culminating in the record low maximum volume this year, I suspect the latter. So even the idealised observer would have voted high[EDIT]low - perhaps in all 6 polls for 2013.

  For the other 3 years, its a more difficult question whether the fluctuation was upwards or downwards.

  In my view we can't even rule out that the idealised observer would have voted high[EDIT]low on the other 3 years as well - and maybe in 23 out of the 24 polls. In that case, there would be no evidence whatsoever for bias in the forum.

  It's a shame we don't have polls from 2012 as well. I suspect 2012 was a downwards fluctuation and so the idealised observer would have voted high in 2012, maybe in all 6 polls.

  Maybe the forum would have voted high as well in most or all of the 6 hypothetical polls for 2012. That would have given a very different look to the graphs in the opening post. Does anyone remember if the forum was generally thinking higher or lower than the final numbers for 2012?
 
  In summary, it's easy to over-interpret the graphs in the opening post and difficult to establish bias from only 4 years data in a complicated and chaotic system.
« Last Edit: June 16, 2017, 08:56:54 AM by slow wing »

Ned W

  • ASIF Citizen
  • Posts: 151
    • View Profile
Re: Accuracy of poll predictions
« Reply #4 on: June 16, 2017, 04:13:58 AM »
Thanks for this, Ned.

It'd be interesting if we could give the bias a number, and then use that to predict the minimum. ASIF prediction + ASIF average bias = X.  ;D
That would be interesting.  But as slow wing says it's a small sample of noisy data.  And there are oter problems:

1. Hopefully, over time we get better at predictions, so the bias should shrink. In fact, the act of observing this and speculating about it might stimulate a change in the bias.    ;)   So the bias is non stationary and the act of observing it renders the observations obsolete.

2. Unfortunately, by the time we get a large enough sample size (say 2030s or so) there will probably be "ice-free" days in most Septembers ... making it easy to reliably predict the daily min.  So the whole thing becomes moot.

3. While waiting for that to happen, characterizing the bias requires being able to compare predictions across multiple years.  But that's hard to do when people change the format of the poll (... cough NSIDC cough OVERLAPPING BINS cough JUNE 2017 cough ...)

Neven

  • Administrator
  • ASIF Governor
  • *****
  • Posts: 3847
    • View Profile
    • Arctic Sea Ice Blog
Re: Accuracy of poll predictions
« Reply #5 on: June 16, 2017, 10:07:04 AM »
3. While waiting for that to happen, characterizing the bias requires being able to compare predictions across multiple years.  But that's hard to do when people change the format of the poll (... cough NSIDC cough OVERLAPPING BINS cough JUNE 2017 cough ...)
Is that really a problem? The overlapping bins, I mean. You still get an average and a median, etc, right?

If it is, I'll do regular bins for the next poll, and you'll just have to homogenize or extrapolate, or apply some other statistical techniques, for this one poll, Ned.  ;)

Maybe the forum would have voted high as well in most or all of the 6 hypothetical polls for 2012. That would have given a very different look to the graphs in the opening post. Does anyone remember if the forum was generally thinking higher or lower than the final numbers for 2012?
The Forum started in 2013, so no polls for 2012 there. But I believe I ran polls on the blog. If I have time later today, I'll go and check.
Il faut cultiver notre jardin

EgalSust

  • ASIF Lurker
  • Posts: 17
    • View Profile
Re: Accuracy of poll predictions
« Reply #6 on: June 16, 2017, 01:31:45 PM »
Thanks for the effort, Ned W!

For comparison, here are some results from Sea Ice Prediction Network:


Figure 13: June, July, and August 2016 SIO contributions of the September pan-Arctic sea ice extent as box plots, broken down by type of method. Boxes show medians and interquartile ranges. Colors identify method types, and n denotes the number of contributions. Individual boxes for each method represent, from left to right, contributions to the June, July, and August SIO. The heavy gray line shows the 2016 observed September sea ice extent from the NSIDC index (as in Figure 1). Figure updated from Hamilton and Stroeve (2016). (source)


Figure 7. Distributions of June, July, and August 2015 Outlook contributions as a series of box and whisker plots, broken down by general type of method. The box color depicts contribution method with the number below indicating total number of contributions by method over the three months. The individual boxes for each method represent, from left to right, each month of June, July, and August. (source)


Figure 5. Distribution of Pan-Arctic Outlook values (August Report) for September 2015 extent. (source same as above)


Figure 3. Box plots show the median, interquartile range, highest, and lowest values of sea ice outlook contributions from June, July, and August, broken down by type of method used. As a group neither the statistical nor modeling methods shows a clear advantage, but some contributions from both groups came close to the observed mean September extent. (source)


Figure 2. Distribution of individual Pan-Arctic Outlook values (August Report) for September 2014 sea ice extent compared to the observed value. Labels on the bar graph are rounded to the tenths for readability. Refer to the Individual Outlooks at the bottom of this report for the full details of individual submissions. (source same as above)


Figure 5. Distribution of 2013 Sea Ice Outlook predictions by method and month, including the August 1 cycle which consists of an informal update. Box plots show median and approximate interquartile range (middle 50%) of each group. The observed mean September 2013 extent, 5.35 million km2, is marked by a horizontal line. Median SIO predictions from all three methods shifted upwards in the August 1 cycle, reflecting adjustments to mid-summer conditions but also changing contributor pools. A more detailed analysis with comparisons to earlier years and other datasets has been drafted by Stroeve and Hamilton (in review), and will be summarized in a later post. (source


Figure 4a. Distributions of Outlook estimates for September 2013 arctic sea ice extent for the June report (based on May data). (source: 2013 post-season report, linked above)


Figure 4b. Distributions of Outlook estimates for September 2013 arctic sea ice extent for the July report (based on June data). (source as above)


Figure 3a. Distributions of Outlook estimates for September 2012 arctic sea ice extent for the June report (based on May data). (source)


Figure 3b. Distributions of Outlook estimates for September 2010 arctic sea ice extent for the July report (based on June data). (source as above)

Late Andrew Slater (R.I.P.) has made some comparisons of August predictions:


At least two out of the last three years (source)


The last two years (source as above)


Three out of the last three years (source as above)

magnamentis

  • ASIF Upper Class
  • Posts: 1119
    • View Profile
    • Nexpaq Modular ARA iOS Software Mobile Computing Phones Philosophy Ethics Numerology Mikrocirkulation Vaskular Therapie Gesundheit Blut Gesundheit Schmerzen Multipelsklerose Diabetes Immunsystem Fibromyalgie
Re: Accuracy of poll predictions
« Reply #7 on: June 16, 2017, 07:04:51 PM »
Thanks for this, Ned.

It'd be interesting if we could give the bias a number, and then use that to predict the minimum. ASIF prediction + ASIF average bias = X.  ;D

chances are high that if we do that this year things will be the other way around which would see for a huge bias towards the other end. i'm confident that this year could even run the predictions out on the lower side and even more confident that the majority is at or in the vicinity of being spot on.
http://magnamentis.com
Knowledge, Understanding & Insight Are Among The Best Sources For Personal Freedom & Vitality !

Steven

  • ASIF Citizen
  • Posts: 360
    • View Profile
Re: Accuracy of poll predictions
« Reply #8 on: June 17, 2017, 02:34:22 PM »
Here are graphs comparing the median prediction from each poll to the final (observed) value for that year:




Thanks.  I modified your graph to include the median predictions by the sea ice prediction network, indicated by the red horizontal line segments:



Chuck Yokota

  • ASIF Lurker
  • Posts: 78
    • View Profile
Re: Accuracy of poll predictions
« Reply #9 on: June 17, 2017, 05:24:07 PM »
I tend to vote pessimistically, as I would rather be pleasantly surprised than the opposite.

Feeltheburn

  • ASIF Citizen
  • Posts: 108
    • View Profile
Re: Accuracy of poll predictions
« Reply #10 on: June 17, 2017, 11:39:09 PM »
I tend to vote pessimistically, as I would rather be pleasantly surprised than the opposite.

That is commendable. <snip, removed the remainder for unnecessary belligerence; J.>
« Last Edit: June 17, 2017, 11:55:27 PM by Jim Pettit »
Feel The Burn!

Juan C. García

  • ASIF Citizen
  • Posts: 431
    • View Profile
Re: Accuracy of poll predictions
« Reply #11 on: June 20, 2017, 04:31:48 AM »
Have you think in the accuracy in which NSIDC makes their monthly averages?
Do you agree that:
Arctic sea ice extent during September 2016 averaged 4.72 million square kilometers (1.82 million square miles), the fifth lowest in the satellite record.
http://nsidc.org/arcticseaicenews/2016/10/


From my point of view, it is completely wrong to convert a 15%+ average on the month to make it 100% on a given grid.

They should -instead- make the statistical average of their daily values (they can use their "one daily value" or the "5 day moving average").

So, part of the inaccuracy that this Forum has, comes from the strange way the NSIDC makes their monthly average (that is not an average at all).
 
Which is the best answer to Sep-2012 ASI lost (compared to 1979-2000)?
50% [NSIDC Extent] or
73% [PIOMAS Volume]

Volume is harder to measure than extent, but 3-dimensional space is real, 2D's hide ~50% thickness gone.
-> IPCC/NSIDC trends [based on extent] underestimate the real speed of ASI lost.

Juan C. García

  • ASIF Citizen
  • Posts: 431
    • View Profile
Re: Accuracy of poll predictions
« Reply #12 on: June 20, 2017, 04:49:52 AM »
Arctic sea ice extent for December 2016 averaged 12.10 million square kilometers (4.67 million square miles), the second lowest December extent in the satellite record. This is 20,000 square kilometers (7,700 square miles) above December 2010, the lowest December extent,...
http://nsidc.org/arcticseaicenews/2017/01/


 :o  :o  :o  :P
Which is the best answer to Sep-2012 ASI lost (compared to 1979-2000)?
50% [NSIDC Extent] or
73% [PIOMAS Volume]

Volume is harder to measure than extent, but 3-dimensional space is real, 2D's hide ~50% thickness gone.
-> IPCC/NSIDC trends [based on extent] underestimate the real speed of ASI lost.

Juan C. García

  • ASIF Citizen
  • Posts: 431
    • View Profile
Re: Accuracy of poll predictions
« Reply #13 on: June 20, 2017, 04:55:12 AM »
Because it's still so early, I'm going to wait the full 10 days before voting. Right now, I have no idea which way this could go.  :)

Between 3.25 and 3.75 million km2.
A little lower than 2012. Just to vote for a new record (given the low volume that we have now) and because I know that I will not know, 10 days after today...  8)

I was only joking, because I do not believe in the accuracy of the NSIDC Monthly Average at all...  8)

I definitely prefer to bet on an IJIS or NSIDC daily value  ;D
Which is the best answer to Sep-2012 ASI lost (compared to 1979-2000)?
50% [NSIDC Extent] or
73% [PIOMAS Volume]

Volume is harder to measure than extent, but 3-dimensional space is real, 2D's hide ~50% thickness gone.
-> IPCC/NSIDC trends [based on extent] underestimate the real speed of ASI lost.

Juan C. García

  • ASIF Citizen
  • Posts: 431
    • View Profile
Re: Accuracy of poll predictions
« Reply #14 on: June 20, 2017, 05:45:14 AM »
Have you think in the accuracy in which NSIDC makes their monthly averages?
Do you agree that:
Arctic sea ice extent during September 2016 averaged 4.72 million square kilometers (1.82 million square miles), the fifth lowest in the satellite record.
http://nsidc.org/arcticseaicenews/2016/10/



Year    Month    Day        Extent
YYYY       MM     DD    10^6 sq km
2016   9   1   4.278
2016   9   2   4.211
2016   9   3   4.263
2016   9   4   4.258
2016   9   5   4.201
2016   9   6   4.203
2016   9   7   4.083
2016   9   8   4.104
2016   9   9   4.137
2016   9   10   4.16
2016   9   11   4.209
2016   9   12   4.236
2016   9   13   4.341
2016   9   14   4.357
2016   9   15   4.378
2016   9   16   4.363
2016   9   17   4.309
2016   9   18   4.434
2016   9   19   4.495
2016   9   20   4.533
2016   9   21   4.593
2016   9   22   4.627
2016   9   23   4.814
2016   9   24   4.899
2016   9   25   4.916
2016   9   26   5.071
2016   9   27   5.143
2016   9   28   5.174
2016   9   29   5.175
2016   9   30   5.187
         
Statistical Monthly Average:       4.505
Official Monthly Average:            4.720
Difference on km2:                 214,933
Which is the best answer to Sep-2012 ASI lost (compared to 1979-2000)?
50% [NSIDC Extent] or
73% [PIOMAS Volume]

Volume is harder to measure than extent, but 3-dimensional space is real, 2D's hide ~50% thickness gone.
-> IPCC/NSIDC trends [based on extent] underestimate the real speed of ASI lost.

Juan C. García

  • ASIF Citizen
  • Posts: 431
    • View Profile
Re: Accuracy of poll predictions
« Reply #15 on: June 20, 2017, 06:32:21 AM »
Back in November 9th, 2012 (I had a bad size on NSIDC grid, but all the analysis seems fine to me):

I have been thinking about the method used to calculate the NSIDC monthly average and the impacts that could have in the future, so let’s image the following scenario:
We are at Sep. 1st, 2019. The NSIDC shows an Arctic sea ice extent (SIE) of one million km2, concentrated basically at the Arctic Basin. The ice extend is decreasing and by September 10th, the sea ice is almost gone. So, we have Arctic free of ice for the first time in several thousand years. By September 20th, some ice starts to build up around the continental coasts. By the end of the month, one million km2 has freeze.
On October, everybody is surprised that the NSIDC has announced a September SIE monthly average of 1.18 million km2. There was not a day with more than one million km2 at September. So, what will be the reason for this monthly average? The answer will be the method used to calculate the NISDC monthly average.
The normal NSIDC satellite grid at the arctic is 35km.* 35km. = 1,225 km2. The 15% of that is 183.75 km2. So basically, if a normal grid has a monthly average of 184 km2 or more, it accounts as 1,225 km2 for that month.

There are several ways in which a grid can qualify as a 100% SIE. By example, a grid at the Arctic Basin:

Day(%)- % - Sea Ice (km2)- Daily graph (km2)
Sep 1 - 100% - 1,225 - 1,225
Sep 2 - 100% - 1,225 - 1,225
Sep 3 - 85% - 1,041 - 1,225
Sep 4 - 65% - 796 - 1,225
Sep 5 - 50% - 613 - 1,225
Sep 6 - 30% - 368 - 1,225
Sep 7 - 14% - 172 - 0
Sep 8 - 10% - 123 - 0
Sep 9 - 8% - 98 - 0
Sep 10 - 3% - 37 - 0

Total: 5,696 km2
Average (divided by 30): 189.9 km2
It adds to monthly average: 1,225 km2

The NSIDC affirms that the ice at the beginning of September at the Arctic Basin accounts for 720,000 km2 and the freeze at continental coasts accounts for the rest, so there is a monthly average of 1.18 million km2 for September 2019.
Let’s have another example. A 325 km2 ice floe is moving from one grid to another on a sea free of ice (except for that floe). It stays 6 days in one grid, 15 days in second one and 9 days in a third one. At the end, it has made that 3 grids count as with 100% ice, so it accounts for 1,225 km2 * 3 = 3,675 km2. That looks incredible, but it counts for more than 10 times his size.
So, if someone is looking for the negative feedbacks that will make the Gompertz curve, the NSIDC method for calculating the monthly average seems that will be one important negative feedback. Rather artificial, but as I understand the method, it will work that way.

http://neven1.typepad.com/blog/2012/11/ct-sia-finally-above-2-million-km2-anomaly-mark.html?cid=6a0133f03a1e37970b017c3343058c970b#comment-6a0133f03a1e37970b017c3343058c970b
Which is the best answer to Sep-2012 ASI lost (compared to 1979-2000)?
50% [NSIDC Extent] or
73% [PIOMAS Volume]

Volume is harder to measure than extent, but 3-dimensional space is real, 2D's hide ~50% thickness gone.
-> IPCC/NSIDC trends [based on extent] underestimate the real speed of ASI lost.

Ned W

  • ASIF Citizen
  • Posts: 151
    • View Profile
Re: Accuracy of poll predictions
« Reply #16 on: June 20, 2017, 01:29:41 PM »
Juan, it looks to me like you are applying the 15% concentration threshold over time (in terms of the monthly average) in addition to over space (within a grid cell). I don't think that's right. 

As I understand it, for your first example, the grid cell would be reported as having 1225 km2 on the first six days, and 0 km2 on the remaining 24.  In that case, it would contribute 245 km2 to the monthly average, not 1225 km2. 

Likewise, in your second example, the ice floe only raises each of the three grid cells to 1225km2 for part of the month.  The first cell would end up contributing (1225 * 6 / 30), the second one (1225 * 15 / 30) and the third one (1225 * 9 / 30) ... which adds up to 1225 total, not three times that. 

oren

  • ASIF Upper Class
  • Posts: 1382
    • View Profile
Re: Accuracy of poll predictions
« Reply #17 on: June 20, 2017, 02:28:26 PM »
Juan, it looks to me like you are applying the 15% concentration threshold over time (in terms of the monthly average) in addition to over space (within a grid cell). I don't think that's right. 
We all agree it's not right, but it's still the way NSIDC calculate monthly averages. The averaging algorithm assumes immobile ice, when the ice is mobile (or melts here and freezes elsewhere) their algorithm produces absurdly high numbers.
If I recall correctly they announced they will create an improved algorithm for caculating the monthly averages, but I'm not sure where that stands.

Ned W

  • ASIF Citizen
  • Posts: 151
    • View Profile
Re: Accuracy of poll predictions
« Reply #18 on: June 20, 2017, 03:39:13 PM »
Thanks, Oren.  Despite following this stuff in a casual way for over a decade, I haven't paid that close attention to some of the details. 

Looking at the explanation here, it sounds like for the monthly extent they do apply a 15% threshold over time, but based on the daily concentration measurements not the daily extent measurements:

On monthly extent images, ice ends and water begins where the concentration estimates of grid cells in the gridded average, or mean, concentration field for that month drop below 15 percent.


That's slightly different from what I remembered, and it makes Juan's first example correct.  The monthly average concentration in his grid cell is 15.5% so it counts as "ice" for the month.

But his second example still doesn't seem to work.  His drifting ice floe represents 26.5% of whatever grid cell it's in.  Multiply that by 6/30, 15/30, and 9/30, and none of the three grid cells it spends time in would reach an average concentration of 15% for the month -- so all three grid cells would show up as non-ice.  In this example, the algorithm is underestimating the extent.

Disclaimer: both my understanding and my calculations could be wrong.

Juan C. García

  • ASIF Citizen
  • Posts: 431
    • View Profile
Re: Accuracy of poll predictions
« Reply #19 on: June 20, 2017, 04:10:22 PM »
But his second example still doesn't seem to work.  His drifting ice floe represents 26.5% of whatever grid cell it's in.  Multiply that by 6/30, 15/30, and 9/30, and none of the three grid cells it spends time in would reach an average concentration of 15% for the month -- so all three grid cells would show up as non-ice.  In this example, the algorithm is underestimating the extent.
Disclaimer: both my understanding and my calculations could be wrong.

You are right, Ned. It has to be a drifting ice floe with a bigger area, to work in this example.
Thank you for making the correction!

Even though, NSIDC makes us have the wrong conclusions, with the algorithm that they are using to calculate their monthly average.

Going back to the discussion in this topic, my point of view is that daily extent depends on winds and currents (or how the ice floes are separeted of each other), so it is not the better way of measuring the ASI when our planet is approaching an ice-free Arctic summer. It gets worst with the NSIDC monthly average algorithm.

So is it there an accuracy in the poll predictions? Luckly, we had some kind of rebound on 2013-2014 (that make our polls very incorrect). But 2015-2016 are not that good. If we see the low extent that we have on May 2016, well, it was difficult to forecast that the tendency will change for September. Even though, I prefer the ADS (JAXA or IJIS) conclusion, that 2016 is the second lowest year on record, than the NSIDC conclusion of fifth lowest. On NSIDC daily values, 2016 was also the second lowest, very close to 2007.

At the end, I believe that the tendency in volume is the one that is going to bring an ice-free Arctic, and if we follow this tendency, that will happen in the near future. Next 10 years? Even less? When it happens, extent will have a dramatic drop, even that it will also depend on "the statistical noise" of how the floes are separated of each other. I like to compare the ice measure in area, at least on September.

So the polls on this Forum are a better way to make a forecast (because in this Forum we are checking the status of the ice on a daily basis and in a way, we also know that an ice-free Arctic is going to happen in the near future), than the tendency that NSIDC says (based on their monthly average algorithm) and sadly, with the IPCC definition of "ice-free Arctic when there is less than 1 million km2 of extent in three consecutive years", well, the true is that they are also misleading.
« Last Edit: June 20, 2017, 06:57:20 PM by Juan C. García »
Which is the best answer to Sep-2012 ASI lost (compared to 1979-2000)?
50% [NSIDC Extent] or
73% [PIOMAS Volume]

Volume is harder to measure than extent, but 3-dimensional space is real, 2D's hide ~50% thickness gone.
-> IPCC/NSIDC trends [based on extent] underestimate the real speed of ASI lost.

Steven

  • ASIF Citizen
  • Posts: 360
    • View Profile
Re: Accuracy of poll predictions
« Reply #20 on: June 20, 2017, 08:20:04 PM »
Juan, it looks to me like you are applying the 15% concentration threshold over time (in terms of the monthly average) in addition to over space (within a grid cell). I don't think that's right. 
We all agree it's not right, but it's still the way NSIDC calculate monthly averages. The averaging algorithm assumes immobile ice, when the ice is mobile (or melts here and freezes elsewhere) their algorithm produces absurdly high numbers.

As shown in the table below, the difference between the official NSIDC September monthly extent (second column) and the average of the daily NSIDC extent values for September (third column) is for most years about 0.1 million km2 (fourth column), and at worst 0.2 million km2.  So I think this is only a relatively minor problem, and it is definitely not the reason for the bias on this forum.


                   NSIDC September extent:
Year              Official       DailyAverage       Difference
2005               5.59               5.50               0.09 million km2
2006               5.95               5.86               0.09
2007               4.32               4.27               0.05
2008               4.74               4.69               0.05
2009               5.39               5.26               0.13
2010               4.93               4.87               0.06
2011               4.63               4.56               0.07
2012               3.63               3.57               0.06
2013               5.35               5.21               0.14
2014               5.29               5.22               0.07
2015               4.68               4.62               0.06
2016               4.72               4.51               0.21

Juan C. García

  • ASIF Citizen
  • Posts: 431
    • View Profile
Re: Accuracy of poll predictions
« Reply #21 on: June 20, 2017, 09:44:24 PM »
As shown in the table below, the difference between the official NSIDC September monthly extent (second column) and the average of the daily NSIDC extent values for September (third column) is for most years about 0.1 million km2 (fourth column), and at worst 0.2 million km2.  So I think this is only a relatively minor problem, and it is definitely not the reason for the bias on this forum.

I believe that it is a subject that is important on this Forum, but it is not polite to talk about it. But waiting to have an ice-free Arctic -instead of talking about it- should be disgusting to the majority of the human beings.

I brought the subject, because I believe it is important in evaluating the accuracy of the polls. Now I am going to be off topic, so I will only make this comment and I will stop talking about it, at least under this topic.

In 2013 I made some analysis on the importance of this difference between the NSIDC statistical and official averages. And the difference are important in other months, specially in October.
What it means to me, is that when we have an ice-free Arctic, it could extent to the middle of October, at least.
Which is the best answer to Sep-2012 ASI lost (compared to 1979-2000)?
50% [NSIDC Extent] or
73% [PIOMAS Volume]

Volume is harder to measure than extent, but 3-dimensional space is real, 2D's hide ~50% thickness gone.
-> IPCC/NSIDC trends [based on extent] underestimate the real speed of ASI lost.

Ned W

  • ASIF Citizen
  • Posts: 151
    • View Profile
Re: Accuracy of poll predictions
« Reply #22 on: July 13, 2017, 02:55:23 PM »
I thought this graph was interesting:



It shows the variance (er, standard deviation) of the drop in extent from [Day X] to the September daily minimum (for JAXA daily extent).

During the first half of July, the "unpredictability" of the minimum is relatively high and relatively constant -- there's not much difference in the variance of the expected drop to minimum from July 1 vs from July 14.  But after July 15 that variance starts to drop fast. 

Also, interestingly the uncertainty in the eventual minimum seems to be slightly lower in mid-June than in July. 

Oh, yeah -- this is based on the 2007-2016 decade.