Please support this Forum and Neven's Blog

Poll

What will the NSIDC 2017 Arctic SIE September average be?

Between 5.0 and 5.5 million km2
1 (0.7%)
Between 4.75 and 5.25 million km2
0 (0%)
Between 4.5 and 5.0 million km2
2 (1.5%)
Between 4.25 and 4.75 million km2
4 (3%)
Between 4.0 and 4.5 million km2
16 (11.9%)
Between 3.75 and 4.25 million km2
15 (11.2%)
Between 3.5 and 4.0 million km2
14 (10.4%)
Between 3.25 and 3.75 million km2
21 (15.7%)
Between 3.0 and 3.5 million km2
22 (16.4%)
Between 2.75 and 3.25 million km2
9 (6.7%)
Between 2.5 and 3.0 million km2
9 (6.7%)
Between 2.25 and 2.75 million km2
5 (3.7%)
Between 2.0 and 2.5 million km2
4 (3%)
Between 1.75 and 2.25 million km2
1 (0.7%)
Between 1.5 and 2.0 million km2
2 (1.5%)
Between 1.25 and 1.75 million km2
2 (1.5%)
Between 1.0 and 1.5 million km2
2 (1.5%)
Between 0.75 and 1.25 million km2
2 (1.5%)
Between 0.5 and 1.0 million km2
2 (1.5%)
Between 0.25 and 0.75 million km2
1 (0.7%)
Between 0 and 0.5 million km2
0 (0%)

Total Members Voted: 133

Voting closed: June 12, 2017, 10:42:16 PM

Author Topic: NSIDC 2017 Arctic SIE September average: June poll  (Read 6937 times)

epiphyte

  • ASIF Citizen
  • Posts: 338
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #50 on: June 12, 2017, 05:22:44 PM »
3-3.5M sq.km for me, for what it's worth, with my oft-repeated caveat that if and when it goes, it will all go at once. So I don't really have much confidence in my own prediction - it's mid-way between two extremes, but that doesn't make it more likely - It's like predicting a draw in a game of Russian roulette.

It could have happened last year (IMO we missed by about two weeks), and it could definitely still happen this year. If it does then Sep extent will be shockingly low - likely <3M. If it doesn't, then we pulled the trigger on an empty chamber, again, and the Sep extent could be much higher. 

Laptev, for example, looks to me to be in the process of being 'bitten' despite all the snow to the south, the only sub-freezing surface temps in the whole Arctic, and the apparently high-albedo surface.

I'm considering also that most of the peripheral extent, especially in Laptev & ESS, is very new (i.e. grew from open water during (an unusually mild) March and April, to fill the void left by the MYI exiting the Fram). The normal early-springtime offshore-onshore oscillation which accelerates ridging, and hence volume build-up, early in the year seems to me to have been replaced with continuous dispersal, fragmentation and a relentless march North. Add in the insulating and albedo-raising effect of early snowfall and, models notwithstanding, we could be looking at >1m km2 of tissue-thin ice which could vanish in literally hours.

So um. yes.
« Last Edit: June 12, 2017, 05:35:04 PM by epiphyte »

Yuha

  • ASIF Citizen
  • Posts: 206
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #51 on: June 12, 2017, 08:04:02 PM »
4.0 - 4.5 as half way between 2012 and 2016.

Decided at last minute to drop down one bucket to 3.75 - 4.25 which too contains the halfway point. The change of mind was based on seeing some darkened ice in Worldview (north of Chukchi Sea) and Wipneus' latest PIOMAS analysis.

Steven

  • ASIF Citizen
  • Posts: 368
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #52 on: June 12, 2017, 10:41:29 PM »
Rob, I wasn't intending to submit this to the SIPN.  The method I used isn't really new, since I'm obviously not the first one who uses sea ice volume and/or area at this time of year to predict the September minimum extent.

The method I used performs better than a simple linear trend of September extent.  But there are certainly better methods.  E.g., Schroeder et al.'s method based on modeled May melt ponds has a better forecast skill than mine.  But apparently their forecasts tend to be somewhat too high in recent years.

Schroeder's May melt pond method has a SD of 500 k km^2 :
https://www.arcus.org/files/sio/25659/sio-2016-june_cpom_schroeder-feltham-flocco-tsamados.pdf

Your method is better than that, and thus I would encourage you to enter in SIPN.


Are you comparing hindcasts or forecasts?

Schroeder et al.'s method has a standard deviation of 330 k km2 for the residuals of the hindcasts, and it has a forecast error for the last few decades* of about 500 k km2.

For comparison, the method I used has a standard deviation of 440 k km2 for the hindcasts, whereas the forecast error of my method for the last few decades would probably be somewhere between 500 and 550 k km2 or so.  That is worse than Schroeder's.

A similar remark applies to your method as well.  You have a SD of 460 k km2 for the residuals of the hindcasts.  But the forecast error of your method for the last few decades is certainly higher than that (probably in the same ballpark as for my method).


*: To calculate the "forecast" that the method would have produced for the year 2009 (say), only data up to 2008 should be used to calculate the coefficients of the regression equation.  In contrast, the hindcast for 2009 is calculated from data for the entire period 1979-2016.
« Last Edit: June 12, 2017, 11:14:45 PM by Steven »

Neven

  • Administrator
  • ASIF Governor
  • *****
  • Posts: 3990
    • View Profile
    • Arctic Sea Ice Blog
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #53 on: June 12, 2017, 11:41:56 PM »
The poll has closed. 134 votes in total, which is pretty great (thanks, everyone). When I find the time I'll post the average and the median (if I remember how to do it, and hopefully the overlapping bins don't make this more complicated  ;) ).
Il faut cultiver notre jardin

Rob Dekker

  • ASIF Citizen
  • Posts: 482
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #54 on: June 13, 2017, 06:57:16 AM »
Are you comparing hindcasts or forecasts?

Schroeder et al.'s method has a standard deviation of 330 k km2 for the residuals of the hindcasts, and it has a forecast error for the last few decades* of about 500 k km2.

For comparison, the method I used has a standard deviation of 440 k km2 for the hindcasts, whereas the forecast error of my method for the last few decades would probably be somewhere between 500 and 550 k km2 or so.  That is worse than Schroeder's.

A similar remark applies to your method as well.  You have a SD of 460 k km2 for the residuals of the hindcasts.  But the forecast error of your method for the last few decades is certainly higher than that (probably in the same ballpark as for my method).

*: To calculate the "forecast" that the method would have produced for the year 2009 (say), only data up to 2008 should be used to calculate the coefficients of the regression equation.  In contrast, the hindcast for 2009 is calculated from data for the entire period 1979-2016.

I'm comparing 'forecast' methods.
That is : methods that include ONLY data that is available before the prediction is made.
For May predictions of September SIE, that should include only data that is available in May of that year. In other words, for real 'forecast' methods, you cannot use data available only in June-September.

There, Schroeder's method (in forecast mode) has a SD of the residuals of 500 k km^2.
My method (implicitly a forecast method) has a SD of the residuals of 460 k km^2.

For your method, I'm not sure now.
When you mentioned 900 k km^2 for the 95% confidence range (2 SD), did you include data from the summer (June->Sept) for the years in the learning period in your linear regression ?

To be clear : 'hindcasts' as defined in Schroeder's method can only be calculated for past years, NOT for the current year, since they include data from June->September. 'hindcasts' are useful to determine how much of natural variability over the summer is captured by your variables, but of course it is useless for true predictions for a new year. As Schroeder himself puts it :

The low error values in the hindcast of September ice extent
do not guarantee that pond fraction can be used for real forecasts
of Arctic September sea ice, because for a forecast only data of
previous years are available to calculate the weights and linear
regression
« Last Edit: June 13, 2017, 07:27:36 AM by Rob Dekker »

Rob Dekker

  • ASIF Citizen
  • Posts: 482
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #55 on: June 13, 2017, 07:15:59 AM »
Thanks for organizing this poll, Neven !
Seems that the majority of votes is between 3.0 and 4.5 this year.
But the spread is quite extensive.
May is still early for predictions, and I'm looking forward to the July poll, based on June data, which tends to be a lot more accurate.

Steven

  • ASIF Citizen
  • Posts: 368
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #56 on: June 13, 2017, 06:22:46 PM »
There, Schroeder's method (in forecast mode) has a SD of the residuals of 500 k km^2.
My method (implicitly a forecast method) has a SD of the residuals of 460 k km^2.

That is an unfair comparison.  You are overselling your model.

I looked at your model a few years ago, and I'm pretty sure that 460k km2 is the standard deviation of the residuals for the hindcasts (rather than forecasts) of your model.

For example, what is the forecast that your model would have produced for the year 2014?  The actual forecast that you made back then is 4.6 M km2 (link).  But I suspect that you are currently using a different "forecast" value for the year 2014 when you are calculating your standard deviation.

Another example: for a simple linear fit of September extent over 1979-2016, the standard deviation of residuals is about 550 k km2 for hindcasts, but almost 600 k km2 in forecast mode.

Regarding my method, the 2*SD = 900 k km2 number that I mentioned on Saturday was just a quick estimate from the hindcasts.  It's much more work to calculate this for the actual forecasts that the method would have produced, since then the coefficients of the regression equation have to be re-calculated for each individual year.  Anyway, I did that calculation yesterday, and it would give about 2*SD = 1.04 M km2.

Rob Dekker

  • ASIF Citizen
  • Posts: 482
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #57 on: June 14, 2017, 05:05:23 AM »
Steven, thank you !

I always thought that a 'hindcast' included data (past performance) between the moment the prediction was made (May in this case) to the moment of prediction (September in this case).
But in reading your response, and re-reading Schroeder's paper, I realized I was mistaken.
Schroeder writes :
To investigate the potential of pond fraction as a predictor for
Arctic September sea ice, we first used the whole data period
to derive the linear regression between spring pond fraction and
September ice extent and applied the regression line to calculate
September ice extent from spring pond fraction (hindcast mode).
And that is exactly what I did (with snow cover rather than pond fraction).

So indeed my method is called a 'hindcast' under that definition. I learned something today  :)

Couple of thoughts :
- The (hindcast) 330 k SD that Schroeder obtains with May data is impressive. I obtain that SD (using snow cover) only with June data.

- I'm also impressed that you obtain (hindcast) SD 450 k km^2 with your method using May data of just PIOMAS volume and ice 'area' as variables. If you have a chance, could you explain your method in more detail (like, what exactly is your regression formula, and how did you determine the weight factors for each variable).

- I'm now confused about Schroeder's 'forecast' method. Seems to me that if you re-calculate the regression parameters for every year (using the years before that) that you will get serious 'over-fitting' for the first couple of years, then in the middle you don't have enough data points, and at the end of the time series you match the results of the 'hindcast' method. I don't see how the SD of residuals of such a strange method tells anything useful.

Steven

  • ASIF Citizen
  • Posts: 368
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #58 on: June 14, 2017, 07:31:54 PM »
The (hindcast) 330 k SD that Schroeder obtains with May data is impressive.


Yes, but Schroeder's model has lots of parameters.  That makes it easier to obtain good hindcasts.  Hindcasts aren't very informative when there are lots of predictor variables:  a low hindcast error in such a case could be simply due to overfitting.  That is the reason why Schroeder only runs his model in forecast mode when he discusses the skill of the model.

Regarding my model: I used two predictor variables (volume and area).  There are built-in software packages in R and in Excel to perform multiple linear regression.  This automatically computes the optimal coefficients of the regression equation.  The raw (unadjusted) SD of the hindcast residuals for my method is 420 k km2.  But I preferred to make an adjustment, to account for the fact that I'm using multiple predictors.  The adjusted hindcast residual SD is 450 k km2.  The adjustment was obtained from using the adjusted R squared of the multiple linear regression.

Regarding the calculation of forecasts: to simulate the forecast for the year 1989 (say), only 10 years of data would be available: 1979-1988.  I think that is too few.  In fact, I prefer to have at least 20 years of data available for each forecast.  For that reason I only simulated forecasts from 1999 onwards  (using 1979-1998 data for the 1999 forecast,  1979-1999 data for the 2000 forecast, and so on).  Doing that for each year between 1999 and 2016, I get a forecast error SD of 520k km2 for my method.

Rob Dekker

  • ASIF Citizen
  • Posts: 482
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #59 on: June 16, 2017, 07:22:59 AM »
The (hindcast) 330 k SD that Schroeder obtains with May data is impressive.

Yes, but Schroeder's model has lots of parameters.  That makes it easier to obtain good hindcasts.  Hindcasts aren't very informative when there are lots of predictor variables:  a low hindcast error in such a case could be simply due to overfitting.  That is the reason why Schroeder only runs his model in forecast mode when he discusses the skill of the model.

I thought that Schroeder's model had only one parameter : "pond fraction".

Rob Dekker

  • ASIF Citizen
  • Posts: 482
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #60 on: June 16, 2017, 07:37:09 AM »
Incidentally, here is another paper (with Schroeder on the author list)
http://onlinelibrary.wiley.com/doi/10.1002/2016EF000495/pdf

which claims that Sea Ice Concentration (SIC) has equal or better skill than the simulated "pond fraction" from Schroeder's original paper.

This (using SIC to determine melt potential) is very similar to the great work by Tealight (Nico Sun) :
https://sites.google.com/site/cryospherecomputing/daily-data

Also interesting is the skill of Melting Onset (MO) metric by Julienne Stroeve, especially for early predictions (March to May) of September Sea Ice.

Steven

  • ASIF Citizen
  • Posts: 368
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #61 on: June 16, 2017, 07:33:53 PM »
I thought that Schroeder's model had only one parameter : "pond fraction".


Not really.  Schroeder et al. use gridded data, and they "calculate weights for each grid point based on the correlation coefficient between the local pond area and the Arctic September ice extent ".  See also this comment by Michael Hauber a few years ago:

http://forum.arctic-sea-ice.net/index.php/topic,902.msg28775.html#msg28775

Steven

  • ASIF Citizen
  • Posts: 368
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #62 on: July 01, 2017, 09:09:24 AM »
The SIPN June 2017 report is out:

https://www.arcus.org/sipn/sea-ice-outlook/2017/june

The median Outlook value for September 2017 sea ice extent is 4.43 million square kilometers with quartiles of 4.10 and 4.71 million square kilometers (See Figure 1 in the Overview section, below). Contributions are based on a range of methods: statistical, dynamical models, heuristic, and two informal polls.
...
To place this year's Outlook in context, consider recent observed values of 4.28 million square kilometers in 2007, 3.60 million square kilometers in 2012, and 4.72 million square kilometers in 2016. Only one participant suggests a new record low is likely, though several others suggest it is possible.





Darvince

  • ASIF Citizen
  • Posts: 197
    • View Profile
    • NSIDC Daily
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #63 on: July 01, 2017, 12:49:14 PM »
The statistical models have clustered higher than the dynamical models...

How many of each take into account sea ice thickness?

Steven

  • ASIF Citizen
  • Posts: 368
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #64 on: July 02, 2017, 04:44:11 PM »
The statistical models have clustered higher than the dynamical models...

How many of each take into account sea ice thickness?

Dynamical models should model sea ice thickness in one way or another.  Some of them use PIOMAS or CryoSat for initial conditions, but most of them seem to use their own model for that.

Regarding the statistical models, only a few of them use thickness data.  There are other predictors (e.g. sea ice concentration, melt pond area, SSTs etc) that are also important.

EgalSust

  • ASIF Lurker
  • Posts: 24
    • View Profile
Re: NSIDC 2017 Arctic SIE September average: June poll
« Reply #65 on: July 13, 2017, 08:44:12 PM »
The forum median prediction was between 3.25 and 3.75 MkmĀ². That would mean second lowest or record low extent. Compared to SIPN/SIO predictions, this sits at the alarmist end of the spectrum.