Support the Arctic Sea Ice Forum and Blog

Author Topic: May melt ponds do not matter  (Read 12584 times)

Michael Hauber

  • Nilas ice
  • Posts: 1142
    • View Profile
  • Liked: 181
  • Likes Given: 16
May melt ponds do not matter
« on: June 18, 2014, 11:41:35 AM »
I refer to recent paper which seems to imply that the September minimum is largely determined by the melt pond fraction in May.  However my analysis is that the strength of the relationship is an effect of overfitting the model, and that in actual fact melt pond fraction in late June is a better indicator of final minimum, and that it is not a significantly better predictor than a simple extrapolation of the long term trend.

First, the correlation between melt pond fraction and September minimum is quite impressive at over 0.8.  Hindcasts made with this method have an average error of 0.33 million km2.

However I've always been suspicious as there is usually almost no melt pond visible from satellite in the central Arctic area until early to mid June.  Figure 1 of the paper reveals that the typical melt pond fraction at the end of may is around 2% at the end of May.  Figure 2 shows that the area of the Arctic for which melt ponds are measured excludes the Hudson Bay and Sea of Okhotsk, but includes much of the Bering Strait, the Baffin bay, Greenland Sea, Barents Sea and Kara see.  At the end of May surely nearly all the melt ponds must be in such fringe areas if the total fraction is only 2%.  How can this determine the fate of the central ice in September? 

A key issue that is not immediately obvious, is the fact that the melt pond fraction is not a straightforward calculation of total melt pond area divided by total ice area.  A geographic weighting is applied with a different rating for each of 1000s of grid squares.  This raised the possibility that the strong correlation is not due to causation, but is due to overfitting the model due to having too many variables available to tweak.  In particular imagine any grid square that has melt ponds in one particular year, but not in any other year.  The weighting for this square can then be tweaked to change the model prediction for that year without affecting the prediction for other years.  Obviously there is a limit to this otherwise the correlation would be perfect instead of good.  But it may cause a stronger correlation than a purely physical causal relationship would otherwise suggest.  I suspect this may explain why there is a better correlation early on in May - there are more opportunities to find grid squares that affect only 1 or a small number of years whereas by July most grid squares would affect many or most years.

One good way to prevent such overfitting is to train the model on data for some of the years, and attempt to use the model to predict the result in other years.  This is done in the paper by making a 'forecast' for each year by using only data for years before that year.  This is effectively what would have been forecast if the method in the paper was used to actually forecast that year when the results for that year (and future years) are not yet available.

The results are much less impressive and suggest that some type of overfitting effect is at play.  The standard error for the end of May melt pond fraction prediction increases from 0.33 to 0.5.  The error for the prediction using melt pond fraction up to June 25 goes from being worse than end of May to being better (0.36 to 0.41).  A prediction error of 0.5 to my eyes looks to be no better than the average error for making a prediction based purely on extrapolating the long term trend, although I'm not motivated enough to try and download the data and perform a calculation to confirm this.
Climate change:  Prepare for the worst, hope for the best, expect the middle.

crandles

  • Young ice
  • Posts: 3379
    • View Profile
  • Liked: 240
  • Likes Given: 81
Re: May melt ponds do not matter
« Reply #1 on: June 18, 2014, 12:46:41 PM »
That is interesting.

For 2012 I ended up with 4.3+/- 1.1m Km^2
http://www.arcus.org/search-program/seaiceoutlook/2012/june
other ranges were from +/-0.2 (unbelievably good?) to +/- 1.3

It seems there are Easy and Difficult Years:
http://www.arcus.org/witness-the-arctic/2014/2/article/21066

In 2013 the July SEARCH outlook consensus was too low by 1.3m Km^2.

So a standard error in an honest forecast mode of 0.5m Km^2 looks good to me, but not so good that other methods should be abandoned.

.

I have a different slight concern about the melt pond fraction method:

Presumably at some point May 31 thicknesses will be such that they melt out regardless of melt pond fraction. If this breaks up into small pieces, the melt pond fraction may be lower so that the method predicts a high minimum.



This doesn't look hugely different to previous years for area with thickness up to about 1.9m, which can melt out. However it might be in future that the profile does change significantly from one year to the next.


Developing that a little more: it might be that if the peripheral areas have lots of thin ice then melt can proceed to the Central Basin earlier and melt out more area there.





This seems to offer more scope for the 2014 prediction of similar to 2013 to be too high a prediction

seaice.de

  • Frazil ice
  • Posts: 132
    • View Profile
  • Liked: 91
  • Likes Given: 21
Re: May melt ponds do not matter
« Reply #2 on: June 18, 2014, 02:50:55 PM »
I refer to recent paper which seems to imply that the September minimum is largely determined by the melt pond fraction in May.  However my analysis is that the strength of the relationship is an effect of overfitting the model, and that in actual fact melt pond fraction in late June is a better indicator of final minimum, and that it is not a significantly better predictor than a simple extrapolation of the long term trend.

If we would have seen such a strong empirical correlation between melt pond coverage in May and September extent we would have reported it in our paper http://www.seaice.de/Roesel_JGR_2012.pdf
But unfortunately, the observational time series is too short to draw valid conclusions

ChrisReynolds

  • Nilas ice
  • Posts: 1764
    • View Profile
    • Dosbat
  • Liked: 20
  • Likes Given: 9
Re: May melt ponds do not matter
« Reply #3 on: June 18, 2014, 08:06:08 PM »
Michael,

Not having access to the paper I can't comment much. But melt ponds in May should not be largely within the Arctic Ocean because temperatures are only just rising to zero in May. Furthermore I view the drop in CT Area in June during recent years as indicative of the spread of melt. There is an  increase in CT Area correlation with September CT Area (detrended) during June, I think this is significant - acting as a barrier to early prediction.

So June melt ponds predicting the melt is believable, May I find hard to believe. Your point about overfitting in light of predicting for other years seems sound, with the caveat that I've not read the paper - paywalled.

It is of course possible that any correlation between early season melt ponding and the minimum is picking up two factors. Albedo pre-conditioning, and the prevalence of FYI across much of the pack (increasing open water formation efficiency). For strong statistical correlations to emerge we really need the post 2007 and post 2010 periods to extent for a few more decades.  ;)

Neven

  • Administrator
  • First-year ice
  • Posts: 9767
    • View Profile
    • Arctic Sea Ice Blog
  • Liked: 1380
  • Likes Given: 622
Re: May melt ponds do not matter
« Reply #4 on: June 18, 2014, 10:55:22 PM »
Thanks for your thoughts on this, Michael. I've given extra attention to melt pond May in the ASIB, but to be frank, I haven't been able to make out the mechanism behind it all. Is there a cut-off date? Which regions are most important melt pond-wise?

But this year doesn't make much sense to me anyhow, what with extensive high-pressure areas, but cold temps and slow SIE/SIA declines...
Make money, not peace

Michael Hauber

  • Nilas ice
  • Posts: 1142
    • View Profile
  • Liked: 181
  • Likes Given: 16
Re: May melt ponds do not matter
« Reply #5 on: June 18, 2014, 11:00:07 PM »

It is of course possible that any correlation between early season melt ponding and the minimum is picking up two factors. Albedo pre-conditioning, and the prevalence of FYI across much of the pack (increasing open water formation efficiency). For strong statistical correlations to emerge we really need the post 2007 and post 2010 periods to extent for a few more decades.  ;)

This is stated in the paper.  Melt pond fraction is calculated on a model, and validated against observations were possible, similar to PIOMAS volume.  Running their model they are able to observe that pre-conditioning such as thin and first year ice are very significant.  First year ice melts at a cooler temperature, and thin ice is flatter so a given amount of melt water will spread further.

And of course we should be able to get perfect correlations once the Arctic is ice free :P
Climate change:  Prepare for the worst, hope for the best, expect the middle.

Nightvid Cole

  • Frazil ice
  • Posts: 438
    • View Profile
  • Liked: 6
  • Likes Given: 0
Re: May melt ponds do not matter
« Reply #6 on: June 18, 2014, 11:56:21 PM »
I refer to recent paper which seems to imply that the September minimum is largely determined by the melt pond fraction in May.  However my analysis is that the strength of the relationship is an effect of overfitting the model, and that in actual fact melt pond fraction in late June is a better indicator of final minimum, and that it is not a significantly better predictor than a simple extrapolation of the long term trend.

First, the correlation between melt pond fraction and September minimum is quite impressive at over 0.8.  Hindcasts made with this method have an average error of 0.33 million km2.

However I've always been suspicious as there is usually almost no melt pond visible from satellite in the central Arctic area until early to mid June.  Figure 1 of the paper reveals that the typical melt pond fraction at the end of may is around 2% at the end of May.  Figure 2 shows that the area of the Arctic for which melt ponds are measured excludes the Hudson Bay and Sea of Okhotsk, but includes much of the Bering Strait, the Baffin bay, Greenland Sea, Barents Sea and Kara see.  At the end of May surely nearly all the melt ponds must be in such fringe areas if the total fraction is only 2%.  How can this determine the fate of the central ice in September? 

A key issue that is not immediately obvious, is the fact that the melt pond fraction is not a straightforward calculation of total melt pond area divided by total ice area.  A geographic weighting is applied with a different rating for each of 1000s of grid squares.  This raised the possibility that the strong correlation is not due to causation, but is due to overfitting the model due to having too many variables available to tweak.  In particular imagine any grid square that has melt ponds in one particular year, but not in any other year.  The weighting for this square can then be tweaked to change the model prediction for that year without affecting the prediction for other years.  Obviously there is a limit to this otherwise the correlation would be perfect instead of good.  But it may cause a stronger correlation than a purely physical causal relationship would otherwise suggest.  I suspect this may explain why there is a better correlation early on in May - there are more opportunities to find grid squares that affect only 1 or a small number of years whereas by July most grid squares would affect many or most years.

One good way to prevent such overfitting is to train the model on data for some of the years, and attempt to use the model to predict the result in other years.  This is done in the paper by making a 'forecast' for each year by using only data for years before that year.  This is effectively what would have been forecast if the method in the paper was used to actually forecast that year when the results for that year (and future years) are not yet available.

The results are much less impressive and suggest that some type of overfitting effect is at play.  The standard error for the end of May melt pond fraction prediction increases from 0.33 to 0.5.  The error for the prediction using melt pond fraction up to June 25 goes from being worse than end of May to being better (0.36 to 0.41).  A prediction error of 0.5 to my eyes looks to be no better than the average error for making a prediction based purely on extrapolating the long term trend, although I'm not motivated enough to try and download the data and perform a calculation to confirm this.

You say May doesn't matter, they say it matters most. I think the truth is probably somewhere in between.

While the pond coverage is low in May, it's also true that melt ponds tend to deepen themselves due to albedo feedback even when the surrounding snow isn't melting much. Melt ponds that form in May have time to get deep enough to absorb quite a bit of energy by late June, while those that form later are still going to be very shallow and a lot of light will be just reflected from the snow/ice underneath. May melt ponds also have time to enlarge themselves by albedo feedback at their "coastline" (edges).

Also, I don't think you can explain away so much by over-fitting, since we're talking also about a physical model of melt ponds which isn't all that dependent upon deriving parameters from the data. Furthermore, the correlation is high even between May temperature and the fate of the ice, which further supports the contention that May does indeed matter noticeably, even if not as much as June.

crandles

  • Young ice
  • Posts: 3379
    • View Profile
  • Liked: 240
  • Likes Given: 81
Re: May melt ponds do not matter
« Reply #7 on: June 19, 2014, 12:08:25 AM »
Michael,

Not having access to the paper I can't comment much.

Someone provided a link on the other thread for this paper as:
http://glaciology.weebly.com/uploads/6/6/9/1/6691883/2014-schroeder_nclimate2203.pdf

Quote
A forecast based on the amount of thin ice has a lower skill
(S = 0.29, Fig. 4b) than that based on pond fraction (S = 0.41).
However, the fact that there is any skill at all using thin ice confirms
that our model simulation is realistic with the inclusion of the
melt-pond model. The differences between simulated and observed
sea-ice extent are small (Fig. 2 for 1996 and 2012). If we use the
pond statistics based on MODIS satellite data16, we get a similar
correlation by integrating over the period May until July, but the
period for which MODIS is available (since 2002) is too short to
draw statistically valid conclusions.

« Last Edit: June 19, 2014, 12:18:52 AM by crandles »

Michael Hauber

  • Nilas ice
  • Posts: 1142
    • View Profile
  • Liked: 181
  • Likes Given: 16
Re: May melt ponds do not matter
« Reply #8 on: June 19, 2014, 12:33:54 AM »
Melt ponds that form in May might have perhaps twice as much time as those that form in late June to absorb sunlight.  However there are ten times as many melt ponds forming in late June.  And ponds that form in May will be mostly in areas that will melt out well before the end of the melting season.

I've downloaded NSIDC extent data to quickly calculate a standard error (as standard deviation, not sure if that is exactly right) for a forecast based on fitting a linear and quadratic trend.  These are 0.56 for linear and 0.49 for the quadratic.  The standard error for end of May melt pond fraction predictions was 0.5.
Climate change:  Prepare for the worst, hope for the best, expect the middle.

ChrisReynolds

  • Nilas ice
  • Posts: 1764
    • View Profile
    • Dosbat
  • Liked: 20
  • Likes Given: 9
Re: May melt ponds do not matter
« Reply #9 on: June 20, 2014, 07:23:12 PM »
Thanks Crandles, Jim sent me a copy.

Nightvid Cole

  • Frazil ice
  • Posts: 438
    • View Profile
  • Liked: 6
  • Likes Given: 0
Re: May melt ponds do not matter
« Reply #10 on: July 31, 2014, 05:46:35 PM »
Look at Figure 9 in http://onlinelibrary.wiley.com/doi/10.1029/2009JC005436/pdf ( Markus et al. , Recent changes in Arctic sea ice melt onset, freezeup, and melt
season length ). It seems that once snow starts melting, it is a self-accelerating process (presumably due to albedo feedback). A couple years had early melt in Laptev/ESS and in Central Arctic begin before day 150 (June 1st) and then they went on to have earlier onset of continuous melt, and had lower September extents. Unfortunately, we do not have anything from the post-2007 era. But the trend of earlier melt onsets is clearly established, so clearly if the snow turns soggy in May it indeed puts the entire melting season ahead of a season that still has dry snow on June 1.

So May melt ponds perhaps don't *directly* matter, however, they are a symptom (not a cause) of liquid water layer formation in the snow pack. This liquid water layer increases the solar absorptivity of the snowpack, and thus via albedo feedback leads to accelerated ice loss.

So despite only a tiny areal coverage of melt ponds in May, they DO correlate with melt onset, snow loss, and subsequently ice loss. This is so not because of the reduced albedo of the ponds themselves, but of the liquid water layer in the snow pack.

Steven

  • Nilas ice
  • Posts: 1062
    • View Profile
    • Arctic sea ice data and graphs
  • Liked: 557
  • Likes Given: 19
Re: May melt ponds do not matter
« Reply #11 on: May 20, 2015, 07:25:01 PM »
New paper: Liu et al. 2015 (open access):

"Revisiting the potential of melt pond fraction as a predictor for the seasonal Arctic sea ice extent minimum"

http://iopscience.iop.org/1748-9326/10/5/054017/article

Quote
Abstract

A recent modeling study [Schroeder et al. 2014] that employed a prognostic melt pond model in a stand-alone sea ice model found that September Arctic sea ice extent can be accurately predicted from the melt pond fraction in May.

Here we show that satellite observations show no evidence of predictive skill in May.

However, we find that a significantly strong relationship (high predictability) first emerges as the melt pond fraction is integrated from early May to late June, with a persistent strong relationship only occurring after late July.

Our results highlight that late spring to mid summer melt pond information is required to improve the prediction skill of the seasonal sea ice minimum.

Furthermore, satellite observations indicate a much higher percentage of melt pond formation in May than does the aforementioned model simulation, which points to the need to reconcile model simulations and observations, in order to better understand key mechanisms of melt pond formation and evolution and their influence on sea ice state.

Nightvid Cole

  • Frazil ice
  • Posts: 438
    • View Profile
  • Liked: 6
  • Likes Given: 0
Re: May melt ponds do not matter
« Reply #12 on: May 20, 2015, 08:52:36 PM »
Near the end of the paper, the authors do admit

Quote

 It should also be mentioned that our results do not necessarily indicate that the predictive skill of the model described in S14 is less than reported. Nevertheless, our findings (low predictive value of observed May melt ponds, and large bias in the amount of modeled May melt ponds) raise the possibility that something other than melt pond formation (such as perhaps surface melt onset and/or above freezing temperatures) could be the source of the model predictability.


What I would really like to see is a similar analysis done with not melt pond fraction, but liquid water  content of snow. I think this may be the key factor, because this starts rising immediately upon initiation of snowmelt and thus represents an earlier phase of the surface melting sequence than the formation of melt ponds.

Also, it is important to exclude the peripheral areas that play no part in year to year variation of September extent because they fully melt every year, such as the Bering Sea, Baffin Bay, and Hudson Bay.

If I had to guess, I'd venture to say that an appropriate measure of liquid water content of snow would have real predictive capacity over September extent before mid-June.

Lord M Vader

  • Nilas ice
  • Posts: 1413
    • View Profile
  • Liked: 67
  • Likes Given: 41
Re: May melt ponds do not matter
« Reply #13 on: May 20, 2015, 09:31:03 PM »
I think there is a good possibility to test the different hypothesis here soon as it looks like the Beaufort high pressure dome most likely will continue for the beginning of June...

A few things I'm thinking about that is worth to watch are:

- how much soot from last years wild fires in Canada did single down in the Arctic?
- what quality is the snow of? Is it "fluffy" or have it been packed a lot?
- will the high pressure dome on the American side continue to linger through June?

Finally, I saw in a post by, I think it was, Robert Scribbler that during El Niño there are a tendency of warm air intrusion and high pressure in the Arctic basin from the Canadian side. What I don't have any info about is whether this is true for the first or the second year of the El Niño? Anyone who knows? If the case is that this is true for the second year of El Niño maybe we're already getting a hint of the 2016 melt season? Neven, maybe something worth a blog post or so later? ;)

Best, LMV

icefest

  • Frazil ice
  • Posts: 258
    • View Profile
  • Liked: 1
  • Likes Given: 0
Re: May melt ponds do not matter
« Reply #14 on: May 20, 2015, 09:54:24 PM »
Isn't neven on holidays for the month, lmv?
Open other end.

oren

  • Moderator
  • Multi-year ice
  • Posts: 10044
    • View Profile
  • Liked: 3784
  • Likes Given: 4298
Re: May melt ponds do not matter
« Reply #15 on: May 20, 2015, 11:23:45 PM »
Isn't neven on holidays for the month, lmv?

Yes he is.

slow wing

  • Grease ice
  • Posts: 823
    • View Profile
  • Liked: 156
  • Likes Given: 546
Re: May melt ponds do not matter
« Reply #16 on: May 21, 2015, 06:59:16 AM »
Wouldn't a powerful tool be to study the local correlation between melt pond formation & melt-out where the ponds have formed?

A large positive correlation might be expected.

This could then lead to region-specific predictive modelling for each melt season.

Nightvid Cole

  • Frazil ice
  • Posts: 438
    • View Profile
  • Liked: 6
  • Likes Given: 0
Re: May melt ponds do not matter
« Reply #17 on: May 21, 2015, 03:20:39 PM »
Wouldn't a powerful tool be to study the local correlation between melt pond formation & melt-out where the ponds have formed?

A large positive correlation might be expected.

This could then lead to region-specific predictive modelling for each melt season.

Yes, I think so. This change, combined with an inclusion of liquid water content in snow, should make a better projection possible from the early season (before June 15 or so).

ktonine

  • Frazil ice
  • Posts: 363
    • View Profile
  • Liked: 0
  • Likes Given: 0
Re: May melt ponds do not matter
« Reply #18 on: May 24, 2015, 03:30:54 PM »
Wouldn't a powerful tool be to study the local correlation between melt pond formation & melt-out where the ponds have formed?

A large positive correlation might be expected.

This could then lead to region-specific predictive modelling for each melt season.

It would be very difficult to track individual melt ponds.