Home brew AMSR2 extent & area calculation

[Wipneus]

Total extent drop -93.6k, it is eating into the CAB now as well (-38.2k). Here is a diff with the previous day.

Bright red/blue: loss/gain of extent (crossing the 15% concentration limit). Light red/blue:  concentration change more than 7%. 

Click for the hi-res picture.

[binntho]

Thanks Wipneus, I really love these diff pictures!

[Shared Humanity]

From that image, we should expect that the melt progress on Pacific side of the CAB will determine this years minimum.

[Killian]

Total extent drop -93.6k, it is eating into the CAB now as well (-38.2k). Here is a diff with the previous day.

Bright red/blue: loss/gain of extent (crossing the 15% concentration limit). Light red/blue:  concentration change more than 7%. 

Click for the hi-res picture.

You using a trailing average or single day for extent above, Wipneus?

[oren]

You using a trailing average or single day for extent above, Wipneus?

I recommend to read the top post of this thread, where Wipneus explained his algorithm. The values are single day, although there is filtering of spurious ice and false coastal ice.

[be cause]

love seeing this thread alive again .. cheers Wipneus ..    b.c.

[Alison]

+1

[petm]

Wipneus, your graphs from the top post stopped updating about a week ago.

[SATire]

I am worried about Wipneus. His updates were like a clock from Switzerland. How can I look at the state of sea ice without his product? This feels so 2012... 

[oren]

Yeah I am sorely missing the Regional area and extent graphs

[HapHazard]

I am assuming Wip is on a vacation / trip so is AFK.

[Pipster]

Yeah I am sorely missing the Regional area and extent graphs

I have been too!  They are freshly updated now.
Thanks so much Wipneus!

[petm]

Hooray, they're back! 

[Wipneus]

The Google API that I am using to automatically update data and graphs has stopped working for me  ("Internal Server Error"). For the moment I am updating some files manually. Send me a PM if you favorite file is still from the 8th of August.

[Wipneus]

A late century drop in extent shows there are still possibilities for second lowest place ( in my limited data set).

Here is an animation of the Arctic Basin compared with 2012. Click to start.

[Rodius]

To me, even though 2012 was significantly lower in extent, 2019 look much worse.
2012 at least looked solid in the middle.
2019 looks like it is shattered to pieces.

[VeganPeaceForAll]

Here is an animation of the Arctic Basin compared with 2012. Click to start.

That is a very good animation, really shows the bad state of the ice. How do you get the original pictures? Can you send a link?
Thanks

[Wipneus]

That is a very good animation, really shows the bad state of the ice. How do you get the original pictures? Can you send a link?
Thanks

V. these are "original pictures" generated by me from binary data from the University of Hamburg.

Perhaps have a look at the first few posts in this thread for more explanations. The current location of the data files are here:
ftp://ftp-projects.cen.uni-hamburg.de/seaice/AMSR2/3.125km

[dnem]

To me, even though 2012 was significantly lower in extent, 2019 look much worse.
2012 at least looked solid in the middle.
2019 looks like it is shattered to pieces.

I totally agree.  This is why I think ANY curve-fitting exercises using extent are a fools errand.  A-Team said several times "A complete late summer blow-out can happen any year" (or very similar words).  I would be shocked if we don't see, by 2030 or before, the late summer pack reduced to separate, isolated patches of ice pushed by wind and current against isolated islands and shores on the Atlantic side.

[Steven]

Wipneus, If it's not too much trouble, could you update the JAXA AMSR2 melt graphs on your website?  Those graphs haven't been updated since August 2019.

https://sites.google.com/site/arctischepinguin/home/sea-ice-extent-area/grf/jaxa-amsr2-melt-extent-ratioC.png

[Steven]

I see that Wipneus has updated the graphs in the meantime (thanks, Wipneus).  The graphs seem to be in line with NSIDC area and SMOS, suggesting strong surface melting in the first week of July 2020, but a slowdown in the past two weeks:

[Wipneus]

I see that Wipneus has updated the graphs in the meantime (thanks, Wipneus).

Just fixed those Stephen. The slowdown in melting area is indeed strong, perhaps I can illustrate it with an animation.

[Wipneus]

Compare that with 2012 which maintains the area in melting mch more.

[igs]

I see that Wipneus has updated the graphs in the meantime (thanks, Wipneus).  The graphs seem to be in line with NSIDC area and SMOS, suggesting strong surface melting in the first week of July 2020, but a slowdown in the past two weeks:

And exactly that can't be true, at least someone would have to come up with a very sophisticated theory to explain that.

The model as mentioned so many times is flawed under certain circumstances and i think the issue is that it's based on area and area itself is flawed under summer high melting conditions.

[Wipneus]

Uni Hamburg AMSR2 2012 sea ice concentration is only available from first of August. Here is an animation of the first six days  with 2020 and 2012 side by side.

[anaphylaxia]

Interesting to see, how the 2015 collapse of the Vavilov ice cap creates a cold finger at SZ, leading to remnants of sea ice in the vicinity of +4 C waters.
*messed up the quotation*

[gandul]

I see that Wipneus has updated the graphs in the meantime (thanks, Wipneus).  The graphs seem to be in line with NSIDC area and SMOS, suggesting strong surface melting in the first week of July 2020, but a slowdown in the past two weeks:

And exactly that can't be true, at least someone would have to come up with a very sophisticated theory to explain that.

The model as mentioned so many times is flawed under certain circumstances and i think the issue is that it's based on area and area itself is flawed under summer high melting conditions.

Dude, he’s talking about three different satellite sensors suggesting the same thing (how surface melting has evolved during this summer). What exactly are you talking about?

[seaice.de]

Please check this out: 

http://seaice.de/Arc_AWI_AMSR2_latest.png 
http://seaice.de/Arc_AWI_AMSR2_latest.tiff

http://seaice.de/Ant_AWI_AMSR2_latest.png 
http://seaice.de/Ant_AWI_AMSR2_latest.tiff

Should be based on latest 24 hour AMSR2 swath data if the automated processing works.

This is still under development. Feedback about the data format is appreciated.

Landmask=101
Missing=102

Is the colorscale fine?
Do you need NetCDF?
Does the GeoTiff projection work with your GIS as it should? I tried QGIS only.

New land mask: Danmark and Hagen Fjord north of Greenland are now visible. This is a bug in most other data products!

After some testing I will provide the reprocessed data and updates through AWI ftp site. UHH product will cease.

[Jim Hunt]

Please check this out: 

Thanks Lars!

Do you need NetCDF?

Yes please, but Wipneus's mileage may vary.

[oren]

Thank you for sharing Lars. Do you have a specification of what is different between the new product and the UHH product?
It looks great, though I can see several very thin arcs of artifacts in the ESS, Laptev, Kara and Barents.

[seaice.de]

Thank you for sharing Lars. Do you have a specification of what is different between the new product and the UHH product?
It looks great, though I can see several very thin arcs of artifacts in the ESS, Laptev, Kara and Barents.

Thanks for the prompt feedback!

This is still a raw version without filtering. The arc artifacts (end of swaths) will be removed in a next version.

Main differences to UHH product by now:

1) landmask for Arctic and Antarctic
2) data format and size (<1 MB GeoTIFF and png)
3) color table, a subtle difference but some might care

[igs]

Dude, he’s talking about three different satellite sensors suggesting the same thing (how surface melting has evolved during this summer). What exactly are you talking about?

Just talk to me in a moderate and non-condescending tone and I'm not a DUDE.

If you have to explain something do it case oriented and i referred to the text not the image.

There can be misunderstandings as well as underlaying implications that can easily be sorted in a more civil manner.

[uniquorn]

Thank you for sharing Lars. Do you have a specification of what is different between the new product and the UHH product?
It looks great, though I can see several very thin arcs of artifacts in the ESS, Laptev, Kara and Barents.

Thanks for the prompt feedback!
This is still a raw version without filtering. The arc artifacts (end of swaths) will be removed in a next version.
Main differences to UHH product by now:
1) landmask for Arctic and Antarctic
2) data format and size (<1 MB GeoTIFF and png)
3) color table, a subtle difference but some might care

The colour Look Up Table is great, might even put an end to the perennial cloud discussion. It's a bit smaller than the large uhh.
edit:<<
Histogram numbers and chart overlayed onto the first image.
Glasbey on dark LUT applied to the second.
GIMP edge detect on third with some white fill on possible cloud.>>

[johnm33]

Please check this out: 

If i may as an amateur, that's a wonderfully clear sea ice graphic, now all we need is a fitted bathymetry which imho would add almost an order of magnitude more information about the ocean. [and save me loads of time] failing that if the pole was actually dead center it would make aligning things to it simpler.

[uniquorn]

development amsr2, jul6-7(removed). A couple of small islands north of NSI are missing.

[oren]

Lars, I have another amateurish question. Have you considered implementing the following filtering methods described by Wipneus in the top post of this thread? Or maybe you already use these methods?

3) Spurious ice is removed from coastlines where there is open water within a "few" grid cells;

4) "phantom" ice fleets that appear and disappear randomly, especially at lower latitudes are detected and removed

[uniquorn]

Hopefully I'm being helpful here. Herald island is also missing. These islands make a difference when searching for shoals/anchored ice

[slow wing]

Really amazing graphics above, thanks to those posting!

[uniquorn]

aug6-8 with LUT overlay. (forgot date labels)
The missing islands are possibly a resolution issue. There are also some missing around Svalbard.
There is probably going to be a graticule, can that be optional so as not to obscure ice data?

[uniquorn]

difference is good, aug7diff8. Have to sort out a scale.
comparison of '3 day darken' in gimp to minimise clouds, equivalent to a worst case scenario for concentration. (crop/scaling not quite right) click

[A-Team]

Lars, thanks for giving us a chance to put in some early end-user suggestions before the concrete sets on the automated archive pipeline.

You recently completed a very extensive leg on the Polarstern this winter. One of the expressed scientific purposes of the Mosaic mission was to study air, ice and water properties over time that affect interpretation satellite imagery taken simultaneously during overhead orbits.

https://twitter.com/seaice_de?lang=en

While that is described to some extent in the original Mosaic planning document and no doubt better in cruise observation-based manuscripts in preparation, can you give us an overview of how the forthcoming new AMSR2 archive will specifically benefit, relative to the old archive at UHH?

The images shared above are intermediate in resolution to the 3.125 and 6.25 km files served by UHH. The former resolution was a bit of an algorithmic stretch (as described in your earlier paper). Is the new resolution better than the 6.25 but not aspirational to 3.125 because of improved error management (ie better concentration accuracy?

I'm also concerned about continuity with the UHH AMSR2 time series which goes back to 01 Aug 2012. My understanding is the new archive will start when it starts and not go back at all.

Normally this wouldn't be a problem (if people have archived UHH before it goes offline) because ordinary users here could open the netCDF .nc files in user-friendly Panoply, put both to a common map scale, map projection, palette, graticule, legend overlays, mp4 time series, arithmetic interactions between different data sets, and spreadsheet data export out of netCDF.

However the netCDF files for UHH were not constructed as fully georeferenced tto provide geoTiff (geo2D) mappable objects. Hopefully we can test your kind offer above to provide sample AMRS2 .nc files so that people here can test for usability.

Of the ~1800 forum members, I would guess no more than 1-2% work in command-line / Linux mode / specialized GIS software. The free Goddard Institute wysiwyg software seems the only option that provides a good level of inclusivity (ie to distribute the workload over more people).

Panoply plots geo-referenced and other arrays from netCDF, HDF, GRIB, and other datasets. Panoply is a cross-platform application that runs on Macintosh, Windows, Linux and other desktop computers. The current version of Panoply is 4.11.5, released 2020-07-15.

https://www.giss.nasa.gov/tools/panoply/

[seaice.de]

development amsr2, jul6-7(removed). A couple of small islands north of NSI are missing.

This problem of the land mask is hopefully solved, the small islands are now in.  I checked e.g. Herald Island.

Thank you for this hint, Uniquorn!

[Jim Hunt]

Of the ~1800 forum members, I would guess no more than 1-2% work in command-line / Linux mode / specialized GIS software.

I am finally a member of the 1%! I even possess the source code to some "specialized GIS software"!!

Welcome back to the fray A-Team.

[uniquorn]

Thanks seaice.de, pixel count over the last 5days
Something harmonic at 33, 50 and 66?

[uniquorn]

Here is a rather gaudy attempt to highlight the very lowest concentration pixels shown in the charts above. While less aesthetically pleasing than the default it does show remaining ice in the hudson bay, which is just about visible on the default.
Hopefully the next version will retain this information for those curious enough to extract it. It would perhaps help prevent over expectation of ice going 'poof'
Forgot to change the land colour. NucMed LUT Split_BlackBlue_RedW with 0 changed to black. Should be able to find or create something more attractive

[A-Team]

Imagej can provide sorted palette capture from the geoTiff example of the new AMSR2 ice concentration file Lars supplied above, using Image -> Color -> Edit LUT.

That palette and its inverse are shown 3x enlarged as gapped and ungapped images. There are 100 colors for the 100 concentration percentages plus 1 for the land mask and 1 for the pole hole lacking satellite data.

The palette is quite pleasant but it's difficult for the eye to distinguish nearby color squares. Perhaps 20 colors would have sufficed in 5% blocks. It would require digging into the raw and processed satellite data to determine whether the instrument really could make such fine distinctions. A ramped palette might also work better.

Here various statistical properties of the data needs to be studied, not only the global palette use histogram (see uniq's data above) but also local variation (do the values vary wildly from the local mean in a 5x5 pixel block or tend to be related?).

That is easy to test in gimp via eyedropper radius settings. Most likely the local variance is minimal in the central CAB but varies a lot towards the edges or over isolated floes so an overlay map of the distribution is needed.

A palette is normally embedded into the image, over land or unused open water, often here probably best as a 10x100 rectangle. Once embedded, shift-clicking on a range allows a palette color or whole range to have its color changed throughout the data.

The gif below lumps rows of the palette into different colors. It shows that much of the palette is hardly used except in the actively melting periphery. Cloud minimization will be very important here to develop accurate images.

Rescaling to a smaller size is very problematic because it cannot be done without dithering of colors. The palette as it comes from ImageJ consists of 10x10 pixel squares so could be halved using 'none' as the interpolation method. However the data layer would dither unless forced to stay within its indexed color palette.

[uniquorn]

Thanks. That's more like what I was aiming for, though I'd still be reluctant to lose the gradient in low concentration.
5day pixel count beginning to show a trend....updated below
Another clumsy example of highlighting very low conc. aug6-13

[A-Team]

Below is a better color-resolved substitution (click to compare at original scale). It uses matched-pair palettes from Colorbrewer2 to replace blocks of 8 original colors in the AWI original. In other words, 1% increments may be 'too much' resolution for the eye (eg 100 colors) whereas as 17 colors simplifies the ice pattern, bringing out the clustering in the Chukchi-Beauford as well as a stream of wind-borne clouds from Ellesmere to the NSI.

Herald Island by Wrangel has been successfully added but two of the DeLong Islands are still missing from the map though the scale is such that a pixel or two is warranted for Jeanette in view of the epic disaster and its effect on drifting ice and perhaps Vilkitsky too for completeness.

[uniquorn]

5day avg pixel count updated below

[uniquorn]

amsr2-awi-dev 5day avg pixel count