Support the Arctic Sea Ice Forum and Blog

Author Topic: Home brew AMSR2 extent & area calculation  (Read 1353642 times)

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2400 on: July 22, 2016, 10:41:11 AM »
TERRA confirms the weakening of the landfast ice on NE Greenland coast. Apart from the notch there is also a hole opening where the ice is thin.
http://go.nasa.gov/2acdEvS

Hi Andreas,

In the attached animation Sentinel 2A is showing (despite clouds)  a long crack (system) running from the top of the image to south of the 79N glacier (orange arrow). That means that half of that fast ice has unfastened.
 

johnm33

  • ASIF Upper Class
  • Posts: 1237
    • View Profile
  • Liked: 52
  • Likes Given: 24
Re: Home brew AMSR2 extent & area calculation
« Reply #2401 on: July 22, 2016, 11:14:50 AM »
Andreas T" there is also a hole opening"
The ice giant wakes from his slumber and prepares to speak

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2402 on: July 22, 2016, 06:46:58 PM »
Small uptick in area, still very close to 2012.

Update 20160721.

Extent: -69.9 (-231k vs 2015, -354k vs 2014, -135k vs 2013, +36k vs 2012)
Area: +9.8 (-282k vs 2015, -662k vs 2014, -479k vs 2013, +1k vs 2012)
 
You will find the updated graphs in the top post

Only small changes in regional extent, ESS largest at -17k.

Regional area rebounded in the CAB: +70k. ESS dropped -25k.

I attach the regional CAB area again. There is an impressive drop of area, only partial recovery by today's rebound. Notice the purple line, that is Jaxa AMSR2 for 2016.
Jaxa shows no impressive drop, neither is there a rebound.
It happened before (see end of May). Jaxa is probably the more accurate assessment here, and most likely the two lines will come together in a few days.

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2403 on: July 22, 2016, 06:53:44 PM »
Animation of the Canadian Archipelago. Ice is mobile through the entire Parry Channel now.

JimboOmega

  • ASIF Citizen
  • Posts: 139
    • View Profile
  • Liked: 1
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2404 on: July 22, 2016, 10:40:32 PM »
Animation is of the Greenlands Sea, where the ice is having a hard time. The fast ice in NE Greenland can been seen dropping a little notch at the top. I expect bigger losses soon.

What are the pulses of white in this image? It looks like waves... is that related to weather or satellite coverage or something?

Nightvid Cole

  • ASIF Citizen
  • Posts: 433
    • View Profile
  • Liked: 5
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2405 on: July 23, 2016, 04:05:08 AM »
Animation is of the Greenlands Sea, where the ice is having a hard time. The fast ice in NE Greenland can been seen dropping a little notch at the top. I expect bigger losses soon.

What are the pulses of white in this image? It looks like waves... is that related to weather or satellite coverage or something?

It's the satellite sensors picking up cloud cover.

Andreas T

  • ASIF Upper Class
  • Posts: 1124
    • View Profile
  • Liked: 12
  • Likes Given: 2
Re: Home brew AMSR2 extent & area calculation
« Reply #2406 on: July 23, 2016, 12:06:51 PM »
It would be wrong to think that these images are only made whiter by weather effects. A-team seemed to suggest that taking the darkest pixels to be found in the animation gives a "truer" picture of the condition of the ice. That is not correct. Weather effects can also darken the image, ice which disappears in the animation does reappear later. It is best  always to check visible imagery to back up what is indicated by the passive microwave product.

A-Team

  • ASIF Upper Class
  • Posts: 2317
    • View Profile
  • Liked: 239
  • Likes Given: 14
Re: Home brew AMSR2 extent & area calculation
« Reply #2407 on: July 23, 2016, 03:03:35 PM »
Quote
Weather effects can also darken the image
Really? Show us an example of this on AMSR2 3K in July 2016. Here is the web site if you are not familiar with it: ftp://ftp-projects.zmaw.de/seaice/AMSR2/3.125km/

Even if you can find some pixels doing this under uncommon atmospheric conditions, if 95% of the time clouds reduce the blue assignment, then overall the image will be vastly improved in accuracy, especially in regions with persistent but not constant cloud cover after correction for ice pack displacement because sea ice concentration in the central Arctic does not often change radically from day to day.

Overall it is astonishing that we can see clouds sweeping across these animations -- always white, never blue -- yet UH has not made more of an effort to remove cloud artifacts.

In previous posts, I proposed four different methods for improving this product that are variations on using the simultaneity of Modis visual and cloud band imaging on the satellite source for the AMSR2 3k to provide a grayscale cloud thickness mask.

It would be great to have daily sea ice concentration at pixel-perfect precision but actually no one aspires to anything more than estimates. The problem really is reducing the wild swings in net rms error from day to day (or year on year).

Frankly speaking, S2AB is far better suited for comparison of product to reality than the 250 m Modis where the 16 pixels available a 1 km x 1 km curved floe already has serious problems with overlap pixels -- previous posts already compared Modis visible to 3-channel 10m S2AB having 10,000 pixels for this same floe.

With active radar like wide swath, high coverage S1A, the image derives from a reflected strong satellite-provided signal, not summed from weak and ambiguous atmospheric, water, snow and ice graybody emissions at vastly inferior resolution not commensurate with the scale of what needs to be imaged (eg narrow leads).

We've looked very closely at simultaneous S1A/S2AB over at the fiducial settings of Jakobshavn and Petermann forums: even the densest clouds have little effect on ground image quality, notably melange floe resolution. There's limited interest in 'fjord sea ice concentration' per se but the issues are very similar to the Arctic Ocean. (Petermann is 900 km north of Barrow AK.)

So to the extent the AMSR2 3k product will ever be improved for the Arctic Ocean, S1A will likely furnish the information. Note that PolarView provides S1A frames directly over a current sea ice concentration product. It doesn't get any more convenient than that -- and suggests this improvement (or replacement!) is already under development. http://www.polarview.aq/arctic

The fact is, some days are more favorable for an accurate AMSR2 3k product in specific regions than others. These better days have information that can improve previous and following days in regions where they are worse. The question is how to best bring that into the picture (at the expense of other but fewer artifacts); below in another variation, each day was paired with the previous using blueness overwriting (available when a over-tint linear palette is used, as at the UH source).

Deciding where is a topic in image segmentation; there's a vast technical literature on how to do that optimally. However it won't be easy to improve on some of the palette replacement schemes already posted.
« Last Edit: July 23, 2016, 05:49:54 PM by A-Team »

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2408 on: July 23, 2016, 03:12:56 PM »
Big declines in extent and area, now mostly within the Arctic Basin.

Update 20160722.

Extent: -131.2 (-215k vs 2015, -442k vs 2014, -165k vs 2013, +31k vs 2012)
Area: -112.5 (-220k vs 2015, -649k vs 2014, -511k vs 2013, -121k vs 2012)
 
You will find the updated graphs in the top post

Regional extent dropped most in ESS (-29k), Beaufort (-28k) and CAB (-28k).

Regional area dopped most in ESS (-49k) and Beaufort (-38k). CAB increased by +26k.

Regional delta map is Beaufort. Lots of leads and polynya opening in the northern part.

AmbiValent

  • ASIF Citizen
  • Posts: 122
    • View Profile
  • Liked: 4
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2409 on: July 23, 2016, 05:11:18 PM »
Is the value for 2012 already from AMSR2 data? If not, when will AMSR2 data start?
Bright ice, how can you crack and fail? How can the ice that seemed so mighty suddenly seem so frail?

Andreas T

  • ASIF Upper Class
  • Posts: 1124
    • View Profile
  • Liked: 12
  • Likes Given: 2
Re: Home brew AMSR2 extent & area calculation
« Reply #2410 on: July 23, 2016, 05:22:53 PM »
this is a comparison of
 ftp://ftp-projects.zmaw.de/seaice/AMSR2/3.125km/Arc_20160711_res3.125.png
with
http://go.nasa.gov/2a5soKS
look at the area between Greenland and the pole. The AMSR2 graphic shows ice concentration of 70% ? in any case similarly low as some areas towards Siberia.
That is not what MODIS shows. If anything the lighter blue shows suggests less watery surface than surrounding areas.
The 89GHz emissivity of water is lower than that of ice, could the algorithm misinterpret low brightness temperature for  low ice concentration? Or am I getting something else wrong?

magnamentis

  • Guest
Re: Home brew AMSR2 extent & area calculation
« Reply #2411 on: July 23, 2016, 06:46:54 PM »
Is the value for 2012 already from AMSR2 data? If not, when will AMSR2 data start?

ready to  be corrected but that was somewhat in summer 2013

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2412 on: July 23, 2016, 06:57:20 PM »
Northern Hemisphere UH AMSR2 data is only available from 1 Jan 2013, SH a few months later.

Now someone at Uni Hamburg could, if he or she found the time, I know it is hard, process the available 2012 AMSR2 data and fill the gap (from the end of July 2012).

AmbiValent

  • ASIF Citizen
  • Posts: 122
    • View Profile
  • Liked: 4
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2413 on: July 23, 2016, 07:00:54 PM »
Northern Hemisphere UH AMSR2 data is only available from 1 Jan 2013, SH a few months later.

Now someone at Uni Hamburg could, if he or she found the time, I know it is hard, process the available 2012 AMSR2 data and fill the gap (from the end of July 2012).
I meant, your area graph has a 2012 line that starts in July 2012, so I thought you have that data. You still already compare 2016 to 2012, so what do you compare it to, and when can you switch to AMSR2 to compare apples with apples?
Bright ice, how can you crack and fail? How can the ice that seemed so mighty suddenly seem so frail?

Jim Hunt

  • ASIF Governor
  • Posts: 4051
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 131
  • Likes Given: 21
Re: Home brew AMSR2 extent & area calculation
« Reply #2414 on: July 23, 2016, 07:20:13 PM »
I meant, your area graph has a 2012 line that starts in July 2012, so I thought you have that data.

Most of the graphs have UH ASI SSMIS for all of 2012. Some have JAXA Bootstrap AMSR2 starting in July 2012. Is that what you're thinking of?
Reality is merely an illusion, albeit a very persistent one - Albert Einstein

AmbiValent

  • ASIF Citizen
  • Posts: 122
    • View Profile
  • Liked: 4
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2415 on: July 23, 2016, 07:25:03 PM »
I meant, your area graph has a 2012 line that starts in July 2012, so I thought you have that data.

Most of the graphs have UH ASI SSMIS for all of 2012. Some have JAXA Bootstrap AMSR2 starting in July 2012. Is that what you're thinking of?
Probably... but which of them does he use when comparing area in his daily posts?
Bright ice, how can you crack and fail? How can the ice that seemed so mighty suddenly seem so frail?

A-Team

  • ASIF Upper Class
  • Posts: 2317
    • View Profile
  • Liked: 239
  • Likes Given: 14
Re: Home brew AMSR2 extent & area calculation
« Reply #2416 on: July 23, 2016, 08:26:20 PM »
In the animation below, the palette usage in the July 22nd AMSR2 3k has been determined by separate pixel counts for each of the 100 color bins. Then the palette was replaced by the indicated colors for blocks of 10 bins and the histogram graph overlaid.

On this date, 59.6% of sea ice concentration was attributed to the 90-100% bin and so forth. Individual bin counts and tallies are given in the attached spreadsheet. It would be interesting to flash these histograms -- which in reality should be quite stable -- over a short time frame of unexceptional weather during which clouds can be seen flitting across the 2D display of sea ice concentration.

For automated motion correction estimation, the best fit to 1 pixel displacement involves four steps (directions). The best fit to a two step displacement (eg east, then north) requires 12 trials. or 4*3n-1 for higher (eg east, then west makes no sense). These are vetted for min|day1-day2| pixel counts.
« Last Edit: July 24, 2016, 12:44:50 AM by A-Team »

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2417 on: July 24, 2016, 08:43:25 AM »

I meant, your area graph has a 2012 line that starts in July 2012, so I thought you have that data. You still already compare 2016 to 2012, so what do you compare it to, and when can you switch to AMSR2 to compare apples with apples?

That graph (and the extent graph as well) mentions: "UH SSMIS 12.6 km (2012)". That is the data I use for 2012 and while not perfect it is the best I could find ( and quite reasonable IMO).

Similar questions are asked (and answered) from time to time, most recently on 26th June and 19th July:
http://forum.arctic-sea-ice.net/index.php/topic,382.msg81392.html#msg81392
http://forum.arctic-sea-ice.net/index.php/topic,382.msg84081.html#msg84081

AmbiValent

  • ASIF Citizen
  • Posts: 122
    • View Profile
  • Liked: 4
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2418 on: July 24, 2016, 09:21:38 AM »
I think the heat is baking my brain, making me stupid... so you do use UH SSMIS values not only in that graph, but your daily posted area value also used that as comparison for 2012?
Bright ice, how can you crack and fail? How can the ice that seemed so mighty suddenly seem so frail?

Rob Dekker

  • ASIF Upper Class
  • Posts: 2269
    • View Profile
  • Liked: 90
  • Likes Given: 105
Re: Home brew AMSR2 extent & area calculation
« Reply #2419 on: July 24, 2016, 09:28:46 AM »
Wipneus, I think Ambivalent is referring to the AMSR2 data stream came on-line in July 2016, which is visible in the "melt extent" graphs that you have been posting until a few days ago :
http://forum.arctic-sea-ice.net/index.php?action=dlattach;topic=382.0;attach=32777;image

For AmbiValent : I think Jim Hunt has it right :

Quote
Most of the graphs have UH ASI SSMIS for all of 2012. Some have JAXA Bootstrap AMSR2 starting in July 2012. Is that what you're thinking of?

Wipneus uses UH ASI SSMIS for all of his 2012 comparisons, and that new stream that came on-line in July 2012 is JAXA AMSR2, which Wipneus uses for the "melt extent" graphs, but NOT for his day-to-day comparisons between 2012 and 2016.

Now, Wipneus, please shoot me down if I (or Jim Hunt) got that wrong  :)
This is our planet. This is our time.
Let's not waste either.

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2420 on: July 24, 2016, 09:56:12 AM »
Neck to neck race continues.

Update 20160723.

Extent: -126.7 (-235k vs 2015, -525k vs 2014, -235k vs 2013, +7k vs 2012)
Area: -71.8 (-223k vs 2015, -616k vs 2014, -535k vs 2013, -78k vs 2012)
 
You will find the updated graphs in the top post

Largest extent declines are in the ESS (-33k), Beaufort (-28k), Chukchi (-24k) and CAA (-24k).

Largest area declines in CAA (-21k) and Laptev (-18k). CAB increased by +24k.

The delta map is from the Canadian Archipelago. Two days ago the ice in the Parry Channel started to get mobile, now there is a lot of ice opening there.


Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2421 on: July 24, 2016, 10:00:59 AM »
I think the heat is baking my brain, making me stupid... so you do use UH SSMIS values not only in that graph, but your daily posted area value also used that as comparison for 2012?

Yes, for the reasons given:

There is no UH AMSR2 2012 data
The UH SSMIS 2012 is the best I can use  as it tracks UH AMSR2 in other years (2013 and 2014) well.

See also the answers in the given links.

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2422 on: July 24, 2016, 10:12:00 AM »
The melting state (from ADS/Jaxa AMSR2 data) in the last week. The melting extent nearly touched 90% at a point. Similar to 2013 and 2015, clearly above 2014. During the coming weeks comparisons with 2012 will be possible.
« Last Edit: July 25, 2016, 02:02:53 PM by Wipneus »

A-Team

  • ASIF Upper Class
  • Posts: 2317
    • View Profile
  • Liked: 239
  • Likes Given: 14
Re: Home brew AMSR2 extent & area calculation
« Reply #2423 on: July 24, 2016, 08:32:35 PM »
Looking now at the netCDF file for 22 July 16 at AMSR2 3k with the latest Panoply 4.5.1 freeware alternative to matlab, it seems that only the landmask was furnished with the georeferencing necessary to display the data in the various projections provided.

However the first image below does get it more or less recognizably displayed without lat/lon grid lines overlying sea ice concentration data. I managed to capture the original 100 color palette (long story in gimp) and change it to an arbitrary palette (long story in gimp), the one shown being a pure gradient tint.

Such a palette is better suited for comparing one day to the next (subtraction of pixel value then stays within palette), though Panoply is all set up to perform the subtraction on the numerical data matrices and display the results over a wide range of dates. The .csv excel file here is ~35 MB.

There are some interesting factoids in the .nc metadata, notably links to the JAXA orbital swath files actually used by the ARTIST algorithm to determine sea ice concentration. Needless to say these won't agree with the WorldView cloud view channels for the given date if these were made by a different process using different swaths with different time stamps, ie the clouds move so quickly relative to swath-taking that only a simultaneous cloud image would be applicable.

It's not clear to me how to build the full url from these partial links ... seems that backslashes define file paths on someone's Window hard drive with .gs being Windows .gzip whereas downloadable URLs have forward slashes. So are these swaths online or not? (The .h5 is another netCDF-like can of worms called hdf5 that Panoply can open.)

Quote
:Comment = "List of swath files used for the daily mean: [\'GW1AM2_201607220009_183D_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607220058_183A_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607220148_199D_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607220237_199A_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607220327_215D_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607220416_215A_L1SGRTBR_2210210.h5.gz\',
...
\'GW1AM2_201607222134_158D_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607222224_158A_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607222313_174D_L1SGRTBR_2210210.h5.gz\']";
  :grid_resolution = "3.125 km";

Summary = "The land is not masked out in the data. The user should mask the data with their own landmask or use the 3.125 km landmask given in the product, which is a regridding of the 12.5 km landmask from NSIDC.";

ftp://ftp-projects.zmaw.de/seaice/AMSR2/3.125km/
« Last Edit: July 25, 2016, 12:09:31 AM by A-Team »

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2424 on: July 25, 2016, 09:00:53 AM »
Looking now at the netCDF file for 22 July 16 at AMSR2 3k with the latest Panoply 4.5.1 freeware alternative to matlab, it seems that only the landmask was furnished with the georeferencing necessary to display the data in the various projections provided.

It uses Polar Sterographic projection (that should be documented somewhere as well). That is the standard for sea ice concentration data (seen others but always beside a PS offering of the same data).

Quote

It's not clear to me how to build the full url from these partial links ... seems that backslashes define file paths on someone's Window hard drive with .gs being Windows .gzip whereas downloadable URLs have forward slashes. So are these swaths online or not? (The .h5 is another netCDF-like can of worms called hdf5 that Panoply can open.)

:Comment = "List of swath files used for the daily mean: [\'GW1AM2_201607220009_183D_L1SGRTBR_2210210.h5.gz\', \'GW1AM2_201607220058_183A_L1SGRTBR_2210210.h5.gz\',
(...)"
 

These are not paths but filenames, it is given by a Python specification that required here the escaping (backslash) of the single quote characters. Lazy programmer that could not imagine someone was going to actually read this stuff, if you ask me.

Full urls would be something like:
sftp://gcom-w1.jaxa.jp/AMSR2/2016/2016.07/L1/L1R/2/<filename>

That is an sftp protocol, not likely to work in your browser.

To use it go to

gcom-w1.jaxa.jp

Do the free registration if needed, and have a look around. I suggest having a look at the user documentation and at the "SFTP" -tab for help with the sftp protocol.
 

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2425 on: July 25, 2016, 01:42:41 PM »
An uptick in area. Still second, but 2015 is coming nearer.

Update 20160724.

Extent: -54.1 (-203k vs 2015, -505k vs 2014, -277k vs 2013, +70k vs 2012)
Area: +39.1 (-130k vs 2015, -530k vs 2014, -410k vs 2013, +42k vs 2012)
 
You will find the updated graphs in the top post

ESS extent keeps declining firmly, now -27k. Chukchi second (-19k) and CAB increased by +16k.

CAB area increased by +43k, also ESS and Beaufort grew in area (+16k and +18k). Area in Laptev declined by -22k.

Regional delta map is East Siberian Sea. Some last remains of the former fast ice is disappearing and the ice edge with the main pack retreating.


iceman

  • ASIF Citizen
  • Posts: 277
    • View Profile
  • Liked: 3
  • Likes Given: 10
Re: Home brew AMSR2 extent & area calculation
« Reply #2426 on: July 25, 2016, 01:44:01 PM »
The melting state (from ADS/Jaxa AMSR2 data) in the last week. 
   ....

Are the dates correct? Looks like the animation is running in reverse.

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2427 on: July 25, 2016, 01:57:06 PM »
Animation of the sea ice concentration in the Laptev section. The ice is declining but slow and much later than in other years. Some ice can be seen is replenished from the main ice pack, I don remember seeing that.

(needs a click to start)

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2428 on: July 25, 2016, 02:00:39 PM »
The melting state (from ADS/Jaxa AMSR2 data) in the last week. 
   ....

Are the dates correct? Looks like the animation is running in reverse.

Check: dates are correct. But the images are in the wrong order, not nice, pain on the brain. I will sort them in a minute. Thanks!

Siffy

  • ASIF Citizen
  • Posts: 179
    • View Profile
  • Liked: 0
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2429 on: July 25, 2016, 05:46:24 PM »
out of interest is the uptick in part due to the large cloud cover currently obscuring sat sensors ability to get an accurate look at the ice?

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2430 on: July 26, 2016, 12:29:38 PM »
out of interest is the uptick in part due to the large cloud cover currently obscuring sat sensors ability to get an accurate look at the ice?

Could be Siffy. However the influence of water vapor and the liquid in cloud gets compensated for (in a necessary imperfect way), so the influence of those may work both ways.
From looking at the regional graphs, comparing Jaxa and Uni Hamburg AMSR2 data, I get the impression that the uptick is a return to normal (area was too low previously).

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2431 on: July 26, 2016, 12:36:59 PM »
The race continues with 2016 in the lead, 2015 area has approached within a single century distance.

Update 20160725.

Extent: -48.4 (-168k vs 2015, -485k vs 2014, -278k vs 2013, -22k vs 2012)
Area: -69.4 (-80k vs 2015, -523k vs 2014, -378k vs 2013, -4k vs 2012)
 
You will find the updated graphs in the top post

No remarkable regional extent changes, Hudson managed to squeeze another -19k from somewhere.

Also modest regional area changes, ESS was -20k, CAB -16k.

 

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2432 on: July 26, 2016, 12:49:07 PM »
Greenland Sea animation, side by side with 2013. In 2013 was a great melt year for the NE Greenland fast ice (in some contrast with more central regions of the Arctic). Nearly all of it gone.  Both years seem to be comparable at the moment, currently most of the fast ice is mobile except a piece off the coast grounded on "Belgica Bank" and Tobias Island.
Of course the forum is a great historical log book:

http://forum.arctic-sea-ice.net/index.php/topic,238.0.html
http://forum.arctic-sea-ice.net/index.php/topic,633.0.html

Click to start the animation.

binntho

  • ASIF Citizen
  • Posts: 459
    • View Profile
  • Liked: 121
  • Likes Given: 46
Re: Home brew AMSR2 extent & area calculation
« Reply #2433 on: July 26, 2016, 12:55:10 PM »
The difference in the ice-edge between the two years is intriguing.

iceman

  • ASIF Citizen
  • Posts: 277
    • View Profile
  • Liked: 3
  • Likes Given: 10
Re: Home brew AMSR2 extent & area calculation
« Reply #2434 on: July 26, 2016, 02:28:06 PM »
Animation of the sea ice concentration in the Laptev section. The ice is declining but slow and much later than in other years. Some ice can be seen is replenished from the main ice pack, I don't remember seeing that.

Looks unusual to me as well.  In another forum thread, someone noted the persistence of cyclonic weather in the central Arctic this season, which could be an explanation.  We'll see a couple more days of the same before the replenished ice gets rained on by a storm moving in from the Siberian coast.

A-Team

  • ASIF Upper Class
  • Posts: 2317
    • View Profile
  • Liked: 239
  • Likes Given: 14
Re: Home brew AMSR2 extent & area calculation
« Reply #2435 on: July 26, 2016, 05:28:48 PM »
Quote
not paths but filenames, it is given by a Python specification that required here the escaping (backslash) of the single quote characters. Lazy programmer that could not imagine someone was going to actually read this stuff, if you ask me.
Thx for backslash explanation. sftp opens ok after getting re-routed from web browsers to Fetch 5.7.5 freeware, except that gcom-w1.jaxa.jp requires a username/password other than 'guest' or 'anonymous' so registration.

sftp://gcom-w1.jaxa.jp/AMSR2/2016/2016.07/L1/L1R/2/GW1AM2_201607220009_183D_L1SGRTBR_2210210.h5.gz

Quote
It uses Polar Stereographic standard for sea ice concentration data 
The netCDF file at AMSR2 3k, as opened in Panoply, only shows the land mask as georeferenced (Geo2D). The polar stereographic standard at WorldView, NSIDC etc is EPSG 3413. Its two free parameters are latitude of exact scale (70º but elsewhere 75º) and featured longitude ('Greenland down' but elsewhere Greenwich meridian, 45º off). Rotations other than multiples of 90º degrade images.

The option for PS projection in Panoply do not include their EPSG number so it becomes necessary to import EPSG 3413 to be sure. The sea ice concentration does come with lat/lon layers. However upon plotting, it gives a very distorted x,y image with plate carree or mercator-like rectangular grid lines. Looking at the polar hole, that appears as elliptical. By manual rescaling the x coordinate to make the hole into a circle, Panoply was able to produce the PS-like image shown above.

Quote
Lazy programmer that could not imagine someone was going to actually use Panoply, if you ask me.
We are now 14 years into Panoply code development at Nasa/Goddard with scarcely a word of explanation of how to use it (no manual, no example). A couple of offsite .edu attempted tutorials 6-7 years ago but these have become obsolete because the sample files are no longer offered at Panoply. However the version history and list of bug fixes is excellent.

http://www.giss.nasa.gov/tools/panoply/versions.html
http://www.giss.nasa.gov/tools/panoply/

Meanwhile, I took a closer look at resolution and palette colors in UH AMSR2 3k maps of sea ice concentration.

First note that two images for each date are provided. The LARGE version is 2x the size of the normal png. By rescaling with interpolation shut off, followed by pixel differencing in gimp, it emerges that the two versions are identical except in coastal detail and rendering of the annoying lat/lon overlay (which should have been provided separately as a transparent overlay or as pole plus crosses at lat/lon intersections).

In other words, it's worth using the larger format -- 4x the file size adds up -- if making animations of the Northwest Passage because the waterways are better distinguished from land.

Both versions are clean. This is highly unusual elsewhere in climate science imagery. Clean means if you additively color-pick each of the 100 palette bars at radius 0, the entire palette and all the sea ice (and nothing else) are selected. This is the easiest way to mask and replace the orange land, inland lakes, and lat/lon gray grid.

The first image below needs to be viewed and saved at full size if you plan to use its masking. Note how its edges merge right into forum boundary gray. That's because png has an option for saving transparency. If you open the image and delete this alpha channel, the gray will be replaced by whatever is set as background color.

The palette itself looks clean but is not entirely. It is not dithered or show lossy compression that uncouples it from map colors, like 99% of what we see. The black and gray-dithered boundary has various inconsequential mistakes that come from not using photoshop-like preparatory software. The color bars have variable widths, a peculiar design error but not harmful.

The 100 RGB values, as a numerical table listable in ImageJ or described individually in gimp, all have 100% saturation. However there is a peculiar jump midway in blue hue with significant consequences to determining day to day change in time series.

The best way to investigate palettes (which often come without any documentation) is to first expand the initial size, here 774 x 52 pixels, to a 774 x 774 square, making sure to exclude there's been no interpolation (ie still have the same 100 colors). While that's too big to display properly on the forum, the size can be halved to 387 x 387 -- despite the irregular widths -- without any change in palette colors.

Next, the square is duplicated to a second layer and rotated 90º, allowing various interactions of the palette with itself (such as differencing) to be characterized. As shown below, this quickly reveals the peculiarities of the AMSR2 3k palette relative to a constructed palette with a pure hue overtint of regularly spaced grayscales.

Gifs are restricted to 255 colors so only selected 24-bit pngs could be shown, the most common operations we use on these forums. Animations have to dither horribly on color gradients.  The AMSR2 palette, be it bug or feature, gives very complex displays in day to day comparisons.

Palette RGBs are best designed in a spreadsheet where values are under strict control because many software menu algorithms don't work as you might expect. They can be displayed in BMP image format and then interchanged.
« Last Edit: July 26, 2016, 05:42:58 PM by A-Team »

A-Team

  • ASIF Upper Class
  • Posts: 2317
    • View Profile
  • Liked: 239
  • Likes Given: 14
Re: Home brew AMSR2 extent & area calculation
« Reply #2436 on: July 26, 2016, 06:00:30 PM »
Here what those same operations do to a pair of crossed 256 color grayscale and blue overtint reference palettes. I'll add the correctly spaced 100 color versions in a bit. Sea ice concentration maps will "stay within their palette" under common day to day comparisons. The AMSR2 3k image series can be losslessly replaced with either of the palettes below using indexed color techniques.

It's quite feasible to stay in numeric mode, make graphics at the end. Indeed Panoply or matlab is all set up to do day to day differencing for big time series within netCDF mode, which itself is just a way of processing a big stack of Excel spreadsheets without the huge overhead.

It appears to me that programs like Panoply can import palettes and projections, so they do. There's been a huge proliferation of both. However for the polar regions, putting in 40 Panoply projections merely adds clutter around polar stereographic (or Landsat type mercator). On the palette side, most of it is eye candy.

There do exist sets of annotated scientific palettes but how they were made and what they were intended to do soon gets lost. Palettes really need a layer of metadata from which they cannot be separated. Especially in these non-graphical platforms, end users have any idea what is going on, they're just looking for a pretty journal display options. So the programmers try to mollify them by importing a couple hundred palette options.

To me this seems backwards from how science is really done -- first by guess and by golly, then only secondarily backing up to do it over with rigorous formal methods. For example, bathymetric features forcing ocean currents like the Taylor column in the Chukchi or Bear Island hot spot in the Barents -- did someone come up with these flying blind (formal maneuvers perusing netCDF numerical arrays) or by a coffee table comparison of a nautical chart and a National Geographic cover story on the ice pack?
« Last Edit: July 26, 2016, 06:44:10 PM by A-Team »

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2437 on: July 27, 2016, 01:29:04 PM »
Update 20160726.

Extent: -52.4 (-140k vs 2015, -471k vs 2014, -266k vs 2013, +60k vs 2012)
Area: -116.8 (-98k vs 2015, -608k vs 2014, -381k vs 2013, -71k vs 2012)
 
You will find the updated graphs in the top post

Regional extent nowhere changed very much, ESS dopped most by -14k.

Biggest changers in are area CAB (-36k), ESS (-28k), Laptev (-19k) and CAA (-16k).

Regional delta map is from the ESS. A couple of polynya have in the central ice pack have widened considerably. Perhaps the beginning of an ESS-bite, or will there be a connection with a similar feature near Chukchi?

seaicesailor

  • Guest
Re: Home brew AMSR2 extent & area calculation
« Reply #2438 on: July 27, 2016, 02:52:11 PM »
Update 20160726.
...
A couple of polynya have in the central ice pack have widened considerably. Perhaps the beginning of an ESS-bite, or will there be a connection with a similar feature near Chukchi?

That is an interesting thought. When you say connection, do you mean that eventually the polynya might get connected, or that there is a connection in the appearance of these polynya?

Note apart, there are two dents in the ice edge of Chukchi sea that seem well aligned with the incoming currents of warmer water.
Also, see the peculiar circular holes, one in Chukchi and the other almost inside the CAB, more visible in this earlier post:

http://forum.arctic-sea-ice.net/index.php/topic,382.msg84677.html#msg84677

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2439 on: July 27, 2016, 05:08:30 PM »
Update 20160726.
...
A couple of polynya have in the central ice pack have widened considerably. Perhaps the beginning of an ESS-bite, or will there be a connection with a similar feature near Chukchi?

That is an interesting thought. When you say connection, do you mean that eventually the polynya might get connected, or that there is a connection in the appearance of these polynya?
I meant the first.
Quote
Note apart, there are two dents in the ice edge of Chukchi sea that seem well aligned with the incoming currents of warmer water.
Also, see the peculiar circular holes, one in Chukchi and the other almost inside the CAB, more visible in this earlier post:

http://forum.arctic-sea-ice.net/index.php/topic,382.msg84677.html#msg84677

Yes I noticed them too. I am a (littlle) bit surprised at the lack of melting in Chukchi, there is still an very sparse ice field west of Wrangel, hoovering below or just above the 15% threshold.

Attached animation compares with 2015. Click to animate.

JimboOmega

  • ASIF Citizen
  • Posts: 139
    • View Profile
  • Liked: 1
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2440 on: July 27, 2016, 06:11:05 PM »
While following this thread I noticed that 2016 and 2012 are still pretty neck and neck.

When do those who expect this year to be nothing like 2012 expect them to diverge?

seaicesailor

  • Guest
Re: Home brew AMSR2 extent & area calculation
« Reply #2441 on: July 27, 2016, 07:22:08 PM »
Next week

Richard Rathbone

  • ASIF Citizen
  • Posts: 480
    • View Profile
  • Liked: 17
  • Likes Given: 6
Re: Home brew AMSR2 extent & area calculation
« Reply #2442 on: July 27, 2016, 11:11:04 PM »
2015 was neck and neck for a while yet. If it follows the same path as last year, its the week after next where the uncloseable gap opens up.

A-Team

  • ASIF Upper Class
  • Posts: 2317
    • View Profile
  • Liked: 239
  • Likes Given: 14
Re: Home brew AMSR2 extent & area calculation
« Reply #2443 on: July 28, 2016, 02:30:01 AM »
The animation below looks at actual use of intermediate values of sea ice concentration at UH AMSR2 3k. The statistics for 10-26 July 16 (upper right corner) show that taking away the 7 highest percentage bins (248-255: 94% to 100%) and the 3 lowest bins 135-137: 0% to 3%) leaves the other 90 bins combined typical totals 10% or less of the blue tinted pixels in the Arctic Ocean map.

It's not clear whether this is a problem in sensor sensitivity, in the algorithm, or whether sea ice concentration spends very little time in a graded transformation from solid ice to open water.

It's not possible to present the whole Arctic at the scale AMSR2 uses within our 700x700 pixel limit. What's shown is 821x728 -- shrinking beyond that will force dithering upon the precious palette and image, creating all manner of artifacts in the current context.

It might make better sense in terms of scientific visualization to adjust the palette proportionately to balance bin utilization. That's come up so often before that most image processing tools have a command for doing just that (second animation, click needed). While that outcome is 'interesting' for intermediate sea ice concentrations and clouds flitting by, it is ultimately better to replace this palette at the beginning with a pure gradient overtint as described in a previous post.

Click on the forum image to see the usage statistics for each of the dates along with the 2D animation of its 2D distribution.
« Last Edit: July 28, 2016, 03:13:19 AM by A-Team »

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2444 on: July 28, 2016, 02:36:04 PM »
Update 20160727.

Extent: -81.6 (-141k vs 2015, -430k vs 2014, -284k vs 2013, +186k vs 2012)
Area: -134.3 (-122k vs 2015, -674k vs 2014, -416k vs 2013, -75k vs 2012)
 
You will find the updated graphs in the top post

Baffin and CAB have the biggest extent declines, both -17k.

Area in the CAB did a another big drop again: -62k.  Laptev (-28k) and CAA (-17k) follow at distance. The ESS increased area: +17k.

A delta map of the CAA follows. The ice between the islands north of the NW-passage is cracking, as can be seen more clearly on MODIS.

A-Team

  • ASIF Upper Class
  • Posts: 2317
    • View Profile
  • Liked: 239
  • Likes Given: 14
Re: Home brew AMSR2 extent & area calculation
« Reply #2445 on: July 28, 2016, 05:24:23 PM »
The 27th of July offered a rare view of polar front sea ice (it's been cloudy for months) on Suomi Modis visible, which is shown tiled vertically into 3 panels at about 0.66 km/pxl resolution. A bit of Franz Josef, Svalbard and Greenland have been included for orientation in the respective tiles.

Those can be compared to AMSR2 3.125km of the same date which is nominally at 4.7 coarser scale. The AMSR2 could be blown up beyond its resolution and tiled to match but that would give 6 images total whereas the forum is limited to 4 per post. The idea here is to facilitate comparison of details of the ice at the front with the final sea ice concentration product from the same date and satellite.
« Last Edit: July 28, 2016, 05:30:50 PM by A-Team »

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2446 on: July 29, 2016, 11:10:25 AM »
The numbers are big, enough to stay in the race.

Update 20160728.

Extent: -119.4 (-197k vs 2015, -512k vs 2014, -305k vs 2013, +190k vs 2012)
Area: -138.0 (-202k vs 2015, -781k vs 2014, -565k vs 2013, -69k vs 2012)
 
You will find the updated graphs in the top post

Some big declines in the regional extent numbers: Beuafort (-35k), CAA (-25k) and Laptev (-20k).

In the regional area arena, it is ESS that brings the biggest change: -51k, followed by CAA (-40k) and Laptve (-20k).

Regional delta map of Beaufort is attached. The big extent decline is in the eastern half.
« Last Edit: July 29, 2016, 11:36:43 AM by Wipneus »

Wipneus

  • ASIF Governor
  • Posts: 3934
    • View Profile
    • Arctische Pinguin
  • Liked: 414
  • Likes Given: 0
Re: Home brew AMSR2 extent & area calculation
« Reply #2447 on: July 29, 2016, 11:24:16 AM »
Animation of Laptev, the remains of the former fast ice are now melting fast. Time seems to be running out for the ice pack.

(needs a kick to start)

Thawing Thunder

  • ASIF Citizen
  • Posts: 223
    • View Profile
  • Liked: 26
  • Likes Given: 31
Re: Home brew AMSR2 extent & area calculation
« Reply #2448 on: July 29, 2016, 06:45:39 PM »
That's thrilling: The ice seems to behave like a viscous liquid - looks like the current moves it through Vilkitshogo strait and Bolshevik island is creating turbulences that are several hundred kilometers long!
« Last Edit: July 29, 2016, 06:59:13 PM by Thawing Thunder »
The Thunder was father of the first people, and the Moon was the first mother. But Maxa'xâk, the evil horned serpent, destroyed the Water Keeper Spirit and loosed the waters upon the Earth and the first people were no more.

jdallen

  • ASIF Governor
  • Posts: 2951
    • View Profile
  • Liked: 131
  • Likes Given: 154
Re: Home brew AMSR2 extent & area calculation
« Reply #2449 on: July 29, 2016, 07:15:01 PM »
Animation of Laptev, the remains of the former fast ice are now melting fast. Time seems to be running out for the ice pack.

(needs a kick to start)
It has the appearance of a very large area of ice that was approximately the same thickness at the start of the melt season (~1.85M or so?) reaching a critical point in its melt.

Presuming moderately consistent energy applied over time over the area, it would make sense for it to reach the same stage at about the same time.
This space for Rent.