This section goes through real-world examples illustrating how the mail merge tool automates the production of climate science data animations by preparing scripts that PanoplyCL can run to generate the maps that make up the frames.
For brevity, only the relevant lines of a PanoplyCL script are shown. Much longer lists (many hundreds) would normally be used instead of the 4-5 shown in the examples (which could easily be generated manually).
You can best follow along by replicating the examples which requires that the respective netCDF files plus Panoply, PanoplyCL and their manuals have been downloaded, installed, and assimilated.
Example 0 This example just shows how the merge tool can generate a list of download urls for ASCAT daily imagery. Since those are available from 2010 on, leading to 2948 files totaling 2.4 GB up to mid-January 2018. Every 5th day however might give simplified but completely adequate depiction of ice motion. The desired dates are easily built by fill-down in a spreadsheet since ASCAT uses day-number instead of day-month.
Because ftp isn't properly enabled at this host site, the files are best retrieved by 'download it all' or 'bulk url opener' web browser plugins. About 1 in 50 ASCAT (or AMSR2) days are satellite malfunctions, giving 404 duds instead of pngs; these can be avoided based on smaller file size.
tool template:https://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-[year][dayNum].sir.gif
mail merge input values:year,dayNum
2018,27
2018,22
2018,17
2018,12
2018,07
2018,02
2017,362
2017,357
output list of urlshttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-201827.sir.gifhttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-201822.sir.gifhttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-201817.sir.gifhttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-201812.sir.gifhttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-201807.sir.gifhttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-201802.sir.gifhttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-2017362.sir.gifhttps://manati.star.nesdis.noaa.gov/ascat_images/ice_image/msfa-NHe-a-2017357.sir.gifExample 1a The netCDF bundle here from ARDEM just provides ocean depth nearing the Bering Strait. It consists of 1D latitude x and 1D longitude y files showing the ocean domain mapped plus a single Geo2D file z providing geolocated bathymetry (and land DEM). Using these xyz coordinates relative to the earth's surface and current sea level, Panoply can draw a map using colors to show depth.
This netCDF could, but does not, provide a time coordinate for past and/or future sea levels that would allow Panoply to generate an animation of inundation and recession during the late Pleistocene/Holocene. However we can use the 'mail merge' tool and auxilary data to create these.
It all begins with map experimentation. Here, that shows that while the continental shelf is quite wide in the East Siberian Sea it falls off rapidly to great depth at about 150 meters. Further, land elevations in the netCDF aren't of interest in this context, though they do indicate river drainages.
This means the default 'fit to data' range settings of -9677 and 5965 (units aren't specified) are far too broad to make good use of the color palette so they must be set. The Arctic Ocean reaches its greatest depth of 5669 m in the Fram at the Malloy Deep, the Litke trench attains 5,449 m, the average depth is ~1,000 m and 60% is less than 200 m.
A simple spreadsheet bumping depth in steps of 5 meters out to the edge of the shelf will show the flooding of Beringia since the Last Glacial Maximum. The key placeholders in the PanoplyCL script are called "scale-min" and "scale-max" but directory file paths and so forth are also important, as are useful names for the output graphic frames. (More could be done in the title and subtitle text boxes.) The placeholders are put in brackets so the mail merge tools knows what to put in where as it goes through its little csv database:
tool template:var ncdata1 = panoply.openDataset ( "
[FilePath]" );
var ncvar1 = ncdata1.getVariable ( "
[Geo2D]" );
var myplot = panoply.createPlot ( "lonlat", ncvar1 );
myplot.set ( "scale-min",
[depth] );
myplot.set ( "scale-max",
[seaLevel]);
myplot.saveImage ( "PNG", "
[pngName].png" );
mail merge input values:FilePath,Geo2D,depth,seaLevel, pngName
/Users/ARDEMv2.0.nc,z,-150,0,Beringia at -150m
/Users/ARDEMv2.0.nc,z,-145,0,Beringia at -145m
/Users/ARDEMv2.0.nc,z,-140,0,Beringia at -140m
...
/Users/ARDEMv2.0.nc,z,-0,0,Beringia -0m
output PanoplyCL scriptvar ncdata1 = panoply.openDataset ( "/Users/ARDEMv2.0.nc" );
var ncvar1 = ncdata1.getVariable ( "z" );
var myplot = panoply.createPlot ( "lonlat", ncvar1 );
myplot.set ( "scale-min", -150 );
myplot.set ( "scale-max", 0);
myplot.saveImage ( "PNG", "Beringia at -150m.png" );
var ncdata1 = panoply.openDataset ( "/Users/ARDEMv2.0.nc" );
var ncvar1 = ncdata1.getVariable ( "z" );
var myplot = panoply.createPlot ( "lonlat", ncvar1 );
myplot.set ( "scale-min", -145 );
myplot.set ( "scale-max", 0);
myplot.saveImage ( "PNG", "Beringia at -145m.png" );
var ncdata1 = panoply.openDataset ( "/Users/ARDEMv2.0.nc" );
var ncvar1 = ncdata1.getVariable ( "z" );
var myplot = panoply.createPlot ( "lonlat", ncvar1 );
myplot.set ( "scale-min", -140 );
myplot.set ( "scale-max", 0);
myplot.saveImage ( "PNG", "Beringia at -140m.png" );
etc.
Example 1b This is just a minor variation on the previous example that runs through 20,000 years of (sea level,year) pairs manually extracted from a careful paleo-reconstruction graph hosted on wikipedia. Because the rate of sea level rise accelerated during the early Holocene and then flattened out whereas the time interval between frames stays constant, a jerkier animation results than the one above.
Here the primary innovation is placing the varying data within the scale caption of Panoply. This can save quite a bit of 'post-production' custom work in Gimp because vector text and raster imagery don't size well together. Note the dates can't be presented with commas as in 20,000 because those are reserved for field separators though there's an easy workaround using two fields.
tool template:myplot.set ( "scale-label-custom", true );
myplot.set ( "scale-label-text", "[SCALE CAPTION]" );
mail merge input values:SCALE CAPTION
-126.0 m at 20 000 years BP
-123.3 m at 19 000 years BP
-120.7 m at 18 000 years BP
output PanoplyCL scriptmyplot.set ( "scale-label-custom", true );
myplot.set ( "scale-label-text", "-126.0 m at 20 000 years BP" );
myplot.set ( "scale-label-custom", true );
myplot.set ( "scale-label-text", "-123.3 m at 19 000 years BP" );
myplot.set ( "scale-label-custom", true );
myplot.set ( "scale-label-text", "-120.7 m at 18 000 years BP" );
https://forum.arctic-sea-ice.net/index.php/topic,416.msg139761.html#msg139761Example 2 The case study here refers to Uni Hamburg's more complex Arctic Ocean ice thickness netCDF bundle. These are updated daily and archived online without registration barriers. Each day provides 10 Geo2D files, notably SMOS ice thickness, bulk ice salinity, land mask, snow surface temperature, total uncertainty, etc.
Imagine now only being interested in the snow surface temperature Geo2D. The archive here dates back to 15 Oct 2010 but takes a melt season break each year between day 228 and day 301. That's 1408 netCDF bundles at 22.34 MB for 31.4 GB of download of which perhaps 8% is project relevant.
UH provides a convenient wget file for command-line mode. A subset of that, say weekly, can be made from that template following instructions in Example 0.
https://icdc.cen.uni-hamburg.de/thredds/catalog/ftpthredds/smos_sea_ice_thickness/v3/catalog.htmlThe settings below fix the map projection to stereographic centered on North Pole in standard 'Greenland down' position with the Arctic Circle as horizon. This allows your map to be rescaled to match maps you might find online for which no netCDF file is provided.
// Open a dataset.
var ncdata1 = panoply.openDataset ( "Downloads/SMOS Hamburg nc/SMOS_Icethickness_v3.1_north_20180103.nc" );
myplot.set ( "proj-name", "Stereographic" );
myplot.set ( "proj-lat0", 90.0 );
myplot.set ( "proj-lon0", -45.0 );
myplot.set ( "proj-xparam-1", 23.6 );
You can adjust these numbers in the friendly Panoply interface; the four lines below show up in the exported CL script. Two other common settings emphasize the Bering Straits (to show conditions in the Chukchi, Beaufort, and East Siberian Sea) or the Fram Strait/Svalbard/Severnaya Zemlya area.
Suppose now you viewed the -45.0 as a placeholder. Making a list that increments this by one degree 360 times, the new mail merge tool will output a concatentated script with the effect of displaying ice data on a rotating globe (restricted to the Arctic).
If the path to the file on your hard drive and the date associated with the netCDFs are also varied, the display will increment the data by a day for each increment of rotation. In other words, construct a 3 row, 365 column table in a spreadsheet, save as comma separated variables (.csv format), and paste into the tool.
var ncdata1 = panoply.openDataset ( "[PlaceholderPath] [PlaceholderDate]" );
myplot.set ( "proj-name", "Stereographic" );
myplot.set ( "proj-lat0", 90.0 );
myplot.set ( "proj-lon0", [PlaceholderLong] );
myplot.set ( "proj-xparam-1", 23.6 );
Example 3. It can become tedious testing dozens of palettes trying to find the one that best displays a particular data set. The automation option generates a slide show of several dozen potential palettes by varying an initial map. According to the PanoplyCl manual, a list of what it currently has in stock will be given with this command:
printColorTableList ( ) voidAlternatively, ImageJ will let you try palette collections: open your Panoply map in any palette, then drag any internet palette (.lut) onto the tool bar to exchange color tables, over all the frames in a stack if in montage view. That way, a new palette (to be used just once maybe) need not be added to the Panoply set which is already cluttered.
https://www.giss.nasa.gov/tools/panoply/colorbars/http://cxc.harvard.edu/ciao/threads/auxlut/ allows quick trials of 196 palettes
http://soliton.vm.bytemark.co.uk/pub/cpt-city/tool template:myplot.set ( "scale-colorbar", "[palette].[colorTable]" );
myplot.set ( "scale-width", 40 );
myplot.set ( "scale-reverse", true );
myplot.set ( "scale-outlier-gap", "THIN" );
myplot.set ( "scale-label-location", "ABOVE" );
mail merge input values:palette,colorTable
CB_RdYlGn,cpt
NEO_modis_sst_45,act
NEO_ceres_insol,lut
NEO_trmm_rainfall,rgb
bathymetry,gct
output PanoplyCL scriptmyplot.set ( "scale-colorbar", "CB_RdYlGn.cpt" );
myplot.set ( "scale-width", 40 );
myplot.set ( "scale-reverse", true );
myplot.set ( "scale-outlier-gap", "THIN" );
myplot.set ( "scale-label-location", "ABOVE" );
myplot.set ( "scale-colorbar", "NEO_modis_sst_45.act" );
myplot.set ( "scale-width", 40 );
myplot.set ( "scale-reverse", true );
myplot.set ( "scale-outlier-gap", "THIN" );
myplot.set ( "scale-label-location", "ABOVE" );
myplot.set ( "scale-colorbar", "NEO_ceres_insol.lut" );
myplot.set ( "scale-width", 40 );
myplot.set ( "scale-reverse", true );
myplot.set ( "scale-outlier-gap", "THIN" );
myplot.set ( "scale-label-location", "ABOVE" );
myplot.set ( "scale-colorbar", "NEO_trmm_rainfall.rgb" );
myplot.set ( "scale-width", 40 );
myplot.set ( "scale-reverse", true );
myplot.set ( "scale-outlier-gap", "THIN" );
myplot.set ( "scale-label-location", "ABOVE" );
myplot.set ( "scale-colorbar", "bathymetry.gct" );
myplot.set ( "scale-width", 40 );
myplot.set ( "scale-reverse", true );
myplot.set ( "scale-outlier-gap", "THIN" );
myplot.set ( "scale-label-location", "ABOVE" );
Example 4 It is not so easy to harvest and re-position text from Panoply's 6 text input boxes once that text has been rendered onto the map background. However it is straightforward to consolidate and reposition text over unused portions of the map.
This is key, in conjunction with plot size 190, in arriving at a final product that fits within the forum's 700x700 pixel size constraints yet retains maximal map resolution. (In other words, the color palette and text lines ordinarily take space away from what's available to the map.)
The trick is to make a one-row montage of the frames, duplicate it, select a line of text across the entire montage, delete its complement to transparency, position it as a new layer over the map montage, then set its Gimp mode to 'darken only'. Since nothing can get darker than black text, it will pasted through; since nothing is less white than pure white, the white will get lost out to whatever lies underneath on the map.
Repeat the design composition with the other lines of text (and the scale color legend). Some of these lines may benefit from vertical resizing (ie, uncoupled from horizonal). Then, rather than flatten or merge, make 'new layer from visible'. This top layer can then be re-sliced in Gimp (100 slices max) or ImageJ back into animation frames.
The animation shows the gain: the map circle is originally 686x686 pixels but reducing the initial Panoply map to forum maximum reduces the 'content circle' to 520x520. This means
only 57% as many pixels remain available to display data.
For a given PanoplyCL 'plot size', the text layers will be in predictable positions. Consequently the process can be automated.
... to be revised/continued.