[This is a repost of my original entry in the "Home brew AMSR2 extent & area calculation" thread]
Corrections for timecodes in cryosphere dataI have been using the cryosphere data (
http://arctic.atmos.uiuc.edu/cryosphere/timeseries.anom.1979-2008) for a
couple of years for fun and no profit (as well as IJIS, PIOMAS and NSIDC).
At the time I set up an automated import into a database (PostgreSQL), and noted that the timecodes were not unique. I did not bother to look further then but just turned of the indexing on the timecode.
I have recently had some time on my hands and decided to have a closer look and after a few hours work found several inconsistencies and probable typos.
There are 366 data points for all years divisible by 4 and 365 for all others (except 1979 only had 364 values, 1979.0000 is missing). Note that 2000 is therefore considered a leap year. I use the term Y4 instead of leap year as a reminder that it is not strictly leap years.
Focusing first on the non-Y4 years I found the following inconsistencies (timecodes that are not
round(d*365/10000) where d is the day of the year), some which appear to be obvious typos:
- 1987.9253 -> 1987.9233
- 2005.1088 -> 2005.1096
- 2005.8650 -> 2005.8658
- 2010.0928 -> 2010.0932
- 2009.4274 -> 2009.4247 (duplicate code, only correct the first occurrence)
In addition, the block of values 2007.9315 - 2007.9808 (19 values) have been shifted 1 step (~0.0027) since the code 2007.9288 is missing and 2007.9808 is a duplicate.
The following will fix this:
- for codes 2007.9342 - 2007.9780, set the code to the code on the previous line
- change 2007.9315 -> 2007.9288 (missing time code)
- change 2007.9808 -> 2007.9780 (first occurrence, duplicate entry).
This will make all non-Y4 years compact (365 values) and monotonically increasing by 1 day (~0.00274).
For the Y4 years there is more to do, I believe that approximately 3/4 of the year is one day off but this requires some more explanation, addressed in following posts.