Support the Arctic Sea Ice Forum and Blog

Author Topic: Test space  (Read 103591 times)

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #300 on: August 31, 2020, 11:26:18 PM »
amsr2-uhh jun1-aug30 no dates crf29
removed
« Last Edit: September 02, 2020, 01:43:01 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #301 on: September 17, 2020, 12:28:09 AM »
tried to do difference on seaice.de post here
Strange that the two years were different pixel sizes. A very rough resize produced this, so I think I've done something wrong. Interesting though.
removed
« Last Edit: September 17, 2020, 01:10:42 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #302 on: September 17, 2020, 12:42:43 AM »
grain extract was better. Might try more accurate scaling tomorrow.
posted elsewhere
« Last Edit: September 17, 2020, 01:11:02 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #303 on: September 24, 2020, 10:10:09 PM »
155 arctic buoys, mar-sep
« Last Edit: September 26, 2020, 03:08:15 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #304 on: October 02, 2020, 01:07:03 AM »
finally starting to get my head around macid's code for this
will probably have to be mp4. Maybe possible to label the years. Cairo might fix the colours or work out how to make them discrete.

edit: whoi itp7,8 and 9 produced over 9million variable and took 6hrs to load so looking at loading them into MySQL and querying from there. I wonder what computer macid was using.
« Last Edit: October 03, 2020, 01:56:34 PM by uniquorn »

yu

  • New ice
  • Posts: 4
    • View Profile
  • Liked: 4
  • Likes Given: 0
Re: Test space
« Reply #305 on: October 03, 2020, 12:29:23 PM »
VISHOP file

oren

  • First-year ice
  • Posts: 9805
    • View Profile
  • Liked: 3584
  • Likes Given: 3922
Re: Test space
« Reply #306 on: October 04, 2020, 08:15:58 AM »
Interesting visualization.

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #307 on: October 05, 2020, 09:49:40 PM »
buoys, april-oct. Ignore the deployment straight lines.
« Last Edit: October 05, 2020, 10:10:58 PM by uniquorn »

SimonF92

  • Grease ice
  • Posts: 592
    • View Profile
  • Liked: 210
  • Likes Given: 87
Re: Test space
« Reply #308 on: October 06, 2020, 04:30:55 PM »
buoys, april-oct. Ignore the deployment straight lines.

 :o thats amazing!
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #309 on: October 16, 2020, 03:41:39 PM »
wake timing a bit quick -removed
« Last Edit: October 18, 2020, 12:53:50 AM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #310 on: October 20, 2020, 10:59:42 PM »
IBCAO_v4_200m-sliced

sidd

  • First-year ice
  • Posts: 6774
    • View Profile
  • Liked: 1047
  • Likes Given: 0
Re: Test space
« Reply #311 on: October 21, 2020, 06:10:01 AM »
very nice.

sidd

yu

  • New ice
  • Posts: 4
    • View Profile
  • Liked: 4
  • Likes Given: 0
Re: Test space
« Reply #312 on: October 22, 2020, 06:02:10 AM »
Hello,

How do i load a pic. that will look good but will be small in size?
Its currently 8 Mb of jpg copied from excel as picture.
Thank you

« Last Edit: October 22, 2020, 06:09:08 AM by yu »

oren

  • First-year ice
  • Posts: 9805
    • View Profile
  • Liked: 3584
  • Likes Given: 3922
Re: Test space
« Reply #313 on: October 22, 2020, 07:24:41 AM »
It looks great! I can't think how you can reduce it much and not lose the level of detail of the second zoom.
However, I don't think size is that much of an issue, as users can decide whether to click it or not. Just warn it's a large image.

John_The_Elder

  • New ice
  • Posts: 58
    • View Profile
  • Liked: 26
  • Likes Given: 8
Re: Test space
« Reply #314 on: October 22, 2020, 04:20:04 PM »
I like the layout, very visual.
Perhaps a legend of years and days for new users would help.
John
John

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #315 on: October 26, 2020, 01:40:43 PM »
a quick look at ascat, some missing days, best viewed slow --
« Last Edit: October 26, 2020, 09:31:00 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #316 on: October 26, 2020, 09:34:16 PM »
with v103 overlay at 60%. Interesting effect with amsr2 moving over the stationary repeated missing frames.
« Last Edit: October 26, 2020, 09:39:44 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #317 on: October 29, 2020, 11:33:00 PM »
not sure about this one. grain extract on sstfnd here, 1 day diff. clearly would be better without the contours.

A-Team

  • Young ice
  • Posts: 2977
    • View Profile
  • Liked: 944
  • Likes Given: 35
Re: Test space
« Reply #318 on: October 30, 2020, 02:25:09 PM »
Grain merge is going to leave a lot of 128 where the difference is 0 so it helps to use the Levels tool (on image and palette) to brighten the colors. SSTfnd will primarily address the Siberian side so there is some resolution gain (for a given size) to cropping to the ROI. Contours here make it rather busy so the number of contour levels could be lowered (or dispensed with altogether).

Note the cryo2smos file should be perfectly complimentary for a given date, meaning the two can be combined in a single image but with a double palette (labels top and bottom or completely distinct). It also provides an ice mask to go with the land mask. Selecting and re-coloring this allows them to be distinguished from the open water 128 which might be left alone or made a light white.
« Last Edit: October 30, 2020, 03:04:09 PM by A-Team »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #319 on: November 20, 2020, 12:44:39 PM »
comparison of 2013 and 2020 melting season in the beaufort using amsr2-uhh
duplicated some missing or poor quality days. It's possible to use higher res images (larger file sizes)
« Last Edit: November 20, 2020, 12:54:28 PM by uniquorn »

BornFromTheVoid

  • Nilas ice
  • Posts: 1339
    • View Profile
  • Liked: 679
  • Likes Given: 299
Re: Test space
« Reply #320 on: November 27, 2020, 10:15:43 PM »
I've been working with the Central England Temperature (CET) series a bit lately, and one thing I've worked out is the most recent warm and cold record for each day of the year. I'm toying around with different visualistion methods.
The one below has the daily records with diamond symbols and the monthly averages with solid lines. The 365 days of the year follow the 360 degrees of the circle. The bands are at 50 day intervals, so the closer to the centre, the more recent the record was set.
However, I still think it's a bit messy, so any suggestion on altering it or whatever else is quite welcome!
I recently joined the twitter thing, where I post more analysis, pics and animations: @Icy_Samuel

kassy

  • Moderator
  • First-year ice
  • Posts: 8234
    • View Profile
  • Liked: 2041
  • Likes Given: 1986
Re: Test space
« Reply #321 on: November 27, 2020, 11:20:59 PM »
For the sort of data that is about as clear as you can get it.

The records also follow an absolute trend so there should be a way to colour code that?
Þetta minnismerki er til vitnis um að við vitum hvað er að gerast og hvað þarf að gera. Aðeins þú veist hvort við gerðum eitthvað.

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #322 on: November 29, 2020, 12:53:02 PM »
Playing with cmems lobelia (My Ocean)  https://tinyurl.com/yyhff8p7
Here using gimp grain extract on one day difference.

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #323 on: November 29, 2020, 02:18:41 PM »
added clahe 2.1 to bring out the thin ice movement in the laptev. file size is a bit bigger

oren

  • First-year ice
  • Posts: 9805
    • View Profile
  • Liked: 3584
  • Likes Given: 3922
Re: Test space
« Reply #324 on: November 30, 2020, 03:26:55 AM »
Great animation.

SimonF92

  • Grease ice
  • Posts: 592
    • View Profile
  • Liked: 210
  • Likes Given: 87
Re: Test space
« Reply #325 on: November 30, 2020, 11:01:21 AM »
added clahe 2.1 to bring out the thin ice movement in the laptev. file size is a bit bigger

I have been wanting to track individual floes (average area and velocity) in the CAB for a while. I got it working with something for my phd (cells) and I know it could probably work on ice floes because effectively they are both moving objects on a plane.

I have been looking for good images of floes for a while and these are by far the best I have seen uniquorn. Is the generation of these images automatable?
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #326 on: November 30, 2020, 09:14:16 PM »
Quote
track individual floes
In this case I think you would be tracking modelled floes.
Quote
Abstract
The Arctic Sea Ice Analysis and Forecast system uses the neXtSIM stand-alone sea ice model running the Maxwell-Elasto-Brittle sea ice rheology on an adaptive triangular mesh of 10 km average cell length. The model domain covers the whole Arctic domain, excluding the Canadian Archipelago, the Baffin and Hudson Bays. neXtSIM is forced with surface atmosphere forcings from the ECMWF (European Centre for Medium-Range Weather Forecasts) and ocean forcings from TOPAZ4, the ARC MFC PHY NRT system (002_001a). neXtSIM runs daily, assimilating OSI-SAF sea ice concentrations (both SSMI and AMSR2) from the SI TAC and providing 7-day forecasts. The output variables are the ice concentrations, ice thickness, ice drift velocity and snow depths, provided at hourly frequency. The adaptive Lagrangian mesh is interpolated for convenience on a 3 km resolution regular grid in a Polar Stereographic projection. The projection is identical to other ARC MFC products.
The movement is interesting though and as it's based on osi-saf it should be reasonably accurate. Available hourly :)

SimonF92

  • Grease ice
  • Posts: 592
    • View Profile
  • Liked: 210
  • Likes Given: 87
Re: Test space
« Reply #327 on: November 30, 2020, 10:15:17 PM »
Thanks for providing the paper. I think you are correct, pretty complicated stuff to get your head around though
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #328 on: December 01, 2020, 05:59:52 PM »
iabp drift oct26-nov30
stop start in the mclure strait.

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #329 on: December 11, 2020, 11:04:46 PM »
cmems vs buoy test, dec1-11. ice motion not too bad.
should have gone with bigger buoys, shame about the cmems glitch dec8, dates not quite synced
« Last Edit: December 11, 2020, 11:43:43 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #330 on: January 18, 2021, 10:11:57 PM »
ascat large 2020334-2021017
a bit too large - removed
« Last Edit: January 20, 2021, 12:26:48 AM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #331 on: January 30, 2021, 12:33:47 AM »
anyone know how to fix image scaling in ffmpeg when using overlay. The buoys are gif->mp4, nullschool wind is avi->mp4. Both are 452x458pixels then I use

ffmpeg -i wind.mp4 -i buoys.mp4 -filter_complex "blend=c0_mode='overlay'"

Nullschool starts at the right size but slowly enlarges.

nullschool wind over buoy drift 2016-2021
crf 37 is way too high but it's a big vid

edit: just figured it out. It's the borders and title on the buoys...that was wrong too. It was an auto increment on the nullschool url
removed
« Last Edit: January 30, 2021, 02:03:57 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #332 on: February 06, 2021, 12:53:59 AM »
testing 3hourly presentation of iabp raw data, 2019-2021  6.7MB mp4
updated below
.
« Last Edit: February 07, 2021, 01:52:09 PM by uniquorn »

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #333 on: February 07, 2021, 01:56:20 PM »
testing 3hourly iabp buoy drift 2018-2021
How to get rid of the permanent 'dead buoys'?
172MB gif to 7.8MB mp4 is not bad
updated below
« Last Edit: February 08, 2021, 04:38:52 PM by uniquorn »

kassy

  • Moderator
  • First-year ice
  • Posts: 8234
    • View Profile
  • Liked: 2041
  • Likes Given: 1986
Re: Test space
« Reply #334 on: February 07, 2021, 10:52:12 PM »
So these are the points left after the trails disappear?
Do they show up as new to draw points in every step and if so why? Or why do they not fade out as they do when moving?

Assuming there is a routine doing the fading when they move it should work to change something there. On the map they look like stationary dots reporting a velocity which must be wrong (or in a vertical plane).

Not sure how you do the fade but what is missing is a way for the dot to see it is in the same place. If that is complicated maybe look at the input data and figure out where there is no movement comparing time steps and then override the velocity vector?
Þetta minnismerki er til vitnis um að við vitum hvað er að gerast og hvað þarf að gera. Aðeins þú veist hvort við gerðum eitthvað.

oren

  • First-year ice
  • Posts: 9805
    • View Profile
  • Liked: 3584
  • Likes Given: 3922
Re: Test space
« Reply #335 on: February 08, 2021, 07:31:29 AM »
Before generating the animation, run a cleanup routine that deletes rows with identical data (except for the date/timestamp of course). To avoid occasional deletions of randomly identical rows, demand at least 2 (or even 3) identical preceding rows in order to delete a given row.
If 2 is identical to 1, do nothing. If 3 is identical to both 2 and 1, delete 3. Repeat.

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #336 on: February 08, 2021, 11:48:07 AM »
Thanks. I'll see which of those options is easiest to code. I might take the velocity colours out altogether as they look pretty bugged.

SimonF92

  • Grease ice
  • Posts: 592
    • View Profile
  • Liked: 210
  • Likes Given: 87
Re: Test space
« Reply #337 on: February 08, 2021, 02:52:49 PM »
Thanks. I'll see which of those options is easiest to code. I might take the velocity colours out altogether as they look pretty bugged.

Add an if statement such that if velocity is greater than a reasonable value, those rows get deleted, this might help to stop them firing around the place
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #338 on: February 08, 2021, 04:38:17 PM »
<add an if statement>
At the moment I delete entries >0.99m/s but I think it's interesting to see the deployment, also some resupply vessels to mosaic and other expeditions. I was thinking the opposite so that all deployment journeys were visible but more accurate, rather than taking short cuts across land ;) . We'll see what's cleanest. I think the problems might be in the way I'm loading the data using rbind.fill. Plus I didn't really check much when I started using the daily full resolution data. I've been lazy just running the existing code in the background.
Busy looking at ess leads today.

here's 2017-2021, crf31 compression is getting higher to reduce 266MB down to 11MB, 3hrly so I can overlay nullschool at some point. Daily didn't really work for me.
2014-2021 is going to be a big file though...

SimonF92

  • Grease ice
  • Posts: 592
    • View Profile
  • Liked: 210
  • Likes Given: 87
Re: Test space
« Reply #339 on: February 08, 2021, 05:04:29 PM »
Really nice, genuinely destroys any other visual material I have seen regarding iabp on academic/industry web pages.

I need to stop sitting on my hands and finish the work I was doing on them. Problem with projects is they never quite feel finished :)
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

oren

  • First-year ice
  • Posts: 9805
    • View Profile
  • Liked: 3584
  • Likes Given: 3922
Re: Test space
« Reply #340 on: February 08, 2021, 05:16:51 PM »
Quote
Problem with projects is they never quite feel finished
For me they're usually not even begun...

SimonF92

  • Grease ice
  • Posts: 592
    • View Profile
  • Liked: 210
  • Likes Given: 87
Re: Test space
« Reply #341 on: February 09, 2021, 01:22:08 PM »
Quote
Problem with projects is they never quite feel finished
For me they're usually not even begun...

Wouldnt normally comment just to say lol, but that did get me
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

yu

  • New ice
  • Posts: 4
    • View Profile
  • Liked: 4
  • Likes Given: 0
Re: Test space
« Reply #342 on: February 20, 2021, 05:08:48 PM »
North Greenland

oren

  • First-year ice
  • Posts: 9805
    • View Profile
  • Liked: 3584
  • Likes Given: 3922
Re: Test space
« Reply #343 on: February 22, 2021, 01:22:55 PM »
Nice animation. I suggest to repost in the freezing season thread.

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #344 on: March 07, 2021, 04:35:10 PM »
cs2smos-piomas-ascat-jan-feb
best viewed full screen

kassy

  • Moderator
  • First-year ice
  • Posts: 8234
    • View Profile
  • Liked: 2041
  • Likes Given: 1986
Re: Test space
« Reply #345 on: March 07, 2021, 04:40:37 PM »
Nice having them all side by side. Looks like an interesting season ahead...
Þetta minnismerki er til vitnis um að við vitum hvað er að gerast og hvað þarf að gera. Aðeins þú veist hvort við gerðum eitthvað.

Jim Hunt

  • First-year ice
  • Posts: 6268
  • Don't Vote NatC or PopCon, Save Lives!
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 893
  • Likes Given: 87
Re: Test space
« Reply #346 on: March 08, 2021, 09:58:10 AM »
cs2smos-piomas-ascat-jan-feb

Very handy uniquorn. How do you go about enhancing the ASCAT imagery?
"The most revolutionary thing one can do always is to proclaim loudly what is happening" - Rosa Luxemburg

uniquorn

  • First-year ice
  • Posts: 5117
    • View Profile
  • Liked: 2163
  • Likes Given: 388
Re: Test space
« Reply #347 on: March 08, 2021, 11:27:53 AM »
A-Team does a better job but here is my method:

1. Download, install and open Fiji (ImageJ with plug-ins)
2. Drag the ascat files into the Fiji console. (start with a small selection, say, 5 days)
3. From the Image menu select Stacks->Images to Stack
4. Click the rectangle or ellipse button, select area of interest then from the Image menu select Crop.
5. From the Image menu select Adjust->Brightness/Contrast. Scroll through the images to view their histograms.
6. In the B&C console I increase the minimum to 41, in winter this doesn't lose too much information. Apply to all slices.
7. From the Image menu select Stack->Label, click the Preview checkbox and play with the numbers.
8. From the Image menu select Stacks->Make Montage. Set Scale Factor to 1 and click OK
9. From the Process menu select Enhance Local Contrast (CLAHE). I use 127,256,5.3 (For me this is usually just below 'too blocky' but A-Team prefers lower. Play with the numbers.
10. From the Image menu select Stacks->Tools->Montage to Stack and click OK.
11. From the Image menu select Stacks->Animation->Animation Options and choose a speed.
12. From the File menu select Save As->Gif

ImageJ has very few 'undo' options and it's easy to lose a lot of changes. Image->Duplicate is your friend. :)

Jim Hunt

  • First-year ice
  • Posts: 6268
  • Don't Vote NatC or PopCon, Save Lives!
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 893
  • Likes Given: 87
Re: Test space
« Reply #348 on: March 08, 2021, 12:39:41 PM »
Download, install and open Fiji (ImageJ with plug-ins)

Thanks! I have never previously experimented with ImageJ.

Now seems like a good time to start  :)
"The most revolutionary thing one can do always is to proclaim loudly what is happening" - Rosa Luxemburg

Jim Hunt

  • First-year ice
  • Posts: 6268
  • Don't Vote NatC or PopCon, Save Lives!
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 893
  • Likes Given: 87
Re: Test space
« Reply #349 on: March 08, 2021, 03:50:30 PM »
Here is my method:

Here is my first experiment.

I moved applying the labels to near the end, otherwise they turned out blue.

Assorted other areas of the images have turned blue after CLAHE too. Where am I going wrong?
"The most revolutionary thing one can do always is to proclaim loudly what is happening" - Rosa Luxemburg