Support the Arctic Sea Ice Forum and Blog

Author Topic: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima  (Read 551 times)

SimonF92

  • Grease ice
  • Posts: 523
    • View Profile
  • Liked: 200
  • Likes Given: 86
Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« on: November 17, 2021, 12:48:54 PM »
Hi everyone

I made a simple machine learning tutorial.

I know some of you enjoy dipping into code, this is the kind of thing which can be fun to play around with and actually has a reasonable learning curve

Any questions, let me know

https://github.com/SimonF92/Arctic/blob/master/Machine_learning_rainfall_temp_minima.ipynb
« Last Edit: November 17, 2021, 12:55:19 PM by SimonF92 »
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

sidd

  • First-year ice
  • Posts: 6094
    • View Profile
  • Liked: 929
  • Likes Given: 0
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #1 on: November 17, 2021, 03:13:36 PM »
Thanks. I played around with overall distributions of precipitations to catch the move to greater precip intensity, but the models i tried didnt do much better than naive Bayes ...

sidd

SimonF92

  • Grease ice
  • Posts: 523
    • View Profile
  • Liked: 200
  • Likes Given: 86
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #2 on: November 17, 2021, 04:37:21 PM »
Thanks. I played around with overall distributions of precipitations to catch the move to greater precip intensity, but the models i tried didnt do much better than naive Bayes ...

sidd

Thanks for the feedback sidd, nice to hear you tried some other models, sounds like you know what you are doing regarding this so I am interested in your feedback.

There are definitely more relevant and better shaped data than what I used, but it seemed an interesting dataset. Sometime down the line I will probably make a tutorial for predicting minima as a continuous variable
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

Jim Hunt

  • First-year ice
  • Posts: 5494
  • Stay Home, Save Lives
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 724
  • Likes Given: 59
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #3 on: November 17, 2021, 05:20:02 PM »
What with one thing and another I haven't had a chance to check out your github yet, but thanks for what looks as though it will be a very interesting project.

What ML platform is it based on? Can it be deployed on a Raspberry Pi? Can it even be trained on a Raspberry Pi?
Reality is merely an illusion, albeit a very persistent one - Albert Einstein

SimonF92

  • Grease ice
  • Posts: 523
    • View Profile
  • Liked: 200
  • Likes Given: 86
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #4 on: November 17, 2021, 11:33:13 PM »
Thanks Jim, id imagine keeping twitterdom protected from endemic climate ignorance is a full time job.

SVM is a very simple (by ML standards) algorithm. Its not computationally expensive. To my knowledge , its nowhere near as intensive as a fully fledged neural network. Plus the input features are actually quite simple.

The SVM library i use is from sklearn https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html

In short, no, a pi should not struggle with this at all. Personally I think you could give it 100x more features and it would be fine

Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

Jim Hunt

  • First-year ice
  • Posts: 5494
  • Stay Home, Save Lives
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 724
  • Likes Given: 59
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #5 on: November 18, 2021, 08:34:09 AM »
I'd imagine keeping twitterdom protected from endemic climate ignorance is a full time job.

Especially when @Jack appears to keep sticking a spanner in our works!

Support Vector Machines. A curious name I've always thought.

I was indeed thinking of something more neural nettish. For example we had a quick look at TinyML for an "Edge AI" project that didn't work out:

https://www.ft.com/content/9446192a-aff1-4e95-93fb-a5adfbc7bbd5
Reality is merely an illusion, albeit a very persistent one - Albert Einstein

SimonF92

  • Grease ice
  • Posts: 523
    • View Profile
  • Liked: 200
  • Likes Given: 86
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #6 on: November 18, 2021, 06:35:53 PM »
I'd imagine keeping twitterdom protected from endemic climate ignorance is a full time job.

Especially when @Jack appears to keep sticking a spanner in our works!

Support Vector Machines. A curious name I've always thought.

I was indeed thinking of something more neural nettish. For example we had a quick look at TinyML for an "Edge AI" project that didn't work out:

https://www.ft.com/content/9446192a-aff1-4e95-93fb-a5adfbc7bbd5

If youre interested, these are the different kinds of models which can be applied. If I was to go for any next it would be a CNN, they are good for time-series forecasting. FYI, the classification score of each model is on each of the subplots (not my own work)

I feel sorry for software developers who work for the government, I bet they and their teams are full of talented individuals- yet the projects always fall short of their intentions. They were using excel to curate databases of covid cases until they 'lost' 60,000 cases due to an excel memory overflow. There are some serious issues if nobody realised excel was a bad idea
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

Jim Hunt

  • First-year ice
  • Posts: 5494
  • Stay Home, Save Lives
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 724
  • Likes Given: 59
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #7 on: November 19, 2021, 01:39:17 PM »
In an unexpected development John Cook of Skeptical Science and 97% consensus fame has just published a paper that includes SVM amongst many more ML algos:

https://twitter.com/GreatWhiteCon/status/1461330238311444481

"Mirjam Nanko & myself trained a machine learning model to automatically detect climate misinformation"

 
Reality is merely an illusion, albeit a very persistent one - Albert Einstein

SimonF92

  • Grease ice
  • Posts: 523
    • View Profile
  • Liked: 200
  • Likes Given: 86
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #8 on: November 19, 2021, 05:37:22 PM »
I wonder if they trained it using the Daily Mail comment section?
Bunch of small python Arctic Apps:
https://github.com/SimonF92/Arctic

Jim Hunt

  • First-year ice
  • Posts: 5494
  • Stay Home, Save Lives
    • View Profile
    • The Arctic sea ice Great White Con
  • Liked: 724
  • Likes Given: 59
Re: Machine Learning Tutorial- UK Weather and Arctic Sea Ice Minima
« Reply #9 on: November 20, 2021, 12:22:56 AM »
Quote
A first draft of the contrarian claims taxonomy was developed based on the list of climate myths at skepticalscience.com.

The analysis data is available at https://socialanalytics.ex.ac.uk/cards/data.zip

The classifiers are available at https://socialanalytics.ex.ac.uk/cards/models.zip

The analysis code is available at https://github.com/traviscoan/cards

Sadly there's no mention of the Daily Fail that I have spotted yet.
Reality is merely an illusion, albeit a very persistent one - Albert Einstein