Support the Arctic Sea Ice Forum and Blog

Poll

How confident are you that there will be a BOE before October 15th 2022?

100% (score +10 if BOE, -30 if not)
0 (0%)
78% (+8 if BOE, -14 if not)
0 (0%)
61% (+4 if BOE, -5 if not)
0 (0%)
55% (+2 if BOE, -2 if not)
0 (0%)
45% (-2 if BOE, +2 if not)
0 (0%)
39% (-5 if BOE, +4 if not)
0 (0%)
22% (-14 if BOE, +8 if not)
5 (19.2%)
0% (-30 if BOE, +10 if not)
21 (80.8%)
Other (post in thread)
0 (0%)

Total Members Voted: 26

Voting closed: January 31, 2022, 03:50:29 PM

Author Topic: 2022 Prediction Challenge: January  (Read 1713 times)

Richard Rathbone

  • Nilas ice
  • Posts: 1418
    • View Profile
  • Liked: 300
  • Likes Given: 18
2022 Prediction Challenge: January
« on: January 01, 2022, 03:50:29 PM »
This is a variant on my prediction challenges from previous years. As before its as much about calibrating how confident you are correctly, as about being accurate.

Scoring is based on Brier Scores, normalised to give 10 for being certain and right and 0 for no opinion either way in order to match my previous challenges and allow running totals to be carried over from that format by those that wish to do so. https://en.wikipedia.org/wiki/Brier_score

If the options available in the poll do not suit, the Brier Score can be calculated for any level of confidence, or degree of risk. Post your level of confidence (or the points you wish to win if correct) in the thread.

For the purpose of this thread, a BOE occurs when the 5-day rolling average NSIDC extent is below 1M.

If there's sufficient demand, I'll do one of these each month.  Feel free to make suggestions for future predictions or adjustments to the format. I'm not expecting a huge variety of response this time, but BOE seems topical at the moment. I'd intend to pitch them at a level to spread the responses out more in future.

kassy

  • First-year ice
  • Posts: 5625
    • View Profile
  • Liked: 1835
  • Likes Given: 1827
Re: 2022 Prediction Challenge: January
« Reply #1 on: January 01, 2022, 04:24:31 PM »
Predicting the outcome of a season in january is a pretty hard/impossible task. Maybe slice them up as winter predictions for Jan-Mar, spring to Apr-May-Jun. Etc.

Þetta minnismerki er til vitnis um að við vitum hvað er að gerast og hvað þarf að gera. Aðeins þú veist hvort við gerðum eitthvað.

The Walrus

  • Nilas ice
  • Posts: 1959
    • View Profile
  • Liked: 114
  • Likes Given: 389
Re: 2022 Prediction Challenge: January
« Reply #2 on: January 01, 2022, 04:59:18 PM »
Not a chance.  Even if the worst possible factors align, we might break the 2012 low, but I cannot foresee that happening this year.  Hence the possibility of a BOE this year is almost nil,

oren

  • Moderator
  • First-year ice
  • Posts: 8443
    • View Profile
  • Liked: 3264
  • Likes Given: 3354
Re: 2022 Prediction Challenge: January
« Reply #3 on: January 01, 2022, 05:49:35 PM »
I think BOE chances for a given year are currently at 5-10%, and slowly growing. As for the poll, I think it would be better to add more low confidence categories, and modify the scoring system to give better odds for choosing these categories.
I am guessing no one thinks a BOE is 100% in the cards. The real range is probably between 0% and 20% or 25% at most, even for die hards.
As the poll is set up it pays off to vote 0%.

crandles

  • Young ice
  • Posts: 3183
    • View Profile
  • Liked: 223
  • Likes Given: 70
Re: 2022 Prediction Challenge: January
« Reply #4 on: January 01, 2022, 06:01:07 PM »
Almost nil: 0.1% to 1% maybe.

So I'll go for 0% option.

Richard Rathbone

  • Nilas ice
  • Posts: 1418
    • View Profile
  • Liked: 300
  • Likes Given: 18
Re: 2022 Prediction Challenge: January
« Reply #5 on: January 01, 2022, 08:18:40 PM »
I think BOE chances for a given year are currently at 5-10%, and slowly growing. As for the poll, I think it would be better to add more low confidence categories, and modify the scoring system to give better odds for choosing these categories.
I am guessing no one thinks a BOE is 100% in the cards. The real range is probably between 0% and 20% or 25% at most, even for die hards.
As the poll is set up it pays off to vote 0%.
You can vote other and then specify 10% or 5% or whatever in a post.

The payoffs are
9.6:-22.4 for 10%
9.9:-26.1 for 5%
9.996:-29.2 for 1%

The Brier Score gives you the best odds when you have accurately calibrated the probability. If its actually 10%, then choosing 10% gives the best result in the long run. In the short run it pays to be overconfident and lucky, but in the long run those 10% chances happen.

I reckon its 0. I wouldn't even give it 0.1% but I also want a neutral structure on the challenge rather than one which incorporates my opinion of what the likelihood is.


crandles

  • Young ice
  • Posts: 3183
    • View Profile
  • Liked: 223
  • Likes Given: 70
Re: 2022 Prediction Challenge: January
« Reply #6 on: January 01, 2022, 09:50:36 PM »

I reckon its 0. I wouldn't even give it 0.1% but I also want a neutral structure on the challenge rather than one which incorporates my opinion of what the likelihood is.

At 1 in a 1000 you can dream up some remarkable weather patterns, or meteorite strikes or ...
I reckon it is hard to eliminate that last 0.1% chance.

interstitial

  • Nilas ice
  • Posts: 1932
    • View Profile
  • Liked: 453
  • Likes Given: 91
Re: 2022 Prediction Challenge: January
« Reply #7 on: January 01, 2022, 10:04:24 PM »
This is a poorly designed poll. I do not like it.

Brigantine

  • Frazil ice
  • Posts: 316
    • View Profile
  • Liked: 99
  • Likes Given: 272
Re: 2022 Prediction Challenge: January
« Reply #8 on: January 03, 2022, 08:24:42 PM »
Nil.
I actually rate it about 0.1%, but I'm not worried about the last 0.08 points on the off chance I'm wrong.

Overall I'm a fan of these prediction challenges, and I like that the Brier Scores always marginally rewarding a more accurately calibrated prediction... but it does feel like it places such a low weight on not under-estimating low-probability-high-impact outcomes.

For comparison, Your Brier Score vs my go-to logarithmic one, say 10+10*ln(p)/ln(N)
100% ... +10 / -30 ... +10 / -inf
45% ... -2 / +2 ... -1.5 / +1.4
39% ... -5 / +4 ... -3.6 / +2.9
22% ... -14 / +8 ... -11.8 / +6.4
10% ... -22.4 / 9.6 ... -23.2 / +8.5
5% ... -26.1 / +9.9 ... -33.2 / +9.3
1% ... -29.2 / +9.996 ... -56.4 / +9.86
0.1% ... -29.92 / +9.99996 ... -89.7 / +9.986
0% ... -30 / +10 ... -inf / +10

It's just one small detail, but it would make the whole poll more relevant for many people weighing up 1% vs 5%.

ChrisReynolds

  • Nilas ice
  • Posts: 1764
    • View Profile
    • Dosbat
  • Liked: 20
  • Likes Given: 9
Re: 2022 Prediction Challenge: January
« Reply #9 on: January 13, 2022, 09:18:17 PM »
Considering PIOMAS sea ice volume (just downloaded and reviewed) as a proxy for energy budgets. Considering the NSIDC 5 day extent (just downloaded and reviewed).

There is absolutely no clucking chance of a 5 day BOE as defined this year (excluding an exogenous force majeure, e.g. direct hit of >1km asteroid on the Arctic Ocean in May or later).
There just isn't the spare energy.

Wow, NSIDC give regions...  ;D
But not PIOMAS.  :(

oren

  • Moderator
  • First-year ice
  • Posts: 8443
    • View Profile
  • Liked: 3264
  • Likes Given: 3354
Re: 2022 Prediction Challenge: January
« Reply #10 on: January 14, 2022, 01:35:39 AM »
We used to get regional daily PIOMAS data from Wipneus two times a month, he used to crunch the raw PIOMAS output files to generate the regional file. However his site ArctischePinguin was somehow screwed up by google a few months ago, and no more data is available at this time.
There are some regional charts which have been posted in the PIOMAS thread, if you scroll back enough.

ChrisReynolds

  • Nilas ice
  • Posts: 1764
    • View Profile
    • Dosbat
  • Liked: 20
  • Likes Given: 9
Re: 2022 Prediction Challenge: January
« Reply #11 on: January 21, 2022, 09:31:38 PM »
Thanks Oren.

Richard Rathbone

  • Nilas ice
  • Posts: 1418
    • View Profile
  • Liked: 300
  • Likes Given: 18
Re: 2022 Prediction Challenge: January
« Reply #12 on: February 02, 2022, 03:16:37 PM »
A decent number of responses, I'll put up one for February based on PIOMAS once PSC have done their January update.

I take Brigantine's point that a different function would discriminate between 0.1% and 1% better, but there aren't going to be enough of these to discriminate at that level anyway. I don't intend to pitch any others in a way that packs most people at one end of the scale.

Richard Rathbone

  • Nilas ice
  • Posts: 1418
    • View Profile
  • Liked: 300
  • Likes Given: 18
Re: 2022 Prediction Challenge: January
« Reply #13 on: February 08, 2022, 12:27:20 PM »