Arctic Sea Ice : Forum

Cryosphere => Arctic sea ice => Topic started by: Richard Rathbone on June 09, 2018, 07:47:24 PM

Title: September Predictions Challenge
Post by: Richard Rathbone on June 09, 2018, 07:47:24 PM
This is a challenge which accumulates scores across all the monthly polls on September sea ice to come up with an overall rating of how good the predictions made are. In addition to making a prediction, entrants are required to rate their confidence in that prediction. The higher the confidence, the narrower the margin of error you are allowed, but the higher score you get if the September ice ends up where you predicted it (and the bigger penalty you take if you miss).

Currently included polls are: JAXA daily minimum area, NSIDC September average.
Other polls may be added during the course of the season if their structure fits this challenge.

Points are scored as follows:
Very High Confidence: 10 points if you pick the correct bin, -10 points for all other bins.
High Confidence: 6 points for the correct bin, 2 points if one bin out, -2 points if two bins out, -6 points for all other bins
Medium Confidence: 4 points for the correct bin, 2 points if one bin out, 1 point if two bins out, -1 if three bins out, -2 if four bins out, -4 points for all other bins
Low Confidence: 2 points for the correct bin, 1 point if within 3 bins, -1 point if 4-6 bins out, -2 points for all other bins
Very Low Confidence: 1 point if in the correct bin, no score (or penalty) for any other bin.

Added for July.
Note on portmanteau and end of range bins: these are excluded from the challenge, you may either select a bin within the portmanteau range, or extend the range beyond the poll endpoint for your challenge entry. If the September values fall out of the normal range, scores will be assessed by extending the bin structure. e.g. an end result of 5.9 counts as two bins out for a 5-5.5 entry (currently the top bin in the July polls)

To enter, post guess and confidence in this thread before the closing date of the poll. Editting a post to change the prediction before the closing date is allowed, editting for any reason after the closing date for a poll will result in disqualification from the challenge.

Currently active polls are:
None

There is no requirement to enter all polls or to match your votes in the predictions threads.

Entries on the following polls have been completed
 June JAXA daily minimum area, https://forum.arctic-sea-ice.net/index.php/topic,2322.0.html, 
 June NSIDC SIE September average, https://forum.arctic-sea-ice.net/index.php/topic,2324.0.html
 July JAXA daily minimum area, https://forum.arctic-sea-ice.net/index.php/topic,2339.0.html
 July NSIDC SIE September average, https://forum.arctic-sea-ice.net/index.php/topic,2340.0.html
August poll for JAXA daily minimum https://forum.arctic-sea-ice.net/index.php/topic,2362.0.html
August poll for NSIDC Sept average https://forum.arctic-sea-ice.net/index.php/topic,2361.0.html

JAXA daily minimum was 4.46
NSUDC Sept average was 4.71

Compilation of entries

Richard Rathbone
June
JAXA - 3.75-4.25, Medium 2
NSDIC - 4.0-4.5, Medium 2
July
NSIDC av 4.5-5.0,  High 6
JAXA min 4.25-4.75, High 6
August
NSIDC: 4.75-5.25, High 2
JAXA: 4.25-4.75, High 6

Total 24

Paddy
June
JAXA - 3.75-4.25, Low 1
NSDIC - 4.25-4.75, Low 2
July
JAXA - 4.0-4.5, Low 2
NSDIC - 4.5-5.0, Low 2
August
JAXA: 4.0 to 4.5, Medium 4
NSIDC: 4.5 to 5.0, Medium 4

Total 15

Brigantine

June
JAXA - 4.00-4.50, Medium 4
NSIDC - 4.25-4.75, Medium 4
July entry:
JAXA - 4.25-4.75 Medium 4
NSIDC - 4.50-5.00 Medium 4
August entry:
JAXA - 4.00-4.50, High 6
NSIDC - 4.50-5.00, High 6

Total 28

Neven
June
NSIDC: Between 4.5 and 5.0 million km2, None 1
JAXA: 4.0 - 4.5 M km^2, None 1
July
JAXA minimum extent:      4.25-4.75,  Medium 4
NSIDC September extent:  4.5-5.0,  Medium 4
August
JAXA minimum extent:      4.0-4.5,  Medium 4
NSIDC September extent:  4.5-5.0,  High 6

Total 20

EgalSust
June
JAXA - 4.0-4.5, Medium 4
NSIDC - 4.5-5.0, Medium 4
July
JAXA - 4.25-4.75, High 6
NSIDC - 4.75-5.25, High 2
August
JAXA - 4.25-4.75, High 6
NSIDC - 4.75-5.25, High 2

Total 24

jdallen
June:
JAXA - 3.75-4.25, Medium 2
NSDIC - 4.0-4.5, Medium 2

Total 4

Daniel B.
June
NSIDC  4.5-5.0, High 6
JAXA  4.0-4.5, Medium 4

Total 10

Stephan
June
JAXA 4.0 to 4.5 Medium 4
NSIDC 4.5 to 5.0 High 6

Total 10

Viggy
June
JAXA - New record, Very High (this is ambiguous, 3.18 fits in two bins, but due to the comment I am taking it as 2.75-3.25, though 3.0 - 3.5 H would also have been a possible choice by the reasoning expressed there) -10
NSIDC - 2.75 - 3.25, High -6
August
JAXA: 3.25 to 3.75, High -6
NSIDC: 3.75 to 4.25, Very High -10

Total -32

gerontocrat
June
NSIDC - 4.0 to 4.5, none 0
JAXA   -  3.75 to 4.25, none 0

RealityCheck
June
JAXA min 3.75-4.25; high conf 2
NSIDC av 4.25-4.75; high conf 6

Total 8

Juan C. Garcia
June:
JAXA - 3.75-4.25, Medium 2
NSDIC - 4.0-4.5, Medium 2
July
JAXA minimum extent:      4.25-4.75,  Medium 4
NSIDC September extent:  4.5-5.0,  Medium 4

Total 12

uniquorn
June
JAXA min 3.5-4.0; medium 1
NSIDC av 3.5-4.0; low 1
July
JAXA min 3.5-4.0; medium 1
NSIDC av 3.75-4.25; medium 1
August
JAXA - 3.5-4.0, medium 1
NSIDC - 3.75-4.25 medium 1

Total 6

Steven
June
JAXA:    3.75-4.25,  Medium 2
NSIDC:  4.0-4.5,  Medium 2
July
JAXA minimum extent:      4.25-4.75,  Medium 4
NSIDC September extent:  4.75-5.25,  Medium 2
August
JAXA minimum extent:      4.0-4.5, High 6
NSIDC September extent:  4.5-5.0,  High 6

Total 22

Tetra
August
JAXA: 4.0 to 4.5, Medium 4
NSIDC: 4.5 to 5.0, Medium 4

Total 8

liefde
JAXA ASI September daily minimum: Between 3.5 and 4.0 million km^2, Very High -10
NSIDC SIE September average: Between 4.25 and 4.75 million km^2, Very High 10

Total 0

Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 09, 2018, 07:53:35 PM
JAXA - 3.75-4.25, Medium
NSDIC - 4.0-4.5, Medium

i.e. JAXA will score positive points for me if it ends up between 3.25-4.75 and NSDIC if it ends up between 3.5 and 5.0

Title: Re: September Predictions Challenge
Post by: Tor Bejnar on June 09, 2018, 08:10:19 PM
I "predict" (defined as "guess") ASI extent, area and volume will just beat out the current records in September.  My confidence is 'extremely' low, so I should get no points for guessing right (or wrong), and 1 point for honesty. ::)
Title: Re: September Predictions Challenge
Post by: Paddy on June 10, 2018, 01:43:30 PM
JAXA - 3.75-4.25, Low
NSDIC - 4.25-4.75, Low

I'm betting on "something similar to 2016" for both polls, but with wide confidence intervals that should cover most of the plausible outcomes.  I was tempted by "Medium" confidence as offering greater rewards in terms of points, but at this point "Low" felt more honest to the actual range of outcomes that could lie ahead.
Title: Re: September Predictions Challenge
Post by: Brigantine on June 12, 2018, 03:43:39 AM
JAXA - 4.00-4.50, Medium
NSIDC - 4.25-4.75, Medium

I approve of this game! It's a great idea.
Title: Re: September Predictions Challenge
Post by: Neven on June 12, 2018, 10:04:59 AM
Okay, I'll play too.  :)

June polls

NSIDC: Between 4.5 and 5.0 million km2
JAXA: 4.0 - 4.5 M km^2
Title: Re: September Predictions Challenge
Post by: be cause on June 12, 2018, 10:44:39 AM
hi Neven ; I note you have no confidence in your predictions .. :) b.c.
Title: Re: September Predictions Challenge
Post by: EgalSust on June 12, 2018, 05:07:58 PM
June:

JAXA - 4.0-4.5, Medium
NSIDC - 4.5-5.0, Medium

Low confidence would have been the honest choice...

edit: NSDIC -> NSIDC
Title: Re: September Predictions Challenge
Post by: gerontocrat on June 12, 2018, 05:56:02 PM
At what time (GMT) on what date do these polls close ?

Waiting to the last moment to get the last ounce of data ( from Mystic Meg and The Farmers' Almanac ) before posting my wholly scientific prediction.
Title: Re: September Predictions Challenge
Post by: jdallen on June 12, 2018, 06:04:22 PM
June:
JAXA - 3.75-4.25, Medium
NSDIC - 4.0-4.5, Medium
Title: Re: September Predictions Challenge
Post by: Daniel B. on June 12, 2018, 06:18:24 PM
NSIDC  4.5-5.0, High (6 of the past 10 years have been in this bin, averaging right in the middle, with only one year more than one bin away)

JAXA  4.0-4.5, Medium (Much greater spread in the data over the past decade.  Although 5 out of 10 have been in this bin, 4 out of 10 being more than one bin away)
Title: Re: September Predictions Challenge
Post by: Stephan on June 12, 2018, 10:04:59 PM
This is how I voted in the two threads:
JAXA 4.0 to 4.5 Medium
NSIDC 4.5 to 5.0 High
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 12, 2018, 10:51:20 PM
At what time (GMT) on what date do these polls close ?

Waiting to the last moment to get the last ounce of data ( from Mystic Meg and The Farmers' Almanac ) before posting my wholly scientific prediction.

2359 BST is the last timestamp that counts. i.e. about 4 days, 2 hours and 8 minutes from now.
Title: Re: September Predictions Challenge
Post by: Viggy on June 12, 2018, 11:03:26 PM
Chalk it up to me being an eternal pessimist but here goes literally nothing -
JAXA - New record, Very High (id like to vote High but the fact that there isn't a 1 bin out on the lower end leads me to think I should just commit to it)
NSIDC - 2.75 - 3.25, High (Given that we are 2 prediction bands below the previous low, I'll hedge with a sensible High bet here)

Win (Lose) big or go home!
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 16, 2018, 12:53:37 PM
The June polls are all closed. This is the last day for challenge entries. Anyone waiting for the last little bit of data to guide them has about 12 hours left.
Title: Re: September Predictions Challenge
Post by: gerontocrat on June 16, 2018, 02:18:35 PM
The June polls are all closed. This is the last day for challenge entries. Anyone waiting for the last little bit of data to guide them has about 12 hours left.
I think I went
NSIDC - 4.0 to 4.5
JAXA   -  3.75 to 4.25

My confidence is zero.
Title: Re: September Predictions Challenge
Post by: RealityCheck on June 16, 2018, 04:54:38 PM
Here goes...
JAXA min 3.75-4.25; high conf
NSIDC av 4.25-4.75; high conf
(High conf due to recently discovered crystal ball, now polished, plus a pinch of 'what the heck' feeling. 😁
Title: Re: September Predictions Challenge
Post by: Juan C. García on June 16, 2018, 05:01:55 PM
June:
JAXA - 3.75-4.25, Medium
NSDIC - 4.0-4.5, Medium


I voted the same way that jdallen.
Confidence? I think that there is a low confidence for everybody, including me. But let's say that I could expect one bin up, but several bins down.
Title: Re: September Predictions Challenge
Post by: uniquorn on June 16, 2018, 05:02:55 PM
JAXA min 3.5-4.0; medium
NSIDC av 3.5-4.0; low

Flatter bottom.
Title: Re: September Predictions Challenge
Post by: jdallen on June 16, 2018, 07:14:02 PM
June:
JAXA - 3.75-4.25, Medium
NSDIC - 4.0-4.5, Medium


I voted the same way that jdallen.
Confidence? I think that there is a low confidence for everybody, including me. But let's say that I could expect one bin up, but several bins down.

I figure my choices have about a 50-50 chance of being correct, thus Medium.

And I agree whole heartedly with your expectation about bins.
Title: Re: September Predictions Challenge
Post by: Steven on June 16, 2018, 09:35:55 PM
JAXA:    3.75-4.25,  Medium
NSIDC:  4.0-4.5,  Medium

This is based on the long term linear trends for September/minimum extent.

I'm not sure what to expect.  On the one hand, NSIDC sea ice area and compactness are pretty high at the moment.  That suggests a mediocre melt season.  On the other hand, warm air advection on the Siberian side of the Arctic has been strong.  So far this mainly affected Kara and Laptev Sea.  It remains to be seen how much it will affect the East Siberian Sea, and whether or not the ice arm in the East Siberian Sea could melt out this year.  Chukchi and Beaufort Sea should have thinner than normal ice according to PIOMAS, but it looks like weather conditions in those regions have been favorable for ice retention so far.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 17, 2018, 01:19:12 AM
June entries are now closed. Looks like an interesting mix of confidence.
Title: Re: September Predictions Challenge
Post by: Paddy on June 20, 2018, 08:22:23 AM
After the fact, I know, but I think the scoring system is a little prejudicial in giving people with a High or Medium confidence significantly better point rewards, eg as good a score when one bin out as people with a Low confidence would get for the correct bin only...
Title: Re: September Predictions Challenge
Post by: Brigantine on June 20, 2018, 09:32:38 AM
After the fact, I know, but I think the scoring system is a little prejudicial in giving people with a High or Medium confidence significantly better point rewards, eg as good a score when one bin out as people with a Low confidence would get for the correct bin only...
But significantly higher penalties if they're 10 bins out. The game's about avoiding penalties as much as it's about scoring points.

You have to get you average in positive territory, before you worry about how high you can go. I suspect most people with high confidence in June will end up on negative points come September, while most people with low confidence will be slightly positive and have a head start next year.

(I went medium confidence, shooting for slightly higher points at a calculated risk of taking negatives instead. In July I'll probably go high confidence at a greater reward but even greater risk)
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 20, 2018, 07:44:26 PM
After the fact, I know, but I think the scoring system is a little prejudicial in giving people with a High or Medium confidence significantly better point rewards, eg as good a score when one bin out as people with a Low confidence would get for the correct bin only...

This is by intent. The lower the confidence level, the further away from the correct bin the rewards crossover. The challenge is intended to reward both being accurate, and having a realistic appreciation of your level of accuracy, and this is a necessary part of that design.

In practice there aren't enough independent polls to really sort entrants by skill, and Napoleon's dictum applies, its better to be lucky than good.  ;)
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 23, 2018, 05:32:36 PM
SIPN median is 4.6, perhaps a touch higher than entrants here, but not much in it.

(I'd rate Rob Dekker's SIPN entry as  4.5 - 5.0  M  here, Nico Sun looks like 4.0 - 4.5 M but there are inconsistencies in the confidence measures he quotes, and it might be VL or H, Slater method 4.75 - 5.25 M, UCL model 4.0 - 4.5 L and the model of the Met Office 4.75 - 5.25 L)

Title: Re: September Predictions Challenge
Post by: Paddy on June 23, 2018, 06:58:32 PM
Quote

This is by intent. The lower the confidence level, the further away from the correct bin the rewards crossover. The challenge is intended to reward both being accurate, and having a realistic appreciation of your level of accuracy, and this is a necessary part of that design.

In practice there aren't enough independent polls to really sort entrants by skill, and Napoleon's dictum applies, its better to be lucky than good.  ;)

I get it, it just seems a bit odd that "Low" confidence gets worse marks when it's one bin wrong than "Medium" or "High". If they all had the same reward for a near miss (either 1 or 2 points, with an equivalent switch in the equivajent penalty slot) I think we would have seen a broader range of confidence levels taken.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 23, 2018, 07:41:13 PM


I get it, it just seems a bit odd that "Low" confidence gets worse marks when it's one bin wrong than "Medium" or "High". If they all had the same reward for a near miss (either 1 or 2 points, with an equivalent switch in the equivajent penalty slot) I think we would have seen a broader range of confidence levels taken.

What counts as a near miss is further away and more spread out for low confidence than for high confidence.
Title: Re: September Predictions Challenge
Post by: Brigantine on June 24, 2018, 01:44:25 AM
I guess the point is that for low confidence, 1 bin out is still a 'hit' and not a near miss at all, and it is slightly jarring that the reward has already dropped by half. Whereas the higher levels of confidence have rewards that look more like low-res bell curves. (even though they're not curved).

But I can accept the point that we want people to be trying to guess the exact correct bin, even if it has a low probability. Ultimately winning +1 point from a June poll is nothing to sneeze at. If the confidence levels had been described in reverse order, there would be no anchor at 10 points, so +1 point wouldn't have felt like such a raw deal.
Title: Re: September Predictions Challenge
Post by: Peter Ellis on June 24, 2018, 01:12:08 PM
Maybe it helps to reframe it.  Think of it like betting on a roulette wheel.  Each of us gets a hundred tokens.  Distribute those across the various bins, and at the end of the day, your score is however many tokens you put in the correct bin.

High confidence means you stack all your tokens in one bin.  If you're right, you score 100, otherwise you get nothing.

Medium confidence means you put 50 in one of the bins, and the rest in the adjacent bins.  If you're dead on - well you still only score 50, because you weren't really sure about it.  If you weren't quite dead on, you get to pick up 25 points or so because you had at least a few tokens in the right bin.

Low confidence means distributing your tokens widely.  Sure, you'll pick up a few points no matter what the result is, but only a few because you basically pulled the numbers out of your fundament.

Or, put yet another way, saying you have "high confidence" in one bin is the same as saying you're certain it won't be in any of the other bins - so why would you expect to score points for them?
Title: Re: September Predictions Challenge
Post by: Peter Ellis on June 24, 2018, 01:13:27 PM
(Side note - next year it might be interesting to try the "distribute 100 tokens" approach since that lets people give a more nuanced description of their confidence intervals, which could be non-normal, asymmetric or even bimodal.)
Title: Re: September Predictions Challenge
Post by: oren on June 24, 2018, 06:08:58 PM
Well explained and a good idea Peter.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 25, 2018, 04:36:19 PM
(Side note - next year it might be interesting to try the "distribute 100 tokens" approach since that lets people give a more nuanced description of their confidence intervals, which could be non-normal, asymmetric or even bimodal.)

However, a reward function that rewarded this sort of entry for getting its distribution right would not be simple. This simple reward of +1 per token in the right bin rewards overconfidence, since whatever the actual uncertainty, putting every token on the mode is the optimum play.

If you have a strongly non-normal distribution of your expectations, such that all the other options have negative expectations, thats what my VL confidence option is for. If the arctic behaviour actually fits your expectations, its the winning play.

This is actually a 10-token game, with restrictions on how you can place them, and the penalties for not having a token in the winning bin are how I deal with penalising the overconfidence that this type of structure would otherwise reward.

 
Title: Re: September Predictions Challenge
Post by: Paddy on June 26, 2018, 09:00:19 AM
Only a 10-token game for medium and high confidence, though. For low confidence it's an 8-token game (if we ignore the away-from-mode negative penalties).
Title: Re: September Predictions Challenge
Post by: Brigantine on June 26, 2018, 09:07:08 AM
Not quite, because the there's a 0.5 Mm2 range of final values that land you in the correct bin, while each bin further away only covers 0.25Mm2 on each side. So it's more like 20 for VH, 16 for High, 14 for Medium, 10 for Low and 2 for VL.
Title: Re: September Predictions Challenge
Post by: Paddy on June 26, 2018, 10:35:48 PM
Ok... but let's also consider the penalties now (up to 3 bins out). And let's also reduce the effective value of each benefit or penalty by 25% of the points/penalties available for each step from the modal value (ie multiply by 0.75 for 1 step out, 0.5 for 2 steps out, and 0.25 for 3 steps out) as the further they are from your modal guess, the less likely you consider them to be.

That makes the net benefit available  -10 for VH, 10 for H, 11.5 for M, 7 for L, and 2 for VL. (According to my mental arithmetic, anyway).

The numbers will shift slightly depending on how you model this, but I'm not surprised most people opted for M.

EDIT: considering these instead as predicted mean points return according to a model of guessing as above where you're right 4 times in ten, wrong by 1 bin 3 times in ten, wrong by two bins 2 times in ten, and wrong by 3 bins 1 time in ten, this gives the average predicted points return of each level of confidence at:
VH: -2
H: 2
M: 2.3
L: 1.4
VL: 0.4

Again, according to my mental arithmetic, which is feeling a tad strained just now.
Title: Re: September Predictions Challenge
Post by: Brigantine on June 26, 2018, 10:48:36 PM
By choosing to go 3 bins out, you're implicitly judging this from the perspective of someone with medium confidence.

How about judge it from the perspective of someone with quite low confidence, and look 9 bins out:
(weights: 1, 0.9, 0.8 etc. to 0.1 9 bins out.)

-70 for VH, -21.2 for H, -2.6 for M, +3.4 for L, +2.0 for VL
Title: Re: September Predictions Challenge
Post by: Paddy on June 26, 2018, 11:53:30 PM
Overall, the different confidence levels would pay off best at the following ranges, by this model:

VH: If this model goes 0 bins out
H: If this model goes 1 bin out (weighted at 1, 0.5) or 2 bins out (weighted at 1, 0.67, 0.33)
M: 3 bins out to X bins out (and I cannot be bothered to solve for X, but I'd guess it's around the 5 mark)
Etc
Title: Re: September Predictions Challenge
Post by: Brigantine on June 27, 2018, 02:08:56 AM
(Side note - next year it might be interesting to try the "distribute 100 tokens" approach since that lets people give a more nuanced description of their confidence intervals, which could be non-normal, asymmetric or even bimodal.)
However, a reward function that rewarded this sort of entry for getting its distribution right would not be simple.
I'd like to suggest this reward function: (TLDR: logs)

If the final result is in a bin where you placed X tokens, you win A*log(X)-C

Where A and C are constants set to scale the range of outcomes from perfect skill = +10, and an honest zero skill = ±0.
A = 10/log[number of bins], C = A*log(100)-10
Assuming you use 25 non-overlapping 0.25Mm2 bins, from <0.25 up to 6.00+, and using natural logs... A = 3.1067, C = 4.3068


If you were so sure the result wouldn't be in a particular bin, that you put 0.00 tokens there, and then it really happens... technically you should be completely discredited and should end up on minus infinity. But without allowing fractional tokens, we can be generous and call it -10 - as though you had placed 0.16 tokens there.

Example scoring: Final result lands in a bin where you placed...
0 tokens -> -10.00 points
1 token --> -4.31
2 tokens -> -2.15
3 tokens -> -0.89
4 tokens -> ±0.00 points (100 tokens spread evenly across 25 bins will get you here, i.e. honest zero skill)
5 tokens -> +0.69
6 tokens -> +1.26
7 tokens -> +1.74
10 tokens -> +2.85
11 tokens -> +3.14
20 tokens -> +5.00 points
21 tokens -> +5.15
50 tokens -> +7.85
51 tokens -> +7.91
90 tokens -> +9.67
91 tokens -> +9.71
99 tokens -> +9.97
100 tokens +10.00 points

For each token you add to a bin up to matching the probability, you expect to gain at least ~0.03 points, but stacking extra tokens in a bin beyond its probability earns you less than ~0.03 for each token.


Having said that, I like the game as it is currently (nice and simple), and do not endorse changing it  :D
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 28, 2018, 02:34:30 PM
I hoped these sorts of reward calculations would happen, and I'm a little disappointed that the predictomat didn't do them.

Nearly July, who is still feeling they made the right choice? When I saw the entries coming in I thought I might have pitched too low a confidence at medium, but I'm feeling that it might have been too high. Melt in the later part of June has not only been slow, but slower than I thought possible.

There's going to be a rule change for July if the poll structure stays the same, I didn't look closely enough at the scales and I need to clarify what happens with portmanteau and end of range bins. (they'll be specifically excluded, and the 0.5 wide bin centered on every 0.25 structure will be deemed to continue indefinitely for the purposes of this challenge) I'll write it out explicitly after the July polls are open but please wait till I have posted to say the challenge is open for July entries before posting them.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on June 30, 2018, 01:38:19 PM
Top post editted with details of the July polls. Entries may now be made here.

While I could make my June entry 6 months in advance, for July there are indicators on the current state that I want to see, so I'm not going to be straight off the mark this time. My hunch is up a bin or two and still medium but I'll want PIOMAS to be out and agree with my hunch of where it is before I firm up on that.
Title: Re: September Predictions Challenge
Post by: Paddy on July 01, 2018, 11:03:26 AM
My predictions:

JAXA - 4.0-4.5, Low
NSDIC - 4.5-5.0, Low

As I've said, "Low" confidence is no way to win points. But I want the positive outcome range to match up to a decent confidence interval, and outliers like 2012 mean that has to be quite wide.
Title: Re: September Predictions Challenge
Post by: echoughton on July 01, 2018, 11:49:04 AM
I would prefer to wait until September to make my bold prediction of SIE extent in September, but I am sticking with my original optimism.
4.50-5-0
Title: Re: September Predictions Challenge
Post by: EgalSust on July 01, 2018, 01:56:22 PM
July
JAXA - 4.25-4.75, High
NSIDC - 4.75-5.25, High
Title: Re: September Predictions Challenge
Post by: uniquorn on July 01, 2018, 09:15:33 PM
the compilation of entries appears to be incomplete
Title: Re: September Predictions Challenge
Post by: Brigantine on July 11, 2018, 08:50:50 AM
JAXA - 4.00-4.50, Medium
NSIDC - 4.25-4.75, Medium
July entry: (subject to modification in the final days)
JAXA - 4.25-4.75 Medium
NSIDC - 4.50-5.00 Medium

Very tough choosing between medium confidence and high confidence. Where's that predict-o-matic up to re confidence intervals?

Also I like the new rules re end-of-range bins.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on July 11, 2018, 12:48:42 PM
the compilation of entries appears to be incomplete

Several added. It should be complete for June now, if I missed anyone post or message me.

Title: Re: September Predictions Challenge
Post by: Richard Rathbone on July 11, 2018, 01:22:17 PM
For July polls I like to use momentum indicators, particularly the Schroeder melt pond analysis, but also the June cliffs on PIOMAS volume, and NSIDC area.

May data from Schroeder was 5.3 with confidence given by a 0.5 standard deviation.
June data is 4.7 +- 0.5 but I'm not sure if thats a standard deviation or a confidence range.
June PIOMAS cliff is there but modest. June NSIDC cliff is absent or in the wrong direction but the data where it shoots up is compromised by satellite tests, so I'm just treating it as absent rather than an upwards cliff in June cancelled by a downwards one in July.

I'm picking the 4.5-5.0 bin because 4.7 is in the middle of that and 4.25-4.75 for the minimum because the minimum tends to be about one bin less than the September average. There are quite a few years in the 4.5-5.0 NSIDC pack and a modest cliff in PIOMAS supports this being another one of them.

Assuming a 0.5 standard deviation gives roughly 40% chance of the picked bin being correct, 30% of being 1 out, 15% of being 2 out, 10% of being 3 out and 5% of being 4 or more out.
I make that an expectation of -2.0 points for VH, 1.8 for H, 2.1 for M. VH looks overconfident, but a 0.5 standard deviation leaves H and M pretty closely matched. If I'd had the Schroeder May projection in time I'd definitely have stuck with medium (though my bin choice would have been different). The June projection from Schroeder ought to have a tighter confidence limit so I'm going to guess that the quoted +-0.5 is actually 2 standard deviations and go High.

NSIDC av 4.5-5.0,  High
JAXA min 4.25-4.75, High
Title: Re: September Predictions Challenge
Post by: Ned W on July 11, 2018, 04:19:41 PM
Where's that predict-o-matic up to re confidence intervals?

Thanks for asking.  Unfortunately the predict-o-matic's confidence is never "High".  Here's the latest (for JAXA):

Predicted daily min = 4.71
95% CI for prediction = 3.68 to 5.74

Here are the probabilities for the bins:

New record0%
Under 2.50%
2.5 to 30%
2.75 to 3.250%
3 to 3.51%
3.25 to 3.753%
3.5 to 47%
3.75 to 4.2515%
4 to 4.526%
4.25 to 4.7535%
4.5 to 537%
4.75 to 5.2532%
5 to 5.523%
Over 5.2515%

The probabilities sum to more than 100% because the bins overlap.  The uncertainty is very wide until late July. 
Title: Re: September Predictions Challenge
Post by: Random_Weather on July 11, 2018, 04:25:23 PM
My own is: 4.24 +-0.33 Mio km^2

Will also published in the SIPN July Outlook, for details go here: https://www.docdroid.net/9FEP0oD/model.docx

For Performance:
(https://forum.arctic-sea-ice.net/proxy.php?request=http%3A%2F%2Ffs5.directupload.net%2Fimages%2F180711%2Feovapyit.png&hash=c2f0c8f12387ec3a4b289311a2ee3c28) (http://www.directupload.net)
Title: Re: September Predictions Challenge
Post by: Steven on July 11, 2018, 08:21:42 PM
June NSIDC cliff is absent or in the wrong direction but the data where it shoots up is compromised by satellite tests, so I'm just treating it as absent rather than an upwards cliff in June cancelled by a downwards one in July.

I don't see a reason to ignore the NSIDC data  (apart from the obvious outlier on 27 June).  In fact, JAXA is in good agreement with NSIDC: Both of them have very high compactness in the second half of June, and a strong decrease of compactness in the first week of July:

(https://i.imgur.com/1enlyBO.png)

May data from Schroeder was 5.3 with confidence given by a 0.5 standard deviation.
June data is 4.7 +- 0.5 but I'm not sure if thats a standard deviation or a confidence range.

It's called forecast error in the Schroeder et al. 2014 paper.  It's obtained by simulating the "predictions" that the model would have generated in previous years, using only data that were available at the moment of the "prediction".  E.g. they simulate a prediction for September 2010 by using the regression equation over 1979-2009 and then feeding the May/June 2010 data into that equation.  They repeat that procedure for every year from 1984 to 2013 and then calculate the standard deviation of the errors of the simulated "predictions", giving them 0.44M km2 in the paper.  I guess the 0.5M km2 is obtained by including the last 4 years of data too (up to 2017) and perhaps rounding to the nearest multiple of 0.1M km2.
Title: Re: September Predictions Challenge
Post by: Brigantine on July 12, 2018, 12:23:36 AM
95% CI for prediction = 3.68 to 5.74
Thanks. Definitely sticking with medium confidence then!
Title: Re: September Predictions Challenge
Post by: uniquorn on July 12, 2018, 11:52:08 PM
uniquorn
JAXA min 3.5-4.0; medium
NSIDC av 3.75-4.25; medium
Title: Re: September Predictions Challenge
Post by: Steven on July 14, 2018, 12:30:38 PM
JAXA minimum extent:      4.25-4.75,  Medium
NSIDC September extent:  4.75-5.25,  Medium

This is based on the calculations discussed here (https://forum.arctic-sea-ice.net/index.php/topic,2340.msg161754.html#msg161754).  Updating those calculations to use the latest July data, the calculated values are slightly lower than when using the June data, but still within the same bin.
Title: Re: September Predictions Challenge
Post by: Neven on July 14, 2018, 01:57:50 PM
JAXA minimum extent:      4.25-4.75,  Medium
NSIDC September extent:  4.5-5.0,  Medium

My predictions for June didn't have any confidence levels, because I didn't get how it works. Can I still change them to 'low'?

Quote
Neven
June
NSIDC: Between 4.5 and 5.0 million km2, None
JAXA: 4.0 - 4.5 M km^2, None
Title: Re: September Predictions Challenge
Post by: Juan C. García on July 14, 2018, 02:14:55 PM
JAXA minimum extent:      4.25-4.75,  Medium
NSIDC September extent:  4.5-5.0,  Medium

Same as Neven.
Just a coincidence. I did not copy him.  ;)
________________________________________
Edit:
This month, the level of confidence is medium.
________________________________________
Last month:
June:
JAXA - 3.75-4.25, Medium
NSDIC - 4.0-4.5, Medium


I voted the same way that jdallen.
Confidence? I think that there is a low confidence for everybody, including me. But let's say that I could expect one bin up, but several bins down.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on July 14, 2018, 03:16:26 PM
JAXA minimum extent:      4.25-4.75,  Medium
NSIDC September extent:  4.5-5.0,  Medium

My predictions for June didn't have any confidence levels, because I didn't get how it works. Can I still change them to 'low'?

Quote
Neven
June
NSIDC: Between 4.5 and 5.0 million km2, None
JAXA: 4.0 - 4.5 M km^2, None

I'm treating None as Very Low, (in all cases, not just yours), which is the option that I put in for those that didn't get it. Its far too late to change.

Almost too late for this month too, a little under 10 hours left from this post.
Title: Re: September Predictions Challenge
Post by: Neven on July 14, 2018, 04:07:52 PM
Very low is perfect.  :)
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on July 23, 2018, 06:58:37 PM
Compilation should now be up to date. Among those that have entered for both months, there is a general trend to higher predictions, almost everyone being 1 or 2 bins higher in July than June but the confidence is very little changed, almost all sticking at medium.

The July SIPN report shows a modest uptick, from 4.6 to 4.7, but almost all the movement is due to statistical methods where the change is pretty similar to the changes people made here.

In June I picked out: Rob Dekker's SIPN entry as  4.5 - 5.0  M  here, Nico Sun looks like 4.0 - 4.5 M but there are inconsistencies in the confidence measures he quotes, and it might be VL or H, Slater method 4.75 - 5.25 M, UCL model 4.0 - 4.5 L and the model of the Met Office 4.75 - 5.25 L.

For July I'd rate Rob Dekker at 5.0-5.5 H (borderline M), Nico Sun 4.25-4.75 M (same problem with interpreting the confidence measures as June), Slater method 4.75-5.25 M, UCL and Met Office did not made a new assessment for July. Random Weather I'd put as 4-4.5 H (also borderline M, there's an element of judgement in translating a model into an entry, so unless the poster makes that judgement in their post, I'm just treating it as commentary rather than an entry). Schroeder I'm putting at 5.0-5.5 M for June and 4.5-5.0 M for July, although that method ought to be have narrower confidence limits in July, and I went H in my own entry here on that basis, their SIPN entry did not change. The predictomat I rate at 4.5-5.0 M.

Looking forward to August, my preferred method switches to Slater which is projecting around 4.6 for Sept 11th at the moment. http://cires1.colorado.edu/~aslater/SEAICE/

Title: Re: September Predictions Challenge
Post by: gerontocrat on July 23, 2018, 10:11:48 PM
Looking forward to August, my preferred method switches to Slater which is projecting around 4.6 for Sept 11th at the moment. http://cires1.colorado.edu/~aslater/SEAICE/

The Slater figure is for the NSIDC DAILY extent, n'est-ce-pas? Which is usually a bit higher than the JAXA daily extent? My incredibly crude use of 10 previous years average remaining melt produces a JAXA daily extent minimum of 4.59 million km2. Perhaps I should apply the same method to NSIDC daily data. Yes I will.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on July 24, 2018, 08:02:54 PM
Looking forward to August, my preferred method switches to Slater which is projecting around 4.6 for Sept 11th at the moment. http://cires1.colorado.edu/~aslater/SEAICE/

The Slater figure is for the NSIDC DAILY extent, n'est-ce-pas? Which is usually a bit higher than the JAXA daily extent? My incredibly crude use of 10 previous years average remaining melt produces a JAXA daily extent minimum of 4.59 million km2. Perhaps I should apply the same method to NSIDC daily data. Yes I will.

The application of it on the link I quoted is a daily forecast for 50 days in advance. It can also be used for forecasts over other periods, (and is done so for SIPN), by those with access to the model details. For the August poll here, I look at what it has said 50 days in advance across September, and subject to a couple of reality checks, roughly average it.
Title: Re: September Predictions Challenge
Post by: gerontocrat on July 24, 2018, 08:58:37 PM
Looking forward to August, my preferred method switches to Slater which is projecting around 4.6 for Sept 11th at the moment. http://cires1.colorado.edu/~aslater/SEAICE/

The Slater figure is for the NSIDC DAILY extent, n'est-ce-pas? Which is usually a bit higher than the JAXA daily extent? My incredibly crude use of 10 previous years average remaining melt produces a JAXA daily extent minimum of 4.59 million km2. Perhaps I should apply the same method to NSIDC daily data. Yes I will.

The application of it on the link I quoted is a daily forecast for 50 days in advance. It can also be used for forecasts over other periods, (and is done so for SIPN), by those with access to the model details. For the August poll here, I look at what it has said 50 days in advance across September, and subject to a couple of reality checks, roughly average it.
The average date for the JAXA minimum (last 10 years) is Sep 12th - 50 days from now. The Slater 50 day projection is likely to be very close to the minimum this year (unless the oddness of this year produces a very mild late summer and fall?) ?
Title: Re: September Predictions Challenge
Post by: Phil42 on July 30, 2018, 01:51:08 PM
I'm taking a little bit of a different approach here to make my (August?) prediction. I looked at the JAXA minimum extent ranks the previous years placed in (see graph below) and from there took the average rank, which is 4.21 from 1990-2017.

For 2018 to be placed at exactly this average rank of 4.21, the JAXA minimum extent would have to be 4'344'239km2.

So my prediction is
4.0 - 4.5 JAXA
4.5 - 5.0 NSIDC

(I am aware that is a really bad method to make a prediction but I was just wondering what the outcome of the calculation would be and though I might as well share it  ;))
Title: Re: September Predictions Challenge
Post by: Paddy on July 31, 2018, 11:01:26 AM
I think for the August round we should generally be able to state things with a higher level of confidence, as we're now only about 45 days from the minimum. And although the range of minima for the past ten years has been very large, it really doesn't look like this year will be on an extreme end of that range.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on July 31, 2018, 05:53:55 PM
I think for the August round we should generally be able to state things with a higher level of confidence, as we're now only about 45 days from the minimum. And although the range of minima for the past ten years has been very large, it really doesn't look like this year will be on an extreme end of that range.

August Polls have been up long enough to get a fair number of entries so even though they aren't stickied yet, I'm opening this Challenge for entries based on them. August 12th (BST) is the final day to post here.

I expect there will be some upwards movement in confidence, though its quite a stretch to go from H to VH for me, that overconfidence penalty is really quite severe. I'm still happy with my picks from last month's polls though, so I'll be thinking about whether I can stretch on the NSIDC average as the deadline approaches.
Title: Re: September Predictions Challenge
Post by: jdallen on July 31, 2018, 06:02:04 PM
If a anything, the last few days have driven home to me just how volatile and unpredictable conditions are. Sticking with medium.
Title: Re: September Predictions Challenge
Post by: Paddy on July 31, 2018, 06:58:14 PM

I expect there will be some upwards movement in confidence, though its quite a stretch to go from H to VH for me, that overconfidence penalty is really quite severe. I'm still happy with my picks from last month's polls though, so I'll be thinking about whether I can stretch on the NSIDC average as the deadline approaches.

Fair point if you're already going H.  But personally, I think it's about time I went up from my L.

If a anything, the last few days have driven home to me just how volatile and unpredictable conditions are. Sticking with medium.

Fair point as well; but there's now less time left for that volatility to happen.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on August 01, 2018, 12:57:07 PM
If a anything, the last few days have driven home to me just how volatile and unpredictable conditions are. Sticking with medium.

This is why I like the Slater model. It gives me a method of assessing the implications of concentration changes now, for the extent in 50 days time. And at the moment its completely serene about them, ticking the amount of ice it expects to survive up slightly over the past week. i.e. ice that was going anyway is going anyway, but the places that are likely to be marginal in September are looking a teeny bit more solid. However, maybe next week those marginal places start getting eaten ... which is why I'm both happy with my July pick, and running down the clock before making the August one.
Title: Re: September Predictions Challenge
Post by: EgalSust on August 01, 2018, 03:03:45 PM
August
JAXA - 4.25-4.75, High
NSIDC - 4.75-5.25, High

for now... sticking with the following Slater method, for now at least

Slater probabilistic is showing very weird behaviour: a late August minimum and a sharp uptick after mid-September:

Title: Re: September Predictions Challenge
Post by: Tor Bejnar on August 01, 2018, 03:47:17 PM
I "predict" (defined as "guess") ASI extent, area and volume will just beat out the current records in September.  ...
I now guess the September minimums will be just above the current records, so (to enumerate the guesses) I choose the lowest bin that starts above the record low for each measure (extent, area, volume).  I still say guesses should be recognized as having no confidence.  As of yet unknowable weather, of course, will be a significant contributor to making or breaking every guess, prediction or sure thing declaration.
Title: Re: September Predictions Challenge
Post by: uniquorn on August 02, 2018, 02:39:39 PM
August
JAXA - 3.5-4.0, medium
NSIDC - 3.75-4.25 medium
Warm winds and waves
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on August 09, 2018, 04:02:56 PM
The clock is ticking and its time I put my entry in. While 2 weeks ago I was happy with the bin I picked last month, Slater has shifted fairly dramatically compared to anomaly persistence during that time, my reading of Wipneus' home brew areas backs up that shift, and I'm going up a bin.

VH or H that is the question. Breakeven expectation if only being 1 bin out is considered happens at a 75% chance of hitting the right bin. (i.e. at 80% correct, 10% 1 high, 10% 1 low, VH comes out ahead) While I am that confident on the high side, I'm not on the low side, so I'll have to stick at H.

NSIDC: 4.75-5.25, High
JAXA: 4.25-4.75, High

Because I'm not that confident on the low side, I've put JAXA two bins lower, rather than my normal 1 bin lower than NSIDC. (historical average is 0.3 lower, slightly more than 1 bin)

Title: Re: September Predictions Challenge
Post by: Brigantine on August 10, 2018, 01:52:22 AM
JAXA - 4.00-4.50, Medium
NSIDC - 4.25-4.75, Medium
July entry:
JAXA - 4.25-4.75 Medium
NSIDC - 4.50-5.00 Medium
August entry: Same bin as June, but with higher confidence
JAXA - 4.00-4.50, High
NSIDC - 4.50-5.00, High


And where's that predict-o-matic up to now, including confidence intervals?
Found it! Mean is 4.23, 95% range is 3.70-4.77. Thanks Ned!
My estimate is slightly higher than that, but not by enough to pick a higher bin. Reason being highish concentration & potential for dispersion.

[Update - joined the bandwagon and raised the NSIDC entry by one bin so that it's 2 bins above JAXA]
Title: Re: September Predictions Challenge
Post by: Paddy on August 10, 2018, 07:19:04 AM
JAXA: 4.0 to 4.5, Medium
NSIDC: 4.5 to 5.0, Medium

Was very tempted to go High this time...

EDIT: my previous votes for comparison:

Quote
June
JAXA - 3.75-4.25, Low
NSDIC - 4.25-4.75, Low
July
JAXA - 4.0-4.5, Low
NSDIC - 4.5-5.0, Low
Title: Re: September Predictions Challenge
Post by: Viggy on August 10, 2018, 12:09:33 PM
JAXA: 3.25 to 3.75, High
NSIDC: 3.75 to 4.25, Very High

I don't know if my rationale is valid or not but accumulated cyclonic energy this hurricane season has been running quite high and all that heat and moisture eventually ends up polewards.

http://tropical.atmos.colostate.edu/Realtime/ (http://tropical.atmos.colostate.edu/Realtime/)

I'd imagine that transfer of energy can only exacerbate a GAC like event late in the melt season. Expecting September to be very slow in its re-freezing because of all this energy.
Title: Re: September Predictions Challenge
Post by: Tetra on August 10, 2018, 10:19:15 PM
JAXA: 4.0 to 4.5, Medium
NSIDC: 4.5 to 5.0, Medium

Was very tempted to go High this time...

 8)

I’ll go for those same odds.
Title: Re: September Predictions Challenge
Post by: Steven on August 11, 2018, 12:45:35 PM
JAXA minimum extent:      4.0-4.5, High
NSIDC September extent:  4.5-5.0,  High

This is slightly lower than my vote in the July polls, but higher than in the June polls.

NSIDC extent is currently 6th lowest or so.  Extent trajectories for previous years suggest about 4.6 million km2 for the NSIDC September extent.  On the other hand, Slater's method seems to suggest about 4.9 million km2 for the NSIDC September extent.  In both cases this is in the 4.5-5.0 bin.

For JAXA extent I choose the 4.0-4.5 bin, which is two bins below NSIDC.  The difference between JAXA minimum and NSIDC September extent was about 0.3 million km2 on average in the past 15 years.  But in the past 6 years (since JAXA started to use AMSR2 data) it was always more than 0.3, and almost 0.4 million km2 on average.
Title: Re: September Predictions Challenge
Post by: Neven on August 11, 2018, 01:15:34 PM
JAXA minimum extent:      4.0-4.5,  Medium
NSIDC September extent:  4.5-5.0,  High

In June it was:
JAXA: 4.0-4.5, None
NSIDC: 4.5-5.0, None

And for July it was:
JAXA: 4.25-4.75, Medium
NSIDC: 4.5-5.0, Medium

My NSIDC prediction has been very consistent!  ;)
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on August 12, 2018, 01:30:45 PM
Last day for challenge entries. Deadline is about 11 hours from this post.
Title: Re: September Predictions Challenge
Post by: liefde on August 12, 2018, 11:39:08 PM
JAXA ASI September daily minimum: Between 3.5 and 4.0 million km^2, Very High
NSIDC SIE September average: Between 4.25 and 4.75 million km^2, Very High

Purely based on a climatic heli-view, based on good friend Jörg (Kachelmann)'s model input.
And with Aug. 16th ECMWF HD predictions taking another pull at the NH, with Greenland and Svalbard meltwater getting warmer than ever, and SST anomalies of all oceanic surroundings at record high for August, thick new heat blankets of CH4 at surface level all over EurAsia. The energy has to go somewhere.
Title: Re: September Predictions Challenge
Post by: Wherestheice on August 16, 2018, 01:48:15 AM
Hopefully the jaxa data can come in here soon, so we can make some good guesses on what the minimum will be.
Title: Re: September Predictions Challenge
Post by: Juan C. García on August 18, 2018, 05:15:27 PM
Last day for challenge entries. Deadline is about 11 hours from this post.

Seems that I like to break the rules...  ;D  :-[
Sorry, I forgot to put here my numbers:

June:
   JAXA: 3.75 - 4.25, Low
   NSIDC: 4.0 – 4.5, Low
July:
   JAXA: 4.25 – 4.75 Medium
   NSIDC: 4.5 – 5.0 Medium
August:
   JAXA: 4.0 – 4.5 High
   NSIDC: 4.25 – 4.75 Medium

I can see that NSIDC September average goes a little above 4.75M km2. It will depend of the freezing on Sep. 15th-30th.
Title: Re: September Predictions Challenge
Post by: Brigantine on September 21, 2018, 10:24:47 PM
It's looking very likely JAXA will be in the 4.25-4.50 range, and NSIDC will be either side of 4.75.

The rest of us are still waiting for NSIDC to make its mind up, but Neven can be scored now.

He got 6 out of 6! Jackpot. +20 points  ;D
(would have been an even higher score if he picked a confidence level in June)


I picked all the same bins as him except for June NSIDC (I picked 4.25-4.75). So I got at least 5/6 right. And I had higher confidence levels in June & for August JAXA.

If NSIDC comes in over 4.75, I get +26 points, if it comes in under, I get +28. It's more luck than skill, I swear!  :o Maybe I should quit while I'm ahead and not play in the freezing season...  :P
Title: Re: September Predictions Challenge
Post by: Paddy on September 21, 2018, 11:22:55 PM
I got 7 points for Jaxa (1 point for missing by 1 bin in June on Low, 2 points for a hit in July on Low and 4 points for a hit in August on Medium). As expected, going for Low confidence dragged down the scores, but it felt like the most honest description, especially in June. Probably 7 or 8 for NSIDC, making it probably 14 or 15 points.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on September 25, 2018, 12:41:04 PM
August predictions editted into the top post, if I missed any, let me know.

One titbit from August SIPN: Slater model went for 4.75 which might well be taken as 4.5-5.0 VH, though there's not really enough information given on the uncertainty to be sure it should be VH rather than H.

I'll tot up the scores once the NSIDC September report is out.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on October 06, 2018, 01:43:50 PM
JAXA daily minimum was 4.46
NSIDC Sept average was 4.71

Scores (details in 1st post if you want to check my scoring)

Richard Rathbone 24
Paddy 15
Brigantine 28
Neven 20
EgalSust 24
jdallen 4
Daniel B. 10
Stephan 10
Viggy -32
gerontocrat 0
RealityCheck 8
Juan C. Garcia 12
uniquorn 6
Steven 22
Tetra 8
liefde 0

Most regular competitors either hit the bull's eye or missed by at most one bin on all entries.
Almost every use of VH was overconfident and took the -10 penalty, although plenty of entrants would have got lucky if they had used it, but it would have turned out badly for me if I had gone VH when I was thinking about it.
Title: Re: September Predictions Challenge
Post by: Brigantine on October 07, 2018, 12:38:53 PM
Interesting that NSIDC was exactly one bin above JAXA - 0.25. So both were just 0.04 below the bin boundary.

A look at the confidence levels used by the 5 most successful participants:

28 Brigantine - 4x Medium, 2x High, maximum luck
24 Richard Rathbone - 4x High, 2x Medium, estimates drifted up 2-3 bins over time
24 EgalSust - 4x High, 2x Medium, narrowly missed out on an incredible score of 32 (if NSIDC were 0.04 higher)
22 Steven - 4x Medium, 2x High, estimates followed the predict-o-matic - up 2-3 bins in July then 1 bin back down
20 Neven - average confidence of Low-Medium (partly by accident), maximum bin accuracy.

Daniel B. and Stephan both did very well in June (10 points, called NSIDC dead accurately with High Confidence), but then didn't come back to this thread in July or August.
Title: Re: September Predictions Challenge
Post by: Stephan on October 07, 2018, 08:28:59 PM
I would have stayed with my bets throughout the summer and therefore I didn't repeat them. Should I have done so?
Title: Re: September Predictions Challenge
Post by: Brigantine on October 07, 2018, 08:52:10 PM
I would have stayed with my bets throughout the summer and therefore I didn't repeat them. Should I have done so?
If you had made the same entry in July and in August, you would have won!
Title: Re: September Predictions Challenge
Post by: Stephan on October 09, 2018, 06:11:37 PM
OK. As a newbie here in this forum and as a non climate scientist I feel honoured.
If there will be a comparable thread in 2019 I will not forget to repeat my June predictions in July and August!
Title: Re: September Predictions Challenge
Post by: Brigantine on January 16, 2019, 05:29:24 AM
Let's re-open this for the January poll for the maximum...
Title: Re: September Predictions Challenge
Post by: Paddy on January 16, 2019, 08:00:30 AM
Let's re-open this for the January poll for the maximum...

We could do that, although we never had a December (June-equivalent) poll this freezing season.
Title: Re: September Predictions Challenge
Post by: Richard Rathbone on January 17, 2019, 02:15:32 PM
Feel free to post your predictions for the maximum. I think I'd go 13.875-14.125 High. I actually voted in the 13.75-14.0 bin, but I was torn between the two and think the risk is mainly upside.

Title: Re: September Predictions Challenge
Post by: Klondike Kat on January 18, 2019, 12:41:28 AM
Looking at last year, the top two for their June predictions were Daniel B. and Stephan, with ten points each.  Congrats to both.
Title: Re: September Predictions Challenge
Post by: Stephan on January 18, 2019, 06:05:41 PM
Thank you...
Title: Re: September Predictions Challenge
Post by: Paddy on January 21, 2019, 02:47:49 PM
14 - 14.25, High.

Trying to get out of the habit of excessively low confidence that I demonstrated in the melting season. It's always possible to be wrong of course, but the numbers in this game absolutely favour going for High confidence whether you're sure or not.
Title: Re: September Predictions Challenge
Post by: Brigantine on January 31, 2019, 03:09:20 AM
January JAXA max prediction: 14 - 14.25 Medium (assuming it's being judged as 0.125M wide bins as per the poll)

(or 14 - 14.5 High if it's being judged as 0.25M wide bins as per predictions of the minimum)
Title: Re: September Predictions Challenge
Post by: Paddy on February 23, 2019, 11:57:25 AM
14 - 14.25, High.

Now I'm wishing I had adjusted this up to 14.125 - 14.375 like I did in the main max poll.
Title: Re: September Predictions Challenge
Post by: Paddy on March 17, 2019, 11:38:08 AM
2 points Paddy, 2 points Brigantine, -2 points Richard?
Title: Re: September Predictions Challenge
Post by: Paddy on June 05, 2019, 08:15:05 PM
Anyone tempted to do this again this year?
Title: Re: September Predictions Challenge
Post by: b_lumenkraft on June 05, 2019, 08:17:03 PM
Sure!
Title: Re: September Predictions Challenge
Post by: Brigantine on June 06, 2019, 06:13:37 AM
It's that time of year again!

Now where are our melt pond products, and where was 2012 in comparison?
(since 2012 extent starts dropping like a rock... now)

Is there a poll thread somewhere already? (yes: JAXA (https://forum.arctic-sea-ice.net/index.php/topic,2718.msg0/topicseen.html#new), NSIDC (https://forum.arctic-sea-ice.net/index.php/topic,2719.msg0/topicseen.html#new))
Title: Re: September Predictions Challenge
Post by: Paddy on June 06, 2019, 07:42:19 AM
2019 June prediction:
JAXA: 3.75 to 4.25, medium
NSIDC: 4.25 to 4.75, medium
Title: Re: September Predictions Challenge
Post by: oren on June 06, 2019, 08:51:47 AM
Might wanna refresh the forum's memory as to the rules.
Title: Re: September Predictions Challenge
Post by: Neven on June 06, 2019, 10:05:21 AM
Maybe open a new thread and add '2019' to the title?
Title: Re: September Predictions Challenge
Post by: Paddy on June 07, 2019, 02:29:30 PM
Ok, I might make a new 2019 post for this. Everyone happy if we stick to the same rules for the sake of continuity?
Title: Re: September Predictions Challenge
Post by: Brigantine on June 08, 2019, 10:44:11 AM
Everyone happy if we stick to the same rules for the sake of continuity?
+1 - The 2018 rules are good.
Title: Re: September Predictions Challenge
Post by: b_lumenkraft on June 08, 2019, 10:49:29 AM
Can you copy and paste the old rules to the new thread, so new folks know what it is about?