|Tornado Outlook Contest|
2013-06-02: Tim Samaras, his son Paul Samaras, and his chase partner Carl Young
were killed in the El Reno tornado on Friday. Tim was an engineer who knew that we need better
field data in order to understand why tornadoes can be so destructive. He was responsible for
the first-ever barometer readings inside a powerful tornado (i.e., the F4 in Manchester, SD,
2003-06-24), which showed the largest pressure drop ever recorded on Earth (to this day). These
data are crucial for tornado theory. The well-known "funnel" shape of a tornado is caused by a
pressure deficit that relaxes in the direction of the flow. Conventional meteorology can't
explain this, because it shouldn't be possible in a standard suction vortex. So this is the
least understood aspect of all, and it figures centrally in my proof of another force in
tornadoes, with the confirmation coming from Tim's instrumented data. Tim had a reputation for
being more prudent than most storm chasers, so we shouldn't dismiss his death as just the
expected outcome for an adrenalin junky, which he was not. His last tweet on Friday was typical
of his attitude, "Storms now initiating south of Watonga along triple point. Dangerous day ahead
for OK -- stay weather savvy!" But it's a simple fact that if you chase tornadoes for 30 years,
sooner or later something will go wrong. Things aren't so risky for the tornado sensationalists
who are out there just to get videos that they can sell to the TV stations -- they can get
decent video from 1/2 mile away. But due to the unpredictable paths of tornadoes, Tim learned
that he had to get within a couple hundred yards to position instruments that would get run over
by the tornadoes. He couldn't sell the data to any TV station, but Tim kept doing this work
anyway, because he knew that it would help save lives. In the end, he gave his own life so that
others may survive. Rest in peace, Tim, and thanks.
Tim Samaras (1958~2013)
2011-04-20: The scoring algorithm has been altered, to eliminate the penalties for missed events. This was because we were finding that some outlooks really had the right idea, but were getting low scores because of a couple of events that occurred on the other side of the country. So now there are no penalties — you get points for the lead time of the outlook, for the percentage of events inside or nearby the polygon, and for the number of events per 100 square miles in your outlook. Visually, it looks like the outlooks that had the right idea (even if they missed a few) are getting the highest scores now. All existing outlooks have been re-scored, and the contestant averages have been recalculated.
2011-04-12: Some minor inaccuracies in the map pick-points, and in the plotting of reports, have been cleaned up. Now all of the plotted points are accurate to the nearest pixel.
2010-05-02: The scoring algorithm now has a new factor, named % nearby. This adds points for near misses that are not recognized by the % in poly factor. Before, it was an all-or-nothing situation, where you got full credit for any events inside the outlook poly, and no credit for events outside of it. With the % nearby factor, you'll get partial credit for events just outside. The degree of credit is on a sliding scale, based on the distance from the poly, going from full credit down to 0 at a distance of 100 miles away. So an event 50 miles outside of the poly will be worth half as much as an event inside it. This will help some of the small but very accurate outlooks score higher, as they should, even though there might have been a lot of events just outside of the poly.
Also, I found and fixed a bug that was resulting in reports getting duplicated in the database, which was throwing the scores off. All of the outlooks have been re-scored, using the new algorithm, and without any duplicate event reports.
2010-04-25: We have a new high score! With a bold 78-hour outlook, nje310 has stepped out in front. Racking up 199 points by issuing an outlook so far in advance, with a relatively small poly that still included 56% of the tornadoes, and where the missed tornadoes were not far away, nje310's 2010-04-25 outlook is now the one to beat!
2010-03-18: Just as a reminder for anybody who doesn't know, if you enjoy this contest, you might also enjoy the Virtual Storm Chase hosted at WxChat.com, which will be running April 20 ~ June 20 this year. The two contests are not mutually exclusive, as the Tornado Outlook Contest tests your ability to predict regional conditions one or more days in advance by looking at the model maps, while the Virtual Storm Chase tests your ability to read the satelite and radar info for that day, to predict where exactly the events might occur.
2010-03-15: I fixed the problem with tornado reports showing up on the wrong maps. All of the existing outlooks have been re-scored, now with the correct reports. Sorry for the confusion.
The purpose of this contest is to allow people with an interest in severe weather to try their luck at predicting, as far in advance, where the events will occur during any given period. The object is to define the smallest area that misses the fewest events.
Please note that this is a contest — it's a game! So don't get bent out of shape if things don't go your way, for whatever reason. The administrator will put forth a normal amount of effort to make sure that the site runs reliably, that the data are backed up regularly, and that the scoring algorithm is as fair as possible with a reasonable amount of programming. But we ain't hookin' up no AI algorithms here. (If we did, then nobody would understand the rules, and everybody would still get bent out of shape!) So play the game if you enjoy it, and only play it for the enjoyment. Maybe we'll all learn something from this, but developing sophisticated pattern analysis algorithms is way beyond the scope of what we're doing here. This is simply a place for enthusiasts to have a healthy, good-natured competition, to see who can guess the best. ;)
It has been suggested that the NWS outlooks be imported, making NWS a competitor in this contest. But that's not going to happen, and for a very simple reason. This is a weather contest. We're just competing to see who can guess the best. NWS, on the other hand, issues outlooks as part of their commitment to public safety. The criteria that they use in deciding where and when to assign probabilities of severe weather are very different from the simple scoring algorithm employed on this site. So it would be an apples-and-oranges error to compare what we're doing here to what NWS does, and that sort of misrepresentation wouldn't serve any legitimate purpose.
So, if an NWS forecaster decides to compete here in his/her spare time, that's fine. We'll probably get out butts kicked, but that's OK, if we learn something every now and again. ;) But we're not competing against NWS as it functions in its official capacity.
Outlooks will be "valid" for the target time +/- 6 hours. NWS issues outlooks that are valid for an entire day. Of course, NWS has the time to figure out how the conditions are going to change throughout that entire period, but most of us weather freaks don't have that kind of time. So this contest is based on specific target times, and only events occurring within a 12-hour period straddling the target time will matter. So if your target time is 00:00 Z, then look at model maps for that time, and issue an outlook for what you think will be happening within 6 hours of that time. It won't be held against you that events occurred in other places at other times during the day, that you didn't have the time to study.
Outlooks can be issued up to 6 days in advance, but no less than 12 hours in advance. The further in advance the outlook is issued, the more points you score. Lead-time scoring will be done in 6 hour increments, since the model maps are only issued every 6 hours. So for example, all of the "24-hour" outlooks will get the same lead-time bonus, even though this will include all outlooks issued 24~29 hours in advance.
The maps issued by other contestants will not become visible until 12 hours before the valid time, at which time no more submissions will be allowed for that time. So no piggy-backing on somebody else's outlooks. :)
To get a high score, you have to define the smallest area that misses the fewest events during the valid period. You'll start off with points for the number of hours in advance that you issued the outlook. Then, when the reports come in, you'll get a bonus for the percentage of events that occurred within your outlook polygon. There is also a bonus for near misses. While you get full credit for reports inside the polygon, you get diminishing credit for events up to 100 miles from the poly. An event just outside the poly is almost as good as an event inside; an event 50 miles from the poly is worth 50% of an event inside; and an event 99 miles outside of the poly will net you almost no credit. You'll also get a bonus for how many events occurred within how small of a polygon you drew.
If you submit an outlook with no polygon, it will get marked as a "less than 2% chance" outlook. But then the only points you'll get will be for the lead time of the outlook, and you won't be able to compete like that. So if you don't think that there will be any events during the valid period, don't submit an outlook.
The way the scoring factors are weighted will be open to public debate, so if you think a great outlook got too low of a score, or a bad outlook got too high of a score, post a message on the associated forum. Then the scoring algorithm will be tweaked, and all outlooks will get re-scored with the new algorithm.
As the scoring algorithm changes, it might lead to a different way of issuing outlooks, and you might want to delete old outlooks that are dragging down your running score. But you cannot selectively delete them. In other words, you can't just delete the outlooks that sucked — you can only delete outlooks one at a time, starting from the oldest. So if the scoring algorithm changes, and you change your outlook strategy accordingly, and/or as you get better at issuing outlooks, you can scrub your earlier outlooks, so they won't be held against you.
Over time, we'll see who can guess the best, but of course, we'll also be reminded that severe weather events are near-random in nature, and on any given day, anything can happen.