With the NASCAR Sprint Cup 2012 season in the books, I’ve gone through all my final race rankings and begun evaluating how I did this year compared with all the actual race results. The short answer: Pretty good, with some room for improvement.
Here are the three categories I’ve looked at first:
1. Race Winners
In 2012, I picked 9 winners out of the 36 races. I was right 25% of the time, so if I’d been betting each race, I would’ve needed 3-1 odds on each bet to break even for the year.
I think 36 races is too small a sample size to definitively conclude my handicapping method will pick winners 25% of the time, but it’s probably in the ballpark. I’ve begun looking back at my 2011 and 2010 picks, and that will give me 108 races, a much more substantial data sample size. Initial impression: I think my winner pick rate will end up somewhere from 20%–25%.
2. Top 5 Percentage
I use the stats wizard on Fantasy Racing Cheatsheet (FRC) to crunch loop data each week, and around mid-season I noticed the site’s Experts Picks section. In this section, FRC lists the top 5 picks for each NASCAR race from eight NASCAR handicapping and fantasy experts.
It also lists how successful each expert was picking the top 5 drivers all season. The experts’ 2012 top 5 pick rates ranged from 34%–43%.
This week I calculated how successful I was at picking the top 5 drivers each week. The result: 41.1%.
So, I wasn’t quite as good as FRC’s top two experts, but I was close. Also, it’s worth noting I continually tweaked and improved my handicapping method as the 2012 season progressed, which gives me hope I’ll do even better next year.
3. Average Difference & Standard Deviation
In each race this year, I ranked all 43 drivers; that is, I predicted where each driver would finish in each Sprint Cup race. I’ve since gone back and calculated the difference between each driver prediction with their actual race result. Example: At Homestead, I predicted Jimmie Johnson would finish 1st, but he actually finished 36th, so the difference was 35.
Finally, I calculated the average difference for all 43 drivers in all 36 races. The result: 7.03.
I’ve also calculated the standard deviation (SD) for all my 2012 predictions. Calculating SD involves squaring the differences, adding the results, calculating the mean and then calculating the square root of the mean. The result: 9.60.
This number is higher than I expected, and I’m thinking it’s due to several factors. First, my methodology needs even more improvement. Second, I got absolutely shellacked at the first race of the year, the Daytona 500—I managed only a ginormous 15 SD there. When I subtract that race, my overall SD drops to 9.36.
Third, my feeling is 2012 produced more variation than usual, particularly in the first third or so of the year. Part of that was due to all the track repaves, I think, but there also seemed to be a bit more bad luck than usual on the track this year.
I know, sour grapes, right? Maybe. On my Yahoo! fantasy NASCAR team, I was literally averaging more than one driver catastrophe per race over the first half of the season. I can’t help but think that was unusual.
In the coming weeks, I plan to begin pulling interesting tidbits from my data, such as which tracks I was best at, which I was worst at, etc. If you have anything in particular you want me to look up and post, post a request in the Comments section below.
I just found your site about half way through the season. You are a definitely much better than my other source (me). I won a few weekly contests from a local paper.
I am in a few fantasy leagues. I was wondering if you ever predict the order of finish before the season starts. We have a bidding draft method and predicted order of finish would be very helpful.
I enjoy you column very much and am looking forward to the first race.
Thanks Dave, glad to hear I helped!
I haven’t yet done a pre-season prediction of the 2013 Cup winner. Primarily because that would require so much work up front. But I’ll look into it. Maybe I could crunch data for the early races that hit all the different track types, and from there speculate on who should score well over the entire season.
I’m working on a post now that discusses consensus picks; i.e., taking a bunch of NASCAR experts’ picks for a race and mashing them together to come up with a consensus top 5. My early research suggests a consensus top 5 beats any individual expert’s top 5 over the long run. Possibly including my own, sadly! Stay tuned.
It seemed like early in the year your model ranked Jeff Gordon rather high. His mechanical problems likely hurt you. I am wondering if some sort of measure similar to volatility in stocks might improve the results or provide a measurement of the risk in selecting a particular driver.
Another idea would be to weekly measure the accuracy of your model by publishing an EWMA of the model’s deviation by driver for the previous 3-5 weeks. You’d predict Jeff Gordon to finish 5th, but his EWMA is running -8. So you might be less inclined to pick Jeff that particular week.
Another thought is with the greying of the NASCAR drivers are you analyzing too much data. Is what Jeff Gordon did 10 years ago relevant to his performance today?
For the first part of 2013 I’d consider adding a term to the model of race team. When the COT debuted in 2007, Gibbs and Hendrick won the poles and races. I think you will see the big teams dominate as they can afford the engineering.
For follies sake, since NASCAR is getting grey you could evaluate how the drivers did prior to the COT as I think the Gen 6 car drives more like that car.
Any chance you can do an evaluation on the old (now new) method of qualifying versus the top 35 rule and race results? I wonder if the old method had a stronger correlation. If it looks promising you may want to adjust the weighting of the qualifying results in your model.
The way we play our pool we select a driver from 5 different groups ranked by their points. So I have to put as much effort into picking between Brad, Jimmy, Tony, etc. as I do picking between Cassil, Patrick, and Stremme. So I need the drivers ranked from 1-43 to make all of my picks. I know with Yahoo Fantasy the bottom half doesn’t really matter. So out of 16 people who play in our pool I finished 1st – Segment 1, 12th – Segment 2, and 1st – Segment 3. Made enough to pay my entry fees and to pay for my website hosting.
Bynum, good points! I’ll try to address a few:
Volatility: Yes, it seems higher early each year. Which makes sense because so much changes in the offseason. I like the idea of trying to measure driver volatility/deviation and track it to spot trends. May have to tinker with that. I imagine the new cars plus all the other offseason changes will make the first few months pretty crazy this year. My model also seems to take awhile to catch up to some guys (e.g., Keselowski at the end of last season).
Age of data: I actually look back only 2 years at most for most races. For the races that occur only once a year (e.g., Atlanta, etc.), I have to reach back 4 years in the loop and prelim data. ‘Cause you’re right, data older than that lacks predictive value.
Adding a race team element: Might be a good idea, but not sure how to execute. Johnson and particularly Gordon killed the early COT races if I remember correctly. Dale Jr. was lost. The earlier we can spot trends like that, the better.
Qualifying weight: Yes, I will keep an eye on that. Probably don’t have the time to go crunch the old/now new method, but I might be able to run some how-much-weight-is-best scenarios with the new data as we get it. Like I did here: https://nascarpredict.com/2013/01/20/nascar-handicapping-how-much-is-practice-qualifying-worth/
Interesting that you’re in a pool that values 1-43 accuracy. You’re right, Yahoo! doesn’t really. It’s easy to handicap the top 7 or so, and the bottom 7. The guys in between are the ones that make it tough and require a consistent math-based approach.
Rankings for all the drivers is important for me as well. I am in a league where we pick 7 drivers and those who are 20th and above in points are just as important as 1 thru 5. Keep up the good work and the analysis. It really helps.