For many years, Spotted Ratings has been a TV ratings site that doesn't do that in any kind of formulaic way. We've talked about these things a lot, but there's never been a "model" or an "index" or anything like you can find at some of the other guys.
That's about to change. This fall, I hope you will embark with me on Renewology, a fledgling attempt to statistically model and predict renewal and cancellation decisions for broadcast scripted series.
With almost every big project I've undertaken, I've tried to bring something new and valuable to the table. I could've done a cheap imitation of the renew/cancel indices you see elsewhere, but this is the first time I felt I could add a lot to the process and not lose my mind doing it. Renewology is the culmination of a lot of different improvements in a variety of areas, and some of these steps have been in development around here for years.
This has taken a lot of work, but it isn't a massive shift in direction; in fact, most of the Renewology DNA comes from the True Power Rankings that came before it. This is just taking many of the ideas that went into the analysis of those lineups and approaching them more formally. Yes, some of this will be similar to what you've seen from what you've seen from renew/cancel predictions at other sites. But here are the big things that will make Renewology unique:
Predicting Future Ratings.
If you've followed this blog for a long time, you may have picked up on one of my pet peeves in TV ratings coverage. One or two times per year, there's a series that premieres at fairly marginal levels, and the TV media is content to label it a decent premiere, as if that series premiere result is just some normal rating that will be duplicated in the future. This year's best example was You, Me and the Apocalypse, but I've been ranting about this for years. (By the way, the end result of the rant in the link: Work It got pulled after just one more episode!)
The reality is that series premiere ratings are almost never duplicated. About 95 percent of new broadcast series drop in week two. The average post-premiere drop just from week one to week two is over 15%, and it's more like a third or more when looking at where the show settles in the end. So when a show premieres at a so-called "decent" level, that actually means its chances of survival are rather low. It has little room to drop, and most newbies drop a lot.
It's long past time to stop treating these earliest weeks on face value. Sure, it's always going to be impossible to perfectly predict what's going to happen, but you're missing out on a lot of predictive value by expecting zero drop, rather than baking in some kind of expected future drop. The Renewology model takes a show's most recent ratings and compares them with historical data about how ratings develop over the course of a season. We'll be assigning a renewal percentage number from the very first rating released, based on where the numbers are expected to go. I hope this will make predictions in the very early weeks more useful than what you see elsewhere, especially from the pundits who say, "that would be a decent number if it holds."
Daily Forecasts.
I'd like to promise that this process will wildly improve on the accuracy of renew/cancel predictions. But the safer bet for improvement is in the delivery of predictions. Renewology will be a more real-time and visual experience than this has ever been before. I won't be saying things like, "New Girl upticks! See if its prediction changes six days from now!!!" Instead, Renewology will put out fresh probabilities every day, alongside each new rating for a broadcast scripted series. I have always dreamed of being able to pin down how much the weekly swings really matter as they're happening, and this will be an attempt at doing that.
With each day a scripted series airs, there will be a table (on the daily ratings post) that shows each show's new renewal odds, as well as how it has changed from week to week. It'll be a bit after the daily post goes up, but the table should go up somewhere in the vicinity of 30-90 minutes after the daily post on most days.
This will be a visual experience, as well. You will be able to see a chart on each show's SpotVault that tracks its renewal odds from episode to episode, and I expect more graphics to develop from there. One thing I do like from the "other guys" is the broad categorizations, with one smiley face representing "certain to be cancelled" and five representing "certain to be renewed." This will take a similar approach, in the form of color-coded zones for 0-20%, 20-40%, 40-60%, 60-80% and 80-100%.
Every Step is Objective.
The Renewology formula takes a Live + Same Day adults 18-49 Nielsen rating and turns it into an actual renewal percentage number, from 0% to 100%, and each step of that process is contained within the formula. Other renew/cancel predictors are quantitative only in creating the index numbers, but then the actual predictions don't really match the numbers; many shows will have a kinder prediction than shows with lower ratings, due to all these additional rules from outside the realm of ratings. In Renewology, even that conversion from rating to prediction is objective... sometimes to a fault.
With each show, we will produce a projected ratings average and a projected "target" ratings average. Each show's Renewology percentage will be strictly based on how close the show is to the target. There are several things that go into the show and target projections. As I said above, projections of future ratings come from looking at how past series have developed over the course of the season. Ratings are adjusted for timeslot difficulty, using our long-running True formula. And the target averages are derived from a few different network-related factors, which we can quantify by looking at other shows on the network. I will be going over these things in much more detail in a future post, but the point is all these things can be measured from within the ratings landscape. To borrow FiveThirtyEight terminology, this is probably more of a "ratings-only" forecast rather than a "ratings-plus" one.
This should not be seen as infallible just because it's math. I hope this formula will capture some ratings uncertainty better than the subjective naked eye, but our brains will be able to process some information more effectively. In order to make this thing perfectly objective, Renewology leaves out some important things that I couldn't quantify due to lack of time and/or ability. In a few cases, this will almost certainly make Renewology a lot worse than other people doing this. (I can pretty much guarantee we will be way too bearish on Madam Secretary.) But I truly believe you can get a good sense on almost all shows with just these few factors.
It's impossible to say how well this thing will do on its maiden voyage. I expect it will be far from perfect, both for reasons I know of now and probably some that I can't anticipate. Or maybe this will just be the year that the TV media is finally right about same-day ratings not mattering. So I hope you won't take it too seriously. It's not the gospel. Let's have fun with it.
The next week may not be so much fun, as we'll be taking a rather dry, deep dive on how Renewology works. One or two of these will show up later this week and the other one or two at the beginning of next week:
What Renewology Will Look Like This Fall. I'm going to start with the relatively fun stuff, which is some info on how the Renewology numbers will be used and disseminated on a daily and weekly basis. Plus, what's ahead for the site's other features from past seasons.
Renewology: The Big Steps. I'm not giving the entire formula away by any means, but I'm going to try to be pretty transparent about the steps behind the process of getting these numbers. This will be a fairly long and detailed description of the thinking behind each of the four big steps in the Renewology process, as well as some thoughts on what is not part of the process.
2016-17 Changes to the True Formula. The timeslot-based adjustments in the True formula are a significant part of Renewology. I worked on the formula more than usual this year and I think it's better than ever. So I'll have a few deets on what I changed this year.
No comments:
Post a Comment