True
The True formula was not much of a priority this summer, so it is extremely similar to last year's version, with most of the work going into calibrating the existing formula for the new season. Last year's intro to Timeslot Stats should still serve as a good explanation of the current formula, and there may be a bit more when the new one comes out. There were only a couple real changes to the foundation:
1. Local programming lead-in. For the last few years the formula has treated a local programming lead-in as literally a non-factor. I've seen enough of these through RJK's generous contributions this summer that it seems clear that the local lead-in is not really league average; it's lower. So the formula instead opts for a percentage of the PUT that seems to roughly equate with what these numbers have suggested. Big four local lead-ins are 4% of the 8:00 PUT on Monday-Friday; CW and Saturday/Sunday local lead-ins are 2% of the PUT. For weeknight shows, this usually results in a very low-1 lead-in, which is lower than the 1.5 it would've been in the old formula.
2. A slightly harsher formula overall, but dialing up "deflation." My impression last year was that Friday True scores were a little undercounted, so I've tried to dial up the effect of being in a viewership-depressed environment. So far, it seems like the Friday shows are indeed getting a little more credit. I've been hesitant to make this kind of change because I'm worried it may look really ridiculous in super-depressed environments, like weekends in the summer/holiday seasons. But hopefully those will be relatively "low-stakes" misses until we can get a better look.
Though deflation is dialed up, overall the formula has been made a bit harsher than previous versions, in an attempt to put the "league average" for both A18-49 and True into nearly perfect sync. (Last year, the True league average was at least a tenth higher than the A18-49 one.) It might make the scores seem kind of low now, but I would argue they should be low early in the season, and hopefully they won't be overly high late in the season. Anyway, the main point I should make here is that this particular adjustment is not biased for or against any individual show. Everything should still line up the same in a relative sense.
Another note on True: I don't have access to half-hour breakdowns anymore, which is a bad thing for the formula at times. It makes competition and lead-ins less precise than they used to be, so that's an unavoidable downside here compared to previous years. It would also hurt PUT a lot, though I've been using the PUT numbers that RJK generously provides each day so hopefully that part is actually an improvement.
Projected Averages
The Plus Power Rankings (and, soon, the Timeslot Stats tables) track not just a show's current A18-49 average but also what it's projected to be at the end of the season. I have made what I think are some significant upgrades to the formulas that generated these projections last year. There are a lot of other things I anticipate trying out in the future, but these are some positive steps.
Projected Average - The Newbies
Last year, with all new shows we simply assumed that a newbie would duplicate its most recent rating for every remaining week in the order. This was clearly way too simplistic, but it was not really that bad in last year's unique situation since the first Plus Power Rankings came out so late in the season.
This year, it was clear that the newbie projections had to include some expectation of future declines. So I set up a sheet that looked at the opening weeks of each big four regular season newbie in the 14-year A18-49+ era. It resulted in this as the average week-to-week trend across those new shows: the average new show declines 16% in week two, 7.5% in week three, almost 4% in week four, 3% in week five, 2.5% in week six, and 2%ish in weeks seven and eight. (Beyond that, the declines are all less than 1% and some are even up, so at least for now this will assume shows to be steady beyond week eight.)
Even though these numbers stretch across fourteen seasons, the above numbers have been used pretty much as is. The only exception is the most important drop of all, the week two drop, which has been dialed up to -18% based on slightly steeper declines in the second half of the era. All the others were pretty consistent across the era. (Though this year has had a ton of big week three drops, so we may revisit it!) This results in the following expected trajectory for a series that premieres with a 100:
Some numbers that you might find interesting here: this means that a new series is expected to settle almost exactly two-thirds of its premiere rating. And a newbie is expected to average about 72% of its premiere rating over the first 13 episodes and, not pictured above, almost exactly 70% if it gets the back nine.
Whenever a new data point comes in, the formula applies all future expected declines to that point. These expected declines should make for much better projections in the early weeks of a series, though there's still a lot more that can be done. Assuming all these week-to-week drops based on just the last episode makes that last one episode very important, so any kind of odd one-week fluctuation can throw things off. So the "last episode" part of the equation may get replaced with something a little more robust. I also anticipate adding some sort of "DST effect" that depresses some of the later episodes in an order when applicable.
Projected Average - The Returnees
Last year, with returning shows we simply assumed that the current year-to-year trend would continue for the rest of the season. I maintain that this is a pretty good approach, especially with series returning to the same timeslot; it can make a rather decent projection of what will happen spring declines even very early in the season. However, it struggles a lot with almost any... irregularities. If a show has a drastic timeslot move within a season (The Blacklist last season), a massive early-season inflation (Family Guy's The Simpsons crossover last season) or just a downright bizarre trajectory (Empire season one), this formula can spit out some really weird stuff.
On some level, it will be impossible to account for all this stuff in an automatic way. But the new version does one thing that really helps: focus on recent year-to-year trends. Specifically, the projections now use the year-to-year trend for only the last one-third of aired episodes, applying it to the rest of the remaining episodes in the order. This helps the projection "catch up" much more quickly if a show has an inflated premiere or simply heats up or cools down.
The main downside to this is that it reduces the sample size, meaning one blip that produces a weird year-to-year result can really skew things. So sometimes there will be a manual adjustment to use the full season-to-date trend if it looks more reasonable. I'm going to try to use an asterisk (*) on the Plus Power Rankings when these manual adjustments are in play. Hopefully I can eventually come up with a way to make this more of an automatic process in the future.
And there are a handful of shows whose numbers are so irregular that year-to-year trends are useless even just using recent numbers. These shows have been corrected as follows:
Empire and The Blacklist - The Blacklist moved from Monday post-Voice to Thursday, creating year-to-year trends for these fall episodes that shouldn't continue into the second half. As we all know, Empire was just plain ridiculous, growing almost every week throughout season one. Neither of those trajectories will continue, so in these cases the projection uses the newbie model, expecting the -18%/-7.5%/-4%/-3% etc. declines post-premiere and not basing the projections on anything that happened last year.
Once Upon a Time - The problem with Once is that it didn't just have an inflated premiere; it had a heavily inflated entire fall season. It's also steadier post-premiere than the other two shows, so treating it like a new show doesn't work either. What we've done instead is use year-to-year trends not with last year but with two years ago, the steadier 2013-14 season. It's possible this one is too optimistic since Once held up so well in the second half of that season, but it's certainly more reasonable than the alternative.
Let me know if you spot any others!
Again, this seems to be an improvement over last year's version, but there is also a hope to make it even better as we go forward. A clear early goal will be to have more reliable, less swingy projections in the early weeks of a season. This could include factoring in a returnee version of the "expected declines" model used with new shows. It could also include some emphasis on how the show was trending at the end of the previous season.
No comments:
Post a Comment