"True" Persons Using TV
One of the riskiest yet most necessary things I did in this whole process was come up with a way to adjust PUT calculations for the new methodology Nielsen uses. I started out needing that just to be able to measure all of the 2010-11 season on an apples-to-apples basis, but eventually I wanted the Old Methodology equivalent because I thought the Old Methodology was more useful in correlating with program ratings. (See this old post for more on that.)
I had to tweak it a couple times, but I eventually came up with a multiplier for each of the six half-hours on TV that would convert the New Methodology number to what I guessed would be the Old Methodology number. The only way to really "test" this is to compare it with viewing levels at the same time last year. Viewing is not going to be exactly the same year-to-year, but it should be pretty close.
I'll be able to do this with more precision starting in a couple weeks, because then I'll start getting access to the half-hour breakdowns from the equivalent weeks in 2010-11. At that point I'll be able to do it half-hour-by-half-hour, but for now I'm just going to look at each hour (since most programs are an hour or longer).
Here's the table for PUT levels through the first five weeks on my 2011-12 spreadsheet (pre-premiere week plus the first four weeks of the regular season). "YAgo" is the PUT levels from those same five weeks last year (using Old Methodology), "PUT" is the actual (New Methodology) PUT estimate just dividing all the ratings by all the shares and "TPUT" applies those multipliers from this summer to guess at the Old Methodology numbers.
| FULL WEEK | |||||
| Time | PUT | Yago | Diff | TPUT | Tdiff |
| 8 | 33.5 | 32.6 | -2.7% | 32.5 | +0.4% |
| 9 | 38.2 | 35.4 | -7.4% | 35.6 | -0.8% |
| 10 | 36.9 | 33.7 | -8.5% | 33.8 | -0.3% |
| All | 36.2 | 33.9 | -6.3% | 34.0 | -0.2% |
The "difference" calculations compare the new vs. the old, which is the same way I did it in the summer to create the conversion. This table would seem to indicate that the "True PUT" multipliers are doing a heck of a job making these numbers close to what they were last year. Another way of looking at it is to compare "Diff" with the differences by hour from last year (this is lifted straight from the Methodology Adjustment post from last summer, though I took out the exact figures being used):
| 8:00 | 9:00 | 10:00 | AVG | |
| Diff | -3.1% | -6.6% | -8.3% | -6.1% |
So, the bottom line: it seems that all the work I put into creating a methodology adjustment has, indeed, given me a pretty reasonable approximation of what the Old Methodology PUT would be. The differences are small and could probably be attributed to the margin of error in my PUT estimates, the lack of year-ago half hour breakdowns, or maybe to actual changes in viewing. But it's pretty close, assuming viewing is about even year-to-year; if Nielsen puts out some report saying that it's up or down 5% from last year on an apples-to-apples basis, then... I guess I'm screwed. So let's hope that doesn't happen...!
The Sunday Problem
If there is one clear "problem" with True Strength so far, it's that it seems to be vastly undercounting Sunday programs. The biggest victim of this is The Good Wife, which has had over a +30% Sitch each of its four weeks thus far! And other shows like Desperate Housewives and Family Guy and The Simpsons are basically kept out of contention for the True Top 25 each week because they air on Sundays.
So what is screwing up Sunday TRUEs? I thought before I started this post that it might be over-inflated viewing levels, but it turns out those are (as on most other days) fairly close to "corrected" by the Methodology Adjustment. Here's the Sunday-only table for the last five weeks:
| SUNDAY | |||||
| Time | Yago | PUT | Diff | TPUT | Tdiff |
| 8 | 37.5 | 39.2 | +4.4% | 38.0 | +1.2% |
| 9 | 39.6 | 42.6 | +7.7% | 39.8 | +0.6% |
| 10 | 38.1 | 40.7 | +7.0% | 37.3 | -1.9% |
| All | 38.4 | 40.8 | +6.4% | 38.4 | -0.0% |
Viewing inflations don't seem to be the cause; 8:00 and 9:00 could be "inflated," but not by what anyone could call significant amounts. And across the whole night, it's basically dead even. So these shows aren't getting killed because the viewing levels are "wrong" (or at least I don't think so).
But they are getting killed, in large part, because the viewing levels are really high compared to the rest of the week. This was also something that happened last fall. What's supposed to be able to offset this is heavy competition from Sunday Night Football (which takes a lot of the responsibility for driving up viewing), but instead I stepped in and wiped out a large chunk of that by creating a sports competition adjustment that counted it only as half-competition.
Why did I do this? Two big reasons: 1) As noted in the thing I linked in the above paragraph, a disproportionately large segment of the sports audience wouldn't normally be watching other TV; and 2) most of the Sunday shows I examined dropped huge in the spring, even though football's gone. That led me to believe football wasn't that big of a deal as competition. That adjustment helped make the Sunday shows (like most shows on other nights) closer to even in TRUE throughout the full season. Many of them actually still dropped, but I decided to be conservative. Apparently even that was way too drastic.
How do we know this is a problem? The most obvious example was the Sunday edition of The X Factor, which you might expect to be Truly weaker when on a special night. But not to this extent. Its first three Wednesday telecasts scored a 4.85 -> 4.14 -> 4.49 in TRUE. Then the Sunday edition gets a 2.30 (on a 3.4 A18-49)! That seems rather ridiculous, especially considering then a special Tuesday edition a couple days later turns around and gets a (preliminary) 3.63 on a 3.9 A18-49. 3.63 is low too, but it's not out of the realm of possibility for a special night airing. 2.30 seems like it probably is.
Another sorta problematic thing for the "half-competition" sports multiplier is how entertainment programming that faced the American League Championship Series was basically unfazed in A18-49 ratings. But counting it as "half-competition" means they should be expected to increase drastically with The X Factor out of the mix; the Fox competition would basically be a 4.1 for X on 10/5/11 and then a 1.25 for the ALCS on 10/12/11! Without the half-competition rule it'd still be 4.1 vs. 2.5, but there'd be a much smaller expected increase there (and several shows did slightly increase on 10/12).
Anyway, I still think I had pretty decent reasoning from a few different angles behind reducing the impact of sports as competition, but I guess there just happened to be a bunch of shows on Sunday that Truly get much weaker in the spring. It doesn't really seem like it should be that way for the Fox animation, since those shows typically don't drop huge year-to-year, but I'm not clear at this moment on what to do about it. In a future post, I'll try to remember to take a look at how some of these shows would be doing if Sunday Night Football were counted as full competition (and maybe some other fractions above one-half).
The Friday Problem?
So far, the only day of the week where the "True PUT" looks off by enough to keep an eye on going forward is Friday. Here's the table:
| FRIDAY | |||||
| Time | Yago | PUT | Diff | TPUT | Tdiff |
| 8 | 26.2 | 27.9 | +6.6% | 27.1 | +3.4% |
| 9 | 28.1 | 32.1 | +14.5% | 30.0 | +6.9% |
| 10 | 29.2 | 32.7 | +12.1% | 30.0 | +2.8% |
| All | 27.8 | 30.9 | +11.2% | 29.0 | +4.3% |
It's still early in the season, so it could just be a small sample size thing and I'm not getting too panicked yet. But it's kinda odd that the other six days of True PUT are all within ±1.5% of the year-ago PUT and this day's off by over 4%. Unfortunately the high PUTs do seem to be affecting shows like CSI: NY which have much less negative Sitch than they did at this time last year. Perhaps Friday is an irregularly big DVR-from-earlier-in-the-week evening at this time of year and that's what's driving these New Methodology numbers up, or again, perhaps it's just a weird anomaly. Since Friday rating/shares are lower, that means my estimates are also more likely to be off the mark. Hey, maybe I'm lucky that this is my only off night and the others will get worse as we go on...!
The Saturday Problem?
This was a thing last year too, but frankly I didn't care enough to try to come up with a "fix," and I'm not really even sure it needs to be fixed. But How to Be a Gentleman moved to Saturday and, after posting a 1.83 TRUE in its second week on Thursday, posted just a 0.73. Now that's still a much smaller gap than it is in raw 18-49 (where it went 2.5 -> 0.7) but it's more evidence that shows that get moved to burnoff duty do not even remotely "hold up" even in TRUE. This was also true of burnoffs of yesteryear like Chase, Outlaw and No Ordinary Family. Again, I don't think that needs to be fixed in the TRUE formula; these shows are relatively new, still not Truly settled, and they also lose all of their lead-in support and all of their promotional support (things that are much more important for new shows than for old ones).
I'd maintain that if you put a new Big Bang Theory in the same timeslot it would not fall nearly that much. So it might be nice to come up with some sort of "resiliency" measure to track how much shows are affected in A18-49 when their situations change drastically. Older shows would typically have more "resiliency," newer shows would typically have less.
Anyway, this isn't something I can just pull out of my ass right now, but I'm just noting it as a possibility for next summer or something.
No comments:
Post a Comment