There’s a lot of consternation in my inbox and Twitter feed tonight about the generic congressional ballot.
Gallup’s generic ballot poll has Republicans up 15 point among likely voters, or at least their traditional model does; their higher-turnout model has Republicans up 10 instead.
Fox News, whose models haven’t had a Republican lean in the past but have something of one this year, has Republicans up 13. CNN has them up 10. Rasmussen, up 9. YouGov, plus 8.
The CBS/New York Times poll has them up 6, as does the survey from Pew Research, as does an NBC/Wall Street Journal poll that asks voters which party they’d prefer to see control Congress (not technically a generic ballot poll, but the questions usually produce similar results.)
The Politico/Battleground poll has Republicans up 5. The ABC News/Washington Post poll has them 4 points ahead. Bloomberg has them up 3. Marist shows a tie. Newsweek, somehow, actually has Democrats ahead 3 points among likely voters.
Some of these polls have Democrats gaining a point or three, and some show just the opposite. Some show the enthusiasm gap widening; others have it narrowing.
You might think all of this has to do with different likely voter models, since each company has its own ‘special recipe’ to separate likely voters from unlikely ones. But that only explains some of the difference. Many of these pollsters have also published generic ballot results among registered voters, which should theoretically be more consistent with one another. But there is a diversity of results there too, running from Republicans plus-6 (CNN) to Democrats plus-6 (Marist and Newsweek). The standard deviation on registered voter polls released within the past 15 days (4.1 points) is almost exactly the same as the standard deviation among likely voter polls (4.2 points). The disagreements are just as great.
The fact is that there’s not really any way to say who’s right — not until Tuesday, at least. There aren’t that many different ways to run a poll, or to construct a likely voter model; most of the choices that pollsters make are relatively trivial most of the time. But these seemingly trivial methodological choices are producing a really wide spread of results this year.
Political scientists have formulas to translate the results of the generic ballot to an estimate of the number of seats that the Republicans will gain. I’m not much of a fan of these formulas for a number of reasons, but let’s see what they say.
The most widely-used formula says that a Republican advantage of 7 points among likely voters on the generic ballot — which is about what the average is — would translate to a Republican gain of 53 seats. But if you plug the Gallup likely voter results into the model instead (the model was originally designed around Gallup data, in fact), it projects a G.O.P. gain of 77 seats. If you use the Newsweek poll, on the other hand, it shows Republicans gaining just 23 seats.
Our model is a lot more sophisticated than that. It does look at the generic ballot, but it doesn’t necessarily assume that it is right. It also looks at local polls in each congressional district, expert ratings, fundraising data — the whole kit and caboodle. Unlike the political science models, it formulates an estimate of the result in each individual congressional district, and not just the overall seat count.
But it tells you basically the same thing. Tonight, our forecast shows Republicans gaining 53 seats — the same as in recent days, and exactly the same answer you get if you plug the generic ballot average into the simple formula. Our model also thinks the spread of potential outcomes is exceptionally wide: its 95 percent confidence interval runs from a 23-seat Republican gain to an 81-seat one.
Now, this is actually something of a coincidence; our model doesn’t think the confidence interval is wide because there is disagreement in the generic ballot polls, but rather for other reasons.
Nevertheless, if a simple formula and a complex one tell you the same thing — yeah, Tuesday’s probably going to be a really good night for Republicans, but we really don’t have a very good idea of exactly how good — it’s probably time to embrace that conclusion. This is a really strange election, or at least one that pollsters are having an awfully difficult time getting a handle on. To claim you can predict Republican gains within a range of 5 or 10 seats isn’t science — it’s superstition.
*-*
So, I can’t tell you exactly what will happen on Tuesday. What I can do is show you some pretty pictures; maybe that will soothe the nerves a bit.
Our model calculates a generic ballot average (which it now has at Republicans +6.8 points — but keep in mind that this is just one of the inputs that the model uses) through a process we adopted from our Presidential forecasts.
Basically, our model looks at how polls are changing from week to week: has the ABC News/Washington Post likely voter poll moved toward Democrats or against them since the last time it was published? How about the Rasmussen poll? This process, we think, is as a little bit more robust than simpler averaging methods, which are subject to a lot of noise based on the mix of pollsters that happen to be in the field at any given time.
After coming up with the week-by-week estimates, we then draw a smooth trendline — what’s called a LOESS curve — to further filter out the noise.
There are a few more wrinkles than that — for example, we adjust the results of registered voter polls to make them comparable to likely voter ones — but that is the basic idea. (Veteran FiveThirtyEight readers will recognize this technique as being essentially identical to the one we used to create our Presidential ‘supertracker’ chart in 2008.)
Here is how our method shows the progress of the generic ballot since January, 2009. (The blue diamonds represent the weekly observations; the red curve is the smoothed trendline.)
Amidst all of chaos, there is some sort of a signal — at least insofar as the trend goes, which is what this technique is trying to divine. The Democrats’ generic ballot position deteriorated extremely rapidly within the first six months of the Obama administration; they started out being 10 points ahead on the generic ballot on a likely voter basis (which is about where they finished in 2008), but were essentially tied with Republicans by July 2009.
Since then, the decline has been slow but quite steady, with the Democrats having lost about a point or so every other month, except for a period this spring when some good economic numbers were coming in and the generic ballot was quite steady for a few months.
Let’s focus now on the trend since the start of the year.
Not a whole heck of a lot has happened; the Democrats started the year about 3 points behind and now they’re about 7 points behind. Some weeks, they’ve happened to get 3 or 4 generic ballot polls that were comparatively better for them, and other weeks, an especially bad group of polls. But most of the damage was actually done in 2009 and there’s only been a little bit of further erosion this year. It’s simply becoming more apparent now because (i) more pollsters (essentially all of them) have switched over to likely voter models, which few were using at the outset and (ii) we’re starting to get a better idea of how an upside-down generic ballot figure for Democrats translates into individual states and districts (and the news isn’t pretty for them).
What we don’t really see, though, is any indication of dramatic, last-minute change. The parties are in pretty much the same predicament that they’ve been in for months. Our first House forecast, which was published around Labor Day, had Republicans gaining 47 seats; we now have them gaining 53, which is a rather trivial difference. The Senate picture has sort of gotten better for Democrats, but only because it now looks like they’ll hold exactly the set of seats the need to hold to keep Republicans to “just” a 6-9 seat gain; some individual races have gotten better for Democrats, while about as many have gotten worse.
Now, this is not meant to contradict that earlier conclusion that there’s a lot of uncertainty about what will happen on Tuesday night. What I’m suggesting, rather, is that just as the conditions promising at least a pretty bad Democratic year (the tepid economy, coupled with the perception that Democratic efforts to fix it have been an expensive failure) have been with us for some time, the uncertainty has been with us all along as well, like a virus with an exceptionally long latency period; on Tuesday, we’ll see which of the polls it has infected.