Yesterday afternoon, a Gallup tracking poll showed Barack Obama leading John McCain nationally by eight points among registered voters, 48% to 40%. A couple of hours later, a Gallup/USA Today poll showed McCain leading Obama among likely voters by four, 49% to 45%.
Both from Gallup, both taken over a similar time period, both national, and both released on the same afternoon.
Indeed, nothing seems to match up right now. The two major tracking polls (Gallup and Rasmussen) offer different results. The two major aggregate averages (RCP and Pollster.com) also offer different results. Ezra wisely concluded, “[T]here are too many polls showing too much noise and we’re too far from the VP selections, conventions, debates, and actual voting for them to mean anything. All these polls were a big topic of discussion in the MSNBC green room a few hours ago but so far as I can tell, none of them matter, or tell us anything on their lonesome.”
I agree with this wholeheartedly. And yet, I can’t help but wonder how, exactly, Gallup/USAT found McCain ahead among likely voters, especially when the same poll found Obama ahead among registered voters. I’m sensitive about appearances here, and I’ll be careful to avoid reaching the point at which I say, “I don’t like the results of the poll, so there must be something wrong with the poll.” That way madness lies.
That said, there really is something wrong with this poll.
Republican John McCain gained ground in a new USA TODAY/Gallup Poll that found Democrat Barack Obama’s highly publicized foreign trip has not broadened confidence in his ability to be commander of the U.S. military.
The poll, taken Friday through Sunday, showed a surge since last month in likely Republican voters and suggested Obama’s trip may have helped energize voters who favor McCain.
Obama had a five-point lead among likely voters a month ago; now he’s behind by four points. That should be the first clue there’s something odd going on with the poll.
As it turns out, the problem lies with the definition of “likely voter.”
By showing a lead for McCain that contradicted Gallup’s own independent tracking results, the poll prompted Emory University political scientist Alan Abramowitz to ask how Gallup and USA Today selected the “likely voters” who purportedly now favor the Arizona Republican. While the answer to that methodological question (and several others) remains a bit murky, USA Today’s full writeup in Tuesday’s dead-tree edition offers a few more details — and one other nugget of information that should drive skepticism of their results through the roof.
As for how “likely voters” were identified, USA Today reports that respondents were asked “how much thought they had given the election, how often they voted in the past and whether they plan to vote this fall.” Fair enough. But the very next sentence raises even more questions about whether USA Today’s effort is actually a snapshot of the electorate, as its website claims, or enters the realm of forward-looking hypothesizing. Buried in the ninth paragraph of USA Today’s own writeup, they reveal that “McCain’s gains came because there was an even number of likely voters from each party. Last month, the Democrats had an 11-point edge.”
Abramowitz says this contradiction is the equivalent of polling malpractice. “It is simply not plausible that there would be an 11-point swing in party ID among likely voters or that there is now an even split in the likely electorate between Republicans and Democrats,” he wrote in an email to the Huffington Post.
And on MSNBC this morning, Chuck Todd sounded off on the “likely voter” results:
“[Frank] Newport, editor of Gallup, a good pollster … even has a warning in USA Today this morning that says don’t believe the likely voter, follow the registered voter model. They report it anyway. … But the point is, is that this is — a good polling organization like Gallup, okay, these guys are in control of the gold standard of polling. They can’t figure it out. I think this is a warning to everybody that polling this year is very difficult. … Everything should be taken with a grain of salt.”
First, that’s good advice. Second, it doesn’t excuse the fact that the Gallup/USAT poll featured on the front page of this morning’s USA Today is deeply flawed.