Small sample, wrong sample

Presidents almost always enjoy a post-State of the Union bump in the polls, and I suspect last night’s speech, which was one of Bush’s better addresses, will be worth at least a few points.

That said, if you hear about the latest CNN/USA Today/Gallup poll, take the results with heaping grains of salt.

President Bush’s State of the Union address raised support for his policies on health care and Social Security among people who watched the speech, according to a CNN/USA Today/Gallup poll conducted Wednesday night.

The percentage of respondents who said the president’s proposals in those areas will help the country rose 15 points from when the same question was asked of the same people in the two days before the speech.

In the post-speech sample, 70 percent of respondents said Bush’s policies on health care were positive, while 66 percent approved of the president’s plan for Social Security. Bush showed almost as much improvement on Iraq, with 78 percent of respondents saying U.S. policy there is heading in the right direction, a 12 percentage point increase over pre-speech polling. Overall, 77 percent of respondents said Bush is taking the country in the right direction after the speech compared to 67 percent beforehand.

Sounds like a homerun for Bush, right? Like all the numbers that had been drifting downwards since Election Day were finally back on track for the White House?

Well, not entirely.

The poll not only included a very small sample size, but the partisan breakdown was ridiculous.

Of the 485 people surveyed, 52 percent identified themselves as Republicans, 25 percent as Democrats and 22 percent as independents.

Naturally, if you conduct a poll in which most of the respondents are Republicans, the numbers will show encouraging numbers for Bush. In reality, nowhere near 52% of the population is Republican, so these results — which are getting far too much play — are highly dubious.

The biggest fear is that the poll will become self-fulfilling. Americans will see on CNN that everyone loved the speech and that Bush’s agenda is receiving broad praise, but they won’t see anything about the skewed sample. A lot of people who may not have even heard the speech will assume it was a hit because CNN is telling them that everyone else says so.