Last week, to the delight of Republicans, National Journal released its latest rankings of U.S. senators, from most liberal to most conservative. The publication, which strives to be scrupulously non-partisan, found that Barack Obama was the chamber’s #1 lib — a development which caused a considerable stir in the media.
For all the reasons we talked about, the methodology for the rankings is a bit of a joke. The good news is, National Journal Editor Charles Green has heard the criticisms, and took the time to do a lengthy Q&A on the magazine’s site to respond to concerns. The bad news is, Green’s pitch isn’t exactly persuasive.
Most of Green’s responses were pretty boilerplate — National Journal was cautious, it didn’t know Obama would come out on top, the rankings aren’t intended to influence the election, yada yada yada. But Green’s take on a similar controversy in 2004 were especially illustrative.
Q: Didn’t you go through the same situation four years ago?
Green: Yes. In 2004, National Journal rated Democratic presidential nominee John Kerry as the most liberal senator in 2003. The rating quickly became a talking point in the campaign, with President Bush, Vice President Cheney, and other Republicans using it to attack Kerry. […]
Q: Have you made any changes in the vote rating system since then?
Green: We made one change. We decided that in order for a member of Congress to receive a composite rating, he or she needed to vote often enough to qualify for scores in each of the three issue categories-economic policy, social policy, and foreign policy-that we measure. In Kerry’s case, he didn’t vote often enough in 2003 to merit scores in the social-policy and foreign-affairs categories. His overall ranking was based on his score in the economic category.
Got that? Senators’ scores are based on three categories — economic policy, social policy, and foreign policy. Kerry didn’t have enough votes in two of the categories, but National Journal gave him a score anyway, announced that he was the chamber’s #1 liberal, and gave Republicans a cudgel they used every day for months. Four years later, National Journal is now willing to concede that the rating was a sham — a flawed result based on a flawed methodology.
Four years later, though, we should take the Obama rating seriously because, well, National Journal says so.
This is all terribly silly. Green argues that some senators weren’t given scores if they missed too many votes. Obama missed a full third of the 99 votes used for the ratings, but that wasn’t enough to disqualify him from the rankings. Why not? Because National Journal’s arbitrary standards, known only to the publication’s editors, say so.
National Journal argues that Obama took the “liberal” approach on 65 out of 66 key votes. There were other senators who cast more liberal votes on more liberal bills, but that doesn’t matter. Why? Because National Journal’s arbitrary standards, known only to the publication’s editors, say so.
When considering votes, the labels themselves are arbitrary.
The National Journal, if you read their report, does a fairly good job of identifying the key votes of the session. But where the National Journal falls down is in assigning “conservative” or “liberal” value to the “yes” or “no” votes.
Why is, for example, requiring 100% inspections of shipping containers for national security threats a “liberal” position? How is establishing English as the official language a “conservative” position? Is a position “conservative” or “liberal” for cutting subsidies to private business to offer student loans? This study says it is “liberal” to do so, although that position is practically of no difference from Ron Paul’s!
No, while there is a vague liberal vs. conservative element to this study, what it really tells us is how partisan the officeholder was (partisan being equal to voting with one’s Party). All that I can conclude from this study is that Obama is a “blue state” Democrat who hasn’t altered his votes to appear more “conservative” in anticipation of a national election.
Any rankings system that insists, right off the bat, that Joe Biden is more liberal than Russ Feingold and Bernie Sanders is automatically suspect, but the closer one looks at the process, the more flawed it appears. The fact that National Journal is now willing to acknowledge that its John Kerry ranking in 2004 was bogus is hardly reassuring — if the magazine was wrong then, perhaps it’s not quite reliable now?
I still think Brian Beutler’s observation is the right one: “[T]his is philistinism masquerading as social science — it’s the U.S. News College Guide of Washington politics. Journalists ought to understand that. And those of conscience ought to ignore it, or lay it bare, but certainly not feed into it.”