Combatting the Polling Problem

In addition to fixing a clearly broken brand and attracting more voters, Republicans need to address getting blind-sided by their own internal polling:

In the weeks before Election Day, both Republicans and Democrats were nervous about their poll numbers. Both sides of the aisle have smart pollsters, they reasoned, so how could the numbers that Democrats were seeing diverge so sharply from the numbers the Republicans were seeing? Deep down, I wrote at the time, both parties secretly worried that their side was missing the boat.

What went wrong:

“Everyone thought the election was going to be close. How did [Republicans] not know we were going to get our ass kicked?” lamented Rob Jesmer, head of the National Republican Senatorial Committee. “I don’t understand how we didn’t know. That’s the part that’s most puzzling and frustrating and embarrassing.”

The underlying causes of the errant numbers are the assumptions that the pollsters made about the nature of the electorate. Most pollsters believed the electorate would look something like the voters who turned out in 2008, just with slightly lower numbers of African-Americans, younger people, and Hispanics heading to the polls.

But exit polls actually showed a much more diverse electorate than the one forecast. Black turnout stayed consistent with 2008, Hispanic turnout was up, and younger voters made up a higher percentage of the electorate than they had four years ago. White voters made up 72 percent of the electorate, according to the exits, down 2 points from 2008 and a continuation of the two-decade long decline in their share of the electorate.

That meant that even though Mitt Romney scored 59 percent of the white vote — a higher percentage than George W. Bush won in 2000 and 2004, higher than Ronald Reagan in 1980 and matching George H.W. Bush’s 1988 score, when he won 426 electoral votes in 40 states — it wasn’t enough to overcome the 80 percent support that Obama scored among nonwhite voters.


Pollsters should fix voter screens, used to weed out of their samples irregular voters who aren’t likely to vote. Including only likely voters often leads to a more Republican-heavy sample. But in an era of fine-tuned turnout machines and get-out-the-vote drives, even those irregular voters are likely to show up. Polling all registered voters, rather than those most likely to make it to the polls, would at least give Republicans an idea of the worst-case scenario.

Pollsters should also control more for age, gender, and race than for party identification. One prominent party pollster pointed to a late survey conducted for Indiana Republican Richard Mourdock that showed him leading Democrat Joe Donnelly by 2 points. That survey, conducted by McLaughlin & Associates, showed that 56 percent of Indiana’s electorate would be over age 55. Exit polls revealed that number to be vastly overstated; only 43 percent of the electorate was over 50.

The party-identification question gets to the heart of another misperception that pollsters make. Tell almost anyone that Romney would have won self-identified independent voters by 5 points and logic would dictate that Romney would win a clear victory. But Democratic pollsters say that metric is flawed, and that many Republicans remain so disaffected by their own party that they refuse to identify with it. Instead, some say that pollsters should look at self-described ideology, rather than party identification. Indeed, Obama beat Romney among the 41 percent of voters who call themselves moderate by 15 points.

Pollsters also recognize that Americans’ daily routines are changing, something that has an impact on their surveys. About one-third of all households do not have a landline, according to the National Health Interview Survey, meaning that a significant swath of the electorate is available to pollsters only by cell phone. The percentage of younger Americans who don’t have a landline is almost double that. Pollsters who don’t include a sufficient number of cell-phone respondents in their surveys risk missing out on younger voters — voters most likely to back Democrats, thus skewing polls to the right.


  1. exe
    Posted November 13, 2012 at 10:42 am | Permalink | Reply

    To me, the real question here is whether Mitt would have taken a different path with accurate polling. Skip trips to MN, PA and focus even more on FL, OH, VA, CO.

    • Kevin Paradine
      Posted November 13, 2012 at 11:11 am | Permalink | Reply

      I don’t think more campaigning would have fixed anything. Romney’s campaign was already pointless in September. It was only the debate that gave any (false) hope.

    • MikeN
      Posted November 13, 2012 at 1:10 pm | Permalink | Reply

      I think Romney was better off going to Pennsylvania and Minnesota and Michigan. It is clear the Obama campaign beat him everywhere they went up with attack ads. Romney needed to expand the map into New Jersey, Maine, Oregon, Washington, California, Illinois.

  2. PeterJ
    Posted November 13, 2012 at 11:17 am | Permalink | Reply

    I don’t remember which article it was not, but one of the post-election ones on polling mentioned that a *looser* likely voter screen produced the more accurate results.

    As for fixing internal republican campaign polls, I guess they might just as well hire PPP. Or just assume the RCP average of public polls is correct and act on that basis. Just don’t waste money next cycle on republican branded polling firms connected with the consultant class that seek to feed at the campaign trough regardless of producing accurate results or not.

  3. Posted November 13, 2012 at 11:20 am | Permalink | Reply

    All these are good responses but there is a more basic one, stop relying on conspiracy theories. Poll after poll showed Democrats winning, it wasn’t just one or two. The GOP response was to laugh them off and denounce as troll anyone who mentioned them. And this happened for months. And then we had the whole war on Nate Silver!

    It’s one thing to rally your base, quite another to lie to yourself. Stop listening to Rush and his ilk, or if you will listen realize they are just entertainers and have no informational value.

    And it wasn’t just the polls. Most economy based models also had Obama winning. This idea that has taken over the GOP, that the economy is in shambles, is also another myth. No it’s not great but we are recovering. And employment is growing, slowly.

    Both sides live in a bubble to some extent but the conservative bubble has gotten out of hand.

    • Prescient11
      Posted November 13, 2012 at 11:41 am | Permalink | Reply

      What time tested “economic model” showed Obama winning?

      Quite the contrary, the only one used for any duration, i.e., the Colorado model, showed just the opposite.

    • Kevin Paradine
      Posted November 13, 2012 at 11:43 am | Permalink | Reply

      Conservatism is disqualifying in the blue states. Liberalism is not disqualifying in red states. That’s the essential problem. The bubbles are there because people like echo chambers, no one wants to be told they are wrong.

    • Prescient11
      Posted November 13, 2012 at 11:45 am | Permalink | Reply

      And one more thing:

      Obama has/had:

      1) $1T in stimulus
      2) 0% interest rates
      3) Fed printing trillions of dollars
      4) No real inflation

      And unemployment is still in the shitter.

      Compare that to Reagan’s record when he took office:

      1) Fed rates at 14-15%
      2) No money printing
      3) Oil crisis
      4) UE skyrocketing above 10%
      5) Double digit inflation for 2 years.

      With Reagan’s policies the country came roaring back and we had the longest period of economic prosperity ever recorded. He OVERCAME all that adversity.

      Obama has HAS HAD ALL THIS HELP mentioned above, and this fucking failure can’t even get us going again. Not only that, $6T in debt has now been added to our children and theirs. This is the scariest part of all of this.

      LIBERALS are idiots and their economic “policies” are incredibly destructive. Here endeth the lesson.

      • Posted November 13, 2012 at 1:02 pm | Permalink

        Maybe topic for another post but Reagan had little to nothing to do with what happened in the economy. It’s a common misconception to blame presidents when economy does well or praise them when it does badly. And no, we did not have the longest period of economic prosperity ever recorded. Not even close. Do I need to go through the numbers?

        Really, why do you guys rely on so many myths?

      • Prescient11
        Posted November 13, 2012 at 2:18 pm | Permalink

        Yes Peter. You need to go through the numbers to try and back up such lies.

        It depends on the president. Did Clinton deserve credit for balancing the budget and welfare reform.

        Obviously not. His credit is that he wasn’t stupid and thus didn’t fight it.

        As far as consecutive figures of GDP growth are concerned, yes Peter, after Reagan took over we had the most consecutive string of positive GDP numbers since they started recording such numbers.

        One final thing, anyone who claims to be a fiscal conservative and yet vote for Obama is a liar, plain and simple.

      • Prescient11
        Posted November 13, 2012 at 2:19 pm | Permalink

        I cannot wait to have this debate re Reagan!!!!

        Bring it on.

      • ET4
        Posted November 13, 2012 at 2:49 pm | Permalink

        “One final thing, anyone who claims to be a fiscal conservative and yet vote for Obama is a liar, plain and simple.”


      • Posted November 13, 2012 at 3:50 pm | Permalink

        OK Prescient.

        Let’s start with the basics. Did you know the economy and the stock market do better under democrats than under republicans? Here’s one link but there are many more. You can Google this.

        So this idea that “liberal” economics does worse than “conservative” economics has no empirical support.

      • Prescient11
        Posted November 13, 2012 at 4:06 pm | Permalink

        Peter, no no no no no no no no no.

        I gave you a specific fact. Consecutive positive GDP growth was longest under Reagan since it has been recorded.

        You said I was wrong.

        Now answer this question, was I right or wrong???

        Then we can move to next steps. But stock market has nothing to do with the economy, and we can delve into that once we establish that Reagan was literally the best.

      • Posted November 13, 2012 at 4:50 pm | Permalink

        Wait, you said “longest period of economic prosperity ever recorded”. That is not the same as “Consecutive positive GDP growth was longest under Reagan”. For example if you have 10 quarters of growing at 2% that is worse than 9 quarters growing at 6% and 1 quarter negative growth.

        Here’s a good table:

        You can see there was nothing special about Reagan’s GDP numbers. You didn’t know that average growth under carter was better than under Reagan?

      • Prescient11
        Posted November 13, 2012 at 5:36 pm | Permalink


        First, we have to understand that both Obama and Reagan were dealing with a significant recession when they came into office. So you cannot look at numbers like “average GDP growth” etc. as apples to apples.

        In other words, the shitstorm sandwich that was Carter was just about to manifest itself when Reagan took over. So Reagan’s shitty first couple of years can be attributed to that fucking idiot Carter.

        And it took Carter a while to fuck up what he had inherited from Ford.


        Just answer a simple factual point, and we can move on to your other points.

        (P.S.: here’s a clue, it was!!!)

      • Posted November 13, 2012 at 6:07 pm | Permalink

        Whoaa, so you want me to take out the bad GDP numbers from Reagan and only keep the good ones? Come on! What kind of an analysis is that? You are OK doing that for Obama’s jobs record as well? If so Obama’s record is now among the best. And why can’t then we blame Carter’s bad numbers on Nixon then?

        Remember I don’t buy that presidents are responsible for the good times (Clinton, Reagan) or the bad times (Bush, Carter). But if you want to compare then we need to compare the whole period not just pick and choose the numbers you want.

        The reality is that Carter had nothing to do with the 80-82 recession and Reagan had nothing to do with the subsequent recovery. Volcker and the Fed were responsible for both. Carter inherited high inflation and, to his credit, appointed Volcker who drove the economy to the ground with higher rates but who managed to lower inflation. Once that was done Volcker eased up on the rates which led to a new economic recovery and “Morning in America”. But there was no real boom, the good numbers for 1984 were mostly due to the economy catching up after the recession. If you look at Reagan’s economic growth across the economic cycle (ie you measure average growth between periods of similar unemployment levels, since unemployment is a measure of capacity constraints) you see that Reagan’s growth record was the same as we had in the post 1973 era and much lower than the 1945-1973 era, which was the real economic boom in the US.

        Clinton did have a boom of sorts, greater than Reagan’s, but I wouldn’t claim that was due to Clinton’s policies.

        Obama’s recovery has been much weaker than prior recoveries but that seems to be the result of the different type of recession, in Obama’s case one from too much private sector debt.

        Finally, much has been made of the new era of deregulation in the 1980s. Who started that? Carter who deregulated airlines, trucking, rail and telecommunications.

      • Derclaw86
        Posted November 13, 2012 at 7:09 pm | Permalink

        Carter also gave us the “Community Redevelopment Act”, which was later expanded by Clinton to give us the implosions of “Fannie Mae”, “Freddie Mac” and the whole sub-prime housing disaster.

      • Posted November 13, 2012 at 7:23 pm | Permalink


        That’s another GOP myth. CRA had nothing to do with the current crisis.

        It boggles the mind the amount of stuff you guys on the right believe that is simply not so.

      • Derclaw86
        Posted November 13, 2012 at 10:02 pm | Permalink

        CRA was a program to “encourage” banks to lend to people in the inner city that would otherwise not be credit-worthy. It wasn’t until the 1990’s that problems occurred. The Clinton Administration not only steeped up the act’s enforcement (The Stick), it also provided lucrative incentives (The Carrot). The article you cite glosses over the fact that complying with the CRA was not only lawful, but it was also very profitable for well-connected Democratic officials. Liberal big-shots like Jamie Gorelick, James Johnson, Franklin Raines and others brought Fannie Mae into this game. And they all made multi-million dollar bonuses doing so. The more bogus mortgages they supported, the bigger their bonus checks. All this was started in the 1990’s. It is true that the CRA was not the entire cause of the Mortgage crisis, nor did I assert that. However, it was an important building block that allowed the next generation of liberal politicians to expand on, leading to disastrous consequences. Many conservatives are suspicious of big government programs not because they were meant to help people, far from it. In fact many such programs start out relatively harmless. It’s the fact that they can later be used as building blocks for future politicians to build their schemes that frightens many of us.

      • ET4
        Posted November 13, 2012 at 11:02 pm | Permalink

        Actually, a lot of comments left by people in the same post Peter linked make the same point you make Derclaw. I guess if Peter finds an opinion piece that fits his worldview, he will quickly cite it as “demolishment” of the Republican “myths.”

      • Posted November 14, 2012 at 1:10 am | Permalink

        ET, Derclaw,

        If you bother to read the article (or any other similar article, there are many that say the same thing) you will learn that most of the institutions involved in the subprime fiasco were not even subject to the CRA, such as mortgage service cos.

        It is also false that Fannie Mae or Freddie Mac are the cause of the crisis, they actually came late to the party.

        You guys really have created an alternative reality for yourselves.

      • Derclaw86
        Posted November 14, 2012 at 2:20 pm | Permalink

        It was in 1999 that the Clinton Administration dramatically eased credit requirements for low-income, risky, inner city borrowers. The Fannie Mae administrators I mentioned above were right in the middle of the process. Just because your article forgets to mention this pesky little detail, doesn’t mean it didn’t happen. Even the New York Times wrote an editorial on being alarmed at this process.

    • Posted November 13, 2012 at 12:38 pm | Permalink | Reply

      Right on Peter. There’s no polling crisis. Most polls nailed it at the end. Pew’s methodology was flawless, and called the final PV almost exactly. When I posted it here I was called a troll.

      But any notion that we were undone by false optimism is largely ludicrous. You want the troops to stay positive. I assume that’s why this website was spinning everything in the end when Keith had to know better.

    • C'ville
      Posted November 13, 2012 at 8:49 pm | Permalink | Reply

      Peter that is not accurate. Until the last national polls, Romney was winning the national vote by 3-5 points which then closed to a tie and undecideds still left went 2/3 to Barry to get him to 50.5 or whatever it was. The national polls were accurate when taken. As for state polls, some were good and some were bad (Romney did not lose Ohio by 5 as some had).

      • Posted November 14, 2012 at 1:12 am | Permalink

        No, this is false. Are we making up the past of only a few weeks ago? Romney was never ahead by 3-5 points, that was only in the Gallup poll which we now now was terrible. PPP and IBD/TIPP. which had some of the best results in this campaign, had Obama winning weeks ahead.

      • ET4
        Posted November 14, 2012 at 8:14 pm | Permalink

        Pew had Romney +4 after the first debate. Stop lying Peter.

  4. Prescient11
    Posted November 13, 2012 at 12:57 pm | Permalink | Reply

    Ray, my only comment on that is with regard to Ras’s party ID. That had never OVERESTIMATED R support, and that is what is deeply troubling.

    That was a huge miss. Plus, all the fraud vote in Philly and OH, of course.

    But the failure to capture the right party ID was a big big issue.

    • PeterJ
      Posted November 13, 2012 at 1:53 pm | Permalink | Reply

      I think Rasmussen does try to get it right and will adapt their methodology next time. Which would mean even more cell phone inputs and looser likely voter screens, instead of adjusting for demographics via census data. It would be nice to see Scott Rasmussen give a post-election critique of his own methods and if anyone sees same please post a link.

  5. Adam R
    Posted November 13, 2012 at 1:53 pm | Permalink | Reply

    Haven’t run the numbers to test this theory, but I’ve always thought that pollsters with a Democratic leaning house effect would tend to do very well in Democratic years, and pollsters with a Republican leaning house effect would tend to do very well in Republican years, but neither would be especially good when the other party has a big year. This makes intuitive sense. A pollster with a Democratic-leaning house effect will do better in an election when their assumptions and pre-conceived thoughts on turnout actually come to bear. For example, Mason-Dixon, a pollster with a pretty big GOP house effect, was arguably the most accurate pollster around in the big GOP years of 2002 and 2004, but have been well below average in the Democratic years since (they had Romney winning Florida convincingly). Everyone is lauding PPP these days, but are they really better than everyone else, or were they just dead-on the last two cycles because they were Democratic years? I suspect PPP would probably be below average in a big GOP year. Thoughts on this theory? The true test of a pollster’s accuracy is whether they can nail the polls correctly in *both* big Democratic years and big Republican years. Has anyone been able to do that?

  6. Derclaw86
    Posted November 13, 2012 at 3:18 pm | Permalink | Reply

    I don’t think our polls were necessarily wrong during the month of October. However, I believe that hurricane Christie (oops, I mean Sandy) really screwed things up. It caused Obama to look presidential and totally undermined Romney’s argument that Obama was small and petty. This showed up in two ways. First, it caused independent and moderate women to abandon Romney and switch back to Obama. Many earlier October polls showed Romney closing the gender gap. After the hurricane, this was gone. Second, the 2-3% of voters that were undecided broke for Obama. Again, I think this was hurricane related.

    • Prescient11
      Posted November 13, 2012 at 3:34 pm | Permalink | Reply

      Adam R and Derclaw,

      Excellent points. Honestly, I think Obama expected to lose. Or really feared it. We know RR expected to win.

      The fuckups of this election and of this campaign are epic. So much is at stake to let incompetent fucking idiots run these things.

      Everyone thinks Rove is some kind of genius. We need to beat him and get his OH strategy, and then keep him in the background. GW won by 3k votes in FL, not really a massive landslide eh Rove.

      • MikeN
        Posted November 14, 2012 at 11:30 am | Permalink

        The Obama’s body language after third debate was one of losing, and the campaign appearances were all in blue states. With their position, why weren’t they campaigning in Virginia and Florida? I think they ended up overperforming their expectations slightly.

    • Adam R
      Posted November 13, 2012 at 4:02 pm | Permalink | Reply

      Well, if we take the exit polls at face value, I think a pretty strong case can be made that Hurricane Sandy cost Mitt Romney the popular vote. Whether it cost him the election is more speculative, because last I checked, they didn’t break out the Sandy question at the individual state level. But the exit polls posted at FoxNews and CNN say that 15% of voters considered the Sandy response the single most important factor in their vote, and they break roughly 73-26 in favor of Obama. Even if we presuppose that those voters were already leaning towards Obama, by let’s say a 55-45 margin, that’s still a 2.7 point boost to Obama’s popular vote share. That tracks pretty closely with the movement in the national polls over the final week, and is also corroborated by the decline in Romney’s independent support from around 15% in pre-Sandy polls to 5% on election day, which would also translate to roughly a 3-point movement overall. If not for Sandy, I think it’s plausible Obama doesn’t get much better than 48.0-48.5% in the popular vote.

      • PeterJ
        Posted November 13, 2012 at 6:40 pm | Permalink

        Sandy probably was one factor among many, and I think it is impossible to isolate its effect from the others. Also the type of voters who are swayed by something like that were likely just one emotion away from switching anyway. And whatever effect Sandy had, it would not have been as severe absent other problems in GOTV, etc.

    • Posted November 13, 2012 at 7:29 pm | Permalink | Reply

      Another myth. Poll based models back in June were predicting Obama would get 332 EC votes.

      • PeterJ
        Posted November 13, 2012 at 9:19 pm | Permalink

        Another myth. The state of the campaign was static and the situation in June exactly predicted election day.

      • Derclaw86
        Posted November 13, 2012 at 11:45 pm | Permalink

        The Pew Poll detected definite movement between their pre-hurricane and post-hurricane polls. The race moved from 47-47 (poll ending 10/28), to 50-47 Obama (poll ending 11/3). Pew even states in their 11/3 poll that, “Obama’s handling of the storm’s aftermath may have contributed to his improved showing. Fully 69% of all likely voters approve of the way Obama is handling the storm’s impact. …more important, so do 63% of swing voters.” It also goes on to mention dramatic improvement in Obama’s performance among women and moderate voters. Sandy clearly played a role in this election. We conservatives are at fault for not being wiling to face up to it prior to the election, liberals are now at fault for denying its influence after it. As far as the votamatic, 538 blog, and other prediction quants are concerned, we never took issue with the particular measurements they were making. Our contention was with the tools they were using to do the measurments, polling data that we believed to be flawed. Nate Silver and others even admitted that their analysis could be wrong if they were using biased data. That was our whole contention. When the data proved to be correct, their analysis also was confirmed. However, their conclusions are hardly remarkable. The RCP final average predicted 49 out of 50 correct states, erring only with razor-thin Florida. This simple average is not very different from their elaborate quantitative modeling. Nor was it in 2008. If we were wiling to believe the RCP polls, we would have probably also arrived at the same conclusion as everyone else.

    • Dave
      Posted November 14, 2012 at 12:40 am | Permalink | Reply

      During times of crisis, american rally around the president. And Christie’s .love fest didn’t help. The irony of a global warming event defeating an anti global warming party is just beyond ironic.

      • Dave
        Posted November 14, 2012 at 12:41 am | Permalink

        That’s what gave Obama the boost. Just like when 9-11 happened or Katrina, Bush would have won reelection without any campaigning.

  7. Prescient11
    Posted November 13, 2012 at 10:17 pm | Permalink | Reply

    Ah, you see Peter, you can’t answer a fucking question. That’s the problem with you libs.

    You claim to be a “fiscal conservative”. If you voted for obama, you are either: 1) a liar; or 2) unhinged.

    Back to Reagan, you spout bullshit but cannot answer a simple question. After Reagan’s policies went into effect, well before the end of his first term, the US started experiencing postiive GDP growth and this continued to be a golden era for the longest consecutive GDP positive numbers that lasted beyond his presidency.

    THAT IS A FACT, and you can’t even concede it.

    Bush’s tax raise, urged by stupid/liar democrats, took us into recession again.

    And let’s go back to the polls, shall we. The lib polls had obama winning obama by 4 and 6. Exactly what did he win FL by again? They were not pinpoint accurate as you claim.

    • Posted November 14, 2012 at 1:15 am | Permalink | Reply

      Creating an alternative reality again? That didn’t work well for you guys last time.

      Just like you guys made up that polls were not accurate so you have made up that Reagan had some great record. You “unskewed” Reagan’s record by simply ignoring the recession years! You guys are hilarious!

    • Prescient11
      Posted November 14, 2012 at 11:10 am | Permalink | Reply

      You’re failure to answer simple factual questions exposes your charade.

  8. Prescient11
    Posted November 14, 2012 at 11:10 am | Permalink | Reply


    • C'ville
      Posted November 14, 2012 at 10:05 pm | Permalink | Reply

      To Peter Palco, yes Romney was up nationally 2 weeks out 3-5 in Gallup and Rassmussen and it closed on final day to 48-48 and 49-48, each with 3-4 undecideds that broke Barry’s way (2/3 accirding to exits) to give him a slight win. The polls were right when taken and then same polls dhowed barry clising. Also, that old saw about markets “performing historically better under d’s” is slightly uninformed – you do realize that GDP includes government spending and isn’t all private sector growth? Big government spending can make that number look better but isn’t really economic growth. Also, running printing presses can make more dollars and result in a higher stock market that doesn’t really evidence new wealth. Raw stock market is up but dollar isn’t as valuable and company earnings sort of stink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: