Challenging the Polls: What is Everyone Fighting Over?

There have been some fantastic pieces the last couple of days analyzing the divergent polls and how partisans seem to be choosing whichever data supports their candidate and arguing for its veracity over the contrary. Today, a great many Republicans look at Mitt Romney’s lead in national polls and point to that as the reason for his expected election victory.  Democrats look at the state polling (since that is where the actual electoral votes come from) and say Obama still has the electoral college advantage regardless of any deficit in national polls.  Sean Trende at Real Clear Politics had a great non-partisan column on national polls versus state polls and how looking at each can lead to opposite conclusions:

The RCP Average currently has Mitt Romney up by 0.8 points nationally. He has held this lead fairly consistently ever since the first presidential debate. Given what we know about how individual states typically lean with respect to the popular vote, a Republican enjoying a one-point lead nationally should expect a three-to-four-point lead in Florida, a two-to-three-point lead in Ohio, and a tie in Iowa. Instead we see Romney ahead by roughly one point in Florida, and down by two in Ohio and Iowa.

That would give the Presidency to Mitt Romney.  But if you reverse engineer the state polls to a national turnout you arrive at a different conclusion:

Since the national vote is a collection of state votes, polls of all states should collectively approximate the national vote (since errors should be randomly distributed, they should cancel out). This is done by a simple weighted average…[T]here are several good arguments for favoring the state polling: (1) you have more polls — a much larger collective “n”; (2) you compartmentalize sampling issues — pollsters focused exclusively on Colorado, for example, seem less likely to overlook downscale Latinos than pollsters with a national focus; and (3) the state pollsters were better in 1996 and 2000, two years that the national pollsters missed (although the truly final national pollsters in 2000 got it right, suggesting that perhaps there was a late shift in the race)…After adding the totals up, the results were plain: If the state polls are right, even assuming Romney performs as well as Bush 2004 did in the states without polling, Obama should lead by 1.18 points in the national vote. Given the high collective samples in both the state and national polling, this is almost certainly a statistically significant difference. It’s also a larger margin than all but one of the polls in the national RCP Average presently show.

But national versus state polls isn’t the only debate. Actual poll results versus the data within those same polls may even be the more contentious (and valuable) debate this cycle. Enter Baseball Crank with a fantastic look at modeling election outcomes based on polls versus looking at the actual data that makes up the polls to forecast election winners:

Mathematical models are all the rage these days, but you need to start with the most basic of facts: a model is only as good as the underlying data, and that data comes in two varieties: (1) actual raw data about the current and recent past, and (2) historical evidence from which the future is projected from the raw data, on the assumption that the future will behave like the past.

[A]n argument Michael Lewis makes in his book The Big Short: nearly everybody involved in the mortgage-backed securities market (buy-side, sell-side, ratings agencies, regulators) bought into mathematical models valuing MBS as low-risk based on models whose historical data didn’t go back far enough to capture a collapse in housing prices. And it was precisely such a collapse that destroyed all the assumptions on which the models rested. But the people who saw the collapse coming weren’t people who built better models; they were people who questioned the assumptions in the existing models and figured out how dependent they were on those unquestioned assumptions. Something similar is what I believe is going on today with poll averages and the polling models on which they are based. The 2008 electorate that put Barack Obama in the White House is the 2005 housing market, the Dow 36,000 of politics. And any model that directly or indirectly assumes its continuation in 2012 is – no matter how diligently applied – combining bad raw data with a flawed reading of the historical evidence.

Nate Silver’s much-celebrated model is, like other poll averages, based simply on analyzing the toplines of public polls…My thesis, and that of a good many conservative skeptics of the 538 model, is that these internals are telling an entirely different story than some of the toplines: that Obama is getting clobbered with independent voters, traditionally the largest variable in any election and especially in a presidential election, where both sides will usually have sophisticated, well-funded turnout operations in the field. He’s on track to lose independents by double digits nationally, and the last three candidates to do that were Dukakis, Mondale and Carter in 1980. And he’s not balancing that with any particular crossover advantage (i.e., drawing more crossover Republican voters than Romney is drawing crossover Democratic voters). Similar trends are apparent throughout the state-by-state polls, not in every single poll but in enough of them to show a clear trend all over the battleground states.

If you averaged Obama’s standing in all the internals, you’d capture a profile of a candidate that looks an awful lot like a whole lot of people who have gone down to defeat in the past, and nearly nobody who has won. Under such circumstances, Obama can only win if the electorate features a historically decisive turnout advantage for Democrats – an advantage that none of the historically predictive turnout metrics are seeing, with the sole exception of the poll samples used by some (but not all) pollsters. Thus, Obama’s position in the toplines depends entirely on whether those pollsters are correctly sampling the partisan turnout.

Battlegroundwatch clearly falls into the Baseball Crank category of looking at the internals and taking the conclusions wherever they lead us.  Following this methodology Baseball Crank concludes thusly with which we have no disagreement:

I stand by my view that Obama is losing independent voters decisively, because the national and state polls both support that thesis. I stand by my view that Republican turnout will be up significantly from recent-historic lows in 2008 in the key swing states (Ohio, Wisconsin, Colorado) and nationally, because the post-2008 elections, the party registration data, the early-voting and absentee-ballot numbers, and the Rasmussen and Gallup national party-ID surveys (both of which have solid track records) all point to this conclusion. I stand by my view that no countervailing evidence outside of poll samples shows a similar surge above 2008 levels in Democratic voter turnout, as would be needed to offset Romney’s advantage with independents and increased GOP voter turnout. And I stand by the view that a mechanical reading of polling averages is an inadequate basis to project an event unprecedented in American history: the re-election of a sitting president without a clear-cut victory in the national popular vote. Perhaps, despite the paucity of evidence to the contrary, these assumptions are wrong. But if they are correct, no mathematical model can provide a convincing explanation of how Obama is going to win re-election. He remains toast.


  1. Posted November 2, 2012 at 8:40 am | Permalink | Reply

    I’ve said it several times, but if the internals of the state polls are right then why isn’t anyone talking about the Democrats retaking the House? High Democratic turnout with greater youth and minority shares would indicate that they should be able to swing the House, but no one thinks that. The composition of the House is looking like it will be about the same which could indicate the same type of turnout we had last time – Rs and Ds even. With Romney leading by double digits (NYT and CBS polling indicate Romney could lead indies by at least 16 points once undecideds break) with indies, the Democrats need a massive turnout greater than 2008 to overcome the deficit. So many of these state polls use that model to come to their conclusions.

    I’ve used Florida, my state, as a model. Many of the state polls for us have by D+7 to D+9. But Florida has a Republican governor, all of our elected cabinet positions are held by the GOP, both houses of our state legislature are 2/3 or better GOP, and our House delegation is 3/4 GOP. And in my county, Pinellas (St. Pete/Clearwater is here), all of our county commissioners are Republicans along with all our mayors. Florida flipped because of Hopenchange fever last time, nothing more. We will elect moderate Democrats, but people here are not thrilled with the President. The liberal leaning Tampa Bay Times and Miami Herald have conducted polls that more accurately reflect our state and show Romney up by 7 – and neither of these papers would want that, but they at least have some shred of reality left to themselves.

    • allthingsgeography1
      Posted November 2, 2012 at 9:54 am | Permalink | Reply

      I hadn’t even thought outside the Presidential race bubble until some of you guys started mentioning the House races. Even with gerrymandering, there are plenty of districts capable of swinging Democratic in a good year, but that’s not happening (unless everyone loves the President, but doesn’t want to give the President they love a Democratic House). I agree, such high turnout models for Democrats just aren’t realistic.

      • Posted November 2, 2012 at 11:38 am | Permalink

        I think a lot of Democrats are setting themselves up for disappointment just like any Republican that thought we had a chance in the closing days of 2008. There were several people that tried to say that the Democratic advantage in turnout could not be that heavy, but no one thought the GOP would be able to swing the House or even come close to it that year. That indicated a strong Democratic turnout akin to 2006’s midterm.

        Most analysis I’ve read has said the House probably will not move by more than single digits in either direction with some indicating that a Republican wave could move to give the GOP small double digit gains, but I’ve heard no talk of the Democrats having a chance to pick up 10 or more seats nationally.

        On top of that we have those Democratically weighted polls showing independents breaking pretty hard for Romney. Then we have three polling orgs with very large samples (Pew, Gallup, Rasmussen) indicating a D+0 election or R+1 or slightly greater, I believe all three showed a sizeable Democratic advantage in 2008, with Gallup overstating Democratic strength – which they historically have done by 2-3 points. If Gallup is showing R+1 then there is the possibility that they have overstated Democratic support as they have done in the past and are detecting a wave.

        But I think most everyone would agree if the party turnout is even nationally that Romney will win by a decent margin. Might not be the 7 point margin of 2008 for President Obama, but a comfortable 4-5 point margin that would bring the Electoral College with it. I can’t see anyone winning by that much not getting the EVs to go over the top.

      • allthingsgeography1
        Posted November 2, 2012 at 12:53 pm | Permalink

        Jeff R: ” If Gallup is showing R+1 then there is the possibility that they have overstated Democratic support as they have done in the past and are detecting a wave. ”

        That is actually my greatest concern. A red tsunami below the radar because of the crappy polling (probably being done for horse-race purposes). Gallup is showing R+1 and the 3rd quarter Ras was R+2.6. If you use the 3rd quarter numbers from previous years, Ras has been off by about 2%, mostly overdoing Dem turnout (except 2008 when it was just slightly underrepresented). In other words, I’m concerned about a potential R+3 or even R+4 electorate. That would be stunning, especially for a presidential election year. Election night would quickly turn into a rout early on with Obama not even having a chance (if you watch the Olympics…think Usain Bolt blasting ahead and leaving everyone else in his dust…Election night would be over pretty quickly).

    • Porchlight
      Posted November 2, 2012 at 11:35 am | Permalink | Reply

      Charlie Cook updated his report to show Dems netting 1 seat – and included the caveat that they might actually lose seats. The OH GOP House members all look to be in good shape. Renacci even canceled his ad buys against Betty Sutton because he was that confident of a win. Cook has moved the race from tossup to leans Republican. Is this the same state that’s projected to go for Obama in state poll after poll? I don’t buy it.

      • Posted November 2, 2012 at 11:41 am | Permalink

        That’s my problem. I look at the Ohio House races and see the GOP still leading the House delegation by a 3:1 ration or close to it while looking back at 2008 and seeing the Democrats coming out with 10 seats against the Republicans’ 8. I think the GOP ends up with 11 or 12 of 16 seats in Ohio with districts that are divided to try to be even with population as best as possible. Turnout in Cuyahoga is down 15% thus far for Democrat early voting and Republican turnout is up there. I tend to think that Romney carries Ohio by a decent margin of 3-5 points. 51-48 sounds about right to me. I also have looked at historic state polls in the past and found they so often under represent the GOP turnout.

      • Porchlight
        Posted November 2, 2012 at 11:47 am | Permalink

        We are on the same page. OH historically votes more Republican than the rest of the country (true in 2008). Other swing states are looking about the same re: the House. I’m a little surprised there isn’t more attention given to this. I think Romney takes OH by 2-3 points minimum, maybe more.

  2. Tedley
    Posted November 2, 2012 at 8:52 am | Permalink | Reply

    No question that Baseball Crank is looking at actual numbers with historical significance, and the Dems are hoping folks love bamster more now than they did four years ago without the first supporting fact. Well, I live in Ohio. I know at least 20 people that voted for Bamster last time but not this time. I haven’t met anyone that voted for McKain who’s switched to Bamster. My brother in law is running for office as a Dem – and he’s voting for Mitt. I could go on – but no way Dems will turn out in anywhere near the numbers the Dems need.

  3. Benson
    Posted November 2, 2012 at 8:54 am | Permalink | Reply

    Bottom line that is undeniable in all states.
    2008 vs 2012- stats show Republicans have more registered voters while democrats have lost numbers.
    2008 vs 2012- stats show Republicans are more motivated than democrats
    2008 vs 2012- polls show more Independents are in support of the republican’s candidate rather than the democrat’s
    Yet the top line models don;t represent any of these variables.

    Ironically the latest polls act as though the 2010 elections never happened.

    • Posted November 2, 2012 at 2:07 pm | Permalink | Reply

      If you really want to question the polls you have to look at the demographic assumptions v. realistic assumptions; not party ID. The polls don’t assume party ID, they assume demographics. So if the poll is incorrect it is because of an incorrect assumption.

      The election largely turns on whether the white vote is 72% (Axelrod’s assumption) or closer to 78% (Gallup’s recent conclusion – The center for immigration studies projected 73.4% white vote.

      Since this spectrum seems to the crux of the matter these assumptions should be examined from each poll (rather than party ID).

    • Andrew
      Posted November 3, 2012 at 2:26 am | Permalink | Reply

      Not quite. The top line numbers DO reflect those variables, and the numbers are closer in every state vs 2008. They are just saying that the increase in Repub enthusiasm isn’t enough to put them over the top.

      Strange that you embrace one set of stats, but question the state polling.

  4. Benson
    Posted November 2, 2012 at 9:05 am | Permalink | Reply

    PS: I live in New York and know a ton of split ticket dem voters. Voting across the line blue except for O. They won’t turn this state red but it shows the pulse of the country.

  5. John
    Posted November 2, 2012 at 9:08 am | Permalink | Reply

    There is no good news for the President in today’s unemployment release. Rate ticked up to 7.9%, average hourly wage down, average hours worked per week down, average non-supervisory hours in private sector down (now 33.6 hrs/wk). So this is a mostly negative report and while the Obama camp will trumpet 171,000 jobs created we need to see how many of those are part time (someone who works 1 hour per week counts as employed) where 250,000-300,000 new jobs per month are needed to really lower unemployment and are what would be expected by this point in the recovery.

    Romney has plenty of ammmo to play off in this report. Combined with the negative headlines now emerging of NYC and NJ (which Obama now owns thanks to his PR stunt on Wednesday) and whatever coverage Benghazi gets over the next few days it should be a long weekend for the president.

  6. Bobloblaw
    Posted November 2, 2012 at 9:11 am | Permalink | Reply

    Obama is now up in the RCP.

    • damien
      Posted November 2, 2012 at 9:52 am | Permalink | Reply

      with the god awful plus 5 nat journal poll…of course he is

      • Alex
        Posted November 2, 2012 at 3:32 pm | Permalink

        The ABC/Washington post poll for 10/28-10/31 is also awful with giving D+5.25 and Romney only up by 3 with Independents.

  7. Tedley
    Posted November 2, 2012 at 9:12 am | Permalink | Reply

    And another thing. RCP isn’t including all of the polls – apparently just the ones that will keep the race looking close. For instance, this blog noted a poll that came out yesterday that showed Mitt up 3 in Ohio – along with Josh Mandel 5 ahead of Brown in the senate race. While the poll was from a right leaning group, the sample was D+5. Why hasn’t RCP included those results? I think it’s because they believe they get more hits when the race looks tight – or maybe they got a Gallup note from Axlerod – but either way, all the Dem calculations that rely entirely on RCP averages amplify the effect of foundational sampling errors by excluding data counter to their dreams of competetiveness.

    • Porchlight
      Posted November 2, 2012 at 11:37 am | Permalink | Reply

      My understanding is that Ras does not include commissioned polls. There was a PPP poll a week or so ago showing O +5 in VA that Ras didn’t include either.

  8. Aaron
    Posted November 2, 2012 at 9:15 am | Permalink | Reply

    I live in Florida and I travel all around the state. Romney will win by at least 5. I live in lee county which is a more conservative area we went for McCain by 7 points but when you drive around now Romney yard signs outnumber Obama literally 30 to 1 I am not joking about this. I know several people who voted Obama and are switching to Romney and the few I know lifelong dems that are voting Romney. One of my best friends is African American and will never vote GOP… He is staying home.

    • Posted November 2, 2012 at 9:45 am | Permalink | Reply

      Seeing similar things in Pinellas and Hillsborough.

    • Porchlight
      Posted November 2, 2012 at 11:38 am | Permalink | Reply

      My parents just got down to Boca Raton for the winter and report “signs are overwhelmingly Republican.” In Palm Beach County!

  9. John
    Posted November 2, 2012 at 9:22 am | Permalink | Reply

    Five days out and early voting is running 15% behind 2008 levels in Obama’s Ohio stronghold, Cuyahoga county (Cleveland). I don’t see them making this up in the final 4 days and if daily trends are an indicator (down 40% the last four days) they may even loose more ground. Unlike polls based on assumptions, this is actual hard data and provides a fact-based manifestation of the enthusiasm drop we’ve all been expecting and anecdotal evidence (lack of yard signs, poor showings at rallies, etc.) are indicating.

  10. JP
    Posted November 2, 2012 at 9:44 am | Permalink | Reply


  11. John
    Posted November 2, 2012 at 10:05 am | Permalink | Reply

    Rasmussen National Friday tied = not good 😦

    • Evan3457
      Posted November 2, 2012 at 10:21 am | Permalink | Reply

      Yep, that’s not good, John.

      • Beef
        Posted November 2, 2012 at 11:54 am | Permalink

        It’s the hurricane coverage. It’s absolutely insane, but Sandy might actually save Obama’s candidacy.

        Things this stupid don’t actually happen, do they?

      • Svigor
        Posted November 2, 2012 at 2:18 pm | Permalink


        Romney’s advantage on the economy. This to me is pretty straightforward. Take the recent NPR poll, which was a bipartisan survey conducted by Resurgent Republic and Democracy Corps. It found Obama’s job approval rating on the economy to be underwater, 47-52. The poll also found Mitt Romney to be more trusted on the economy over Obama, 50 to 46 percent.

        Poll after poll, I generally see the same thing. Romney has an edge on the economy. That includes most of the state polls.

        Moreover, this election looks to hinge on the economy, and little else. The recent Fox News poll broke the top issues into four: economic issues (like jobs); social issues (like abortion); national security issues (like terrorism); and fiscal issues (like taxes). To my mind, economic and fiscal issues are one and the same, meaning: 75 percent of respondents willing to pick a top issue picked the economy or fiscal issues.

        I do not know of an election where the electorate was so singularly focused on one set of issues, and the person trusted less on them nevertheless won.

        This makes 2012 different than 2004, when the electorate was focused on four issues, in roughly equal proportion – terrorism, moral values, Iraq, and the economy. Bush dominated the first two, Kerry the second two. This cycle, Team Obama tried to transform the culture into a second front in this electoral war, but they have clearly failed. Per the Fox News poll, just 13 percent of voters list that as their top concern.

        Nobody cares about the storm. Everyone cares about the economy.

    • Posted November 2, 2012 at 10:35 am | Permalink | Reply

      Let’s settle down and stay focused.
      I heard Ras. changed his sample to D+3 or D+4 for this result.

      • Aaron
        Posted November 2, 2012 at 10:38 am | Permalink

        Also it is not actually tied it is Mitt +.9 rounding up for O and down for R is showing the tie, not that I am happy about it….

    • Posted November 2, 2012 at 11:43 am | Permalink | Reply

      Rasmussen went to D+4. Cook and others don’t see any significant changes in the House. D+4 would mean that the Democrats should be able to pick up 10 seats or slightly more, yet no one thinks they will. I think D+0 of 2004 is far more likely and the GOP is far more energized than they were in 2004. Bad economies move the Republicans better than foreign wars do.

      • EpiphoneKnight
        Posted November 2, 2012 at 2:39 pm | Permalink

        Why did they change the sampling?

  12. Fred S
    Posted November 2, 2012 at 10:53 am | Permalink | Reply

    I had heard somewhere that in the week before the election, the incumbent gets a slight bounce, but then the election breaks for the challenger. I am not happy to see this, but he is still well below 50% and that is a very bad place for an incumbent to be 5 days before an election, esp. when the challenger has an energized base who will lead a huge GOTV effort over the last few days of the campaign.
    Don’t be discouraged!

    • Posted November 2, 2012 at 11:45 am | Permalink | Reply

      Usually there is a slight bounce going into the weekend and then a loss of three points or more for the incumbent as we had into the election day in the final polls. Remember, Carter was up 6-8 points in polling at this time and Reagan won by 10 with a third party candidate taking some of his votes. Economic malaise does not work well for the incumbent.

      • EpiphoneKnight
        Posted November 2, 2012 at 2:52 pm | Permalink

        Where are you guys getting this? Didn’t it go the other way in ’04? Of course, that was because the previous polls were intentionally faulty.

  13. Tim
    Posted November 2, 2012 at 10:53 am | Permalink | Reply

    Hey fellow conservatives-
    I must admit rass having race tied is concerning as I wonder if Obama is getting a late surge due to storm etc. this race is such a nail biter down the stretch. Any words of encouragement for this concerned conservative would be appreciated. I live in Seattle and myself and a buddy flew to Colorado last weekend to knock on doors. I have never been so involved in a political campaign as the stakes are so high. This site has been great!

    • jmar
      Posted November 2, 2012 at 11:09 am | Permalink | Reply

      Tim, make sure you don’t miss the forest for the trees. The words of encouragement you seek can be found right under your nose. Read Keith’s entry above that you actually commented under! Kudos for your work in Colorado!

  14. Posted November 2, 2012 at 11:49 am | Permalink | Reply

    I am sitting in Botswana Africa and am watching Romney’s speech from WI. How refreshing. Of course it is not on CNN but thank God for electronic marvels of today, I have a Slingbox connected to my home Directv and watch Fox on my computer. Very powerful speech. On of the best I have heard. He has been talking about 20 minutes and still going strong.

  15. Posted November 2, 2012 at 11:52 am | Permalink | Reply

    The speech is his closing arguments to the American people.

  16. cbr66
    Posted November 2, 2012 at 12:13 pm | Permalink | Reply

    The problem with the Trend article is that Obama is now up in the RCP averages which I believe still includes Gallup with RR +5.

  17. SR
    Posted November 2, 2012 at 1:07 pm | Permalink | Reply

    Guys, calm down. There is nothing to indicate that trends are being reversed. What matters are actual polls and the EV numbers, turnout advantage, enthusiasm etc. favor Repubs. Also, note that Sandy will vanish from the headlines soon…Sandy may also have had an effect on suppressing EV in dem areas. We just dont know. The fundamentals favor Romeny and a couple of incidents here and there are not likely to change anything.

  18. SR
    Posted November 2, 2012 at 1:13 pm | Permalink | Reply

    See NRO article Early voting is down 15% in Dem strongholds. I doubt if they will catch up the next few days.

  19. Buckeye Bob
    Posted November 2, 2012 at 1:26 pm | Permalink | Reply

    On the House races, if it were some wave election, then you would see many House seats move to the “wave” party. In this election, I just think we’re too divided to have any margin greater than 4-5 points, and while that could pull some challengers to a win, it doesn’t seem to be a player here. I live in Ohio, and for almost three months, Obama ads ran on local TV with little Romney exposure. I can buy the polls showing a slight Obama lead here, no matter if not a single Republican House member loses. The gerrymandered districts play a big part of that, but ticket splitting plays a big role too. Ohio is going to be close, and I do think (perhaps wishful) the natural Republican tilt on the Presidential level will swing Ohio to Romney on election day, but I would not be shocked if Ohio went for Obama. I am hopeful the enthusiam I see and hear at the Romney events translate to an election day victory, but the Democrats have a strong GOTV. I think Mark Halperin of Time pegged it best with three possible outcomes: 1) a close Obama win 2) a close Romney win or 3)a Romney “blowout” of 4-5 points. If the next Gallup poll shows Romney up by at least 3-4 points (post Sandy), I think Romney wins both the popular and electoral votes. I also think Romney is wise to hit PA, WI, and Iowa in case the early Obama ad onslaught has put Ohio out of reach.

  20. Svigor
    Posted November 2, 2012 at 2:12 pm | Permalink | Reply

    The thing that’s really striking to me right now is how people keep saying “if you look at the polls,” or “if you look at the poll averages,” etc. Um no. What they mean is, “if you look only at the top lines of the state polls,” or “if you look only at the averages of the top lines of the polls,” etc.

    If the polls are the thing, why isn’t anyone reading them?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: