Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

All of Nate Silver's State-Level Polling Predictions Proved True

timothy posted about a year and a half ago | from the panned-out-you-might-say dept.

Government 576

kkleiner writes "For the last few months, the political pundit class has been at war with NYT/FiveThirtyEight blogger Nate Silver. Joe Scarborough of MSNBC called him a "joke," while an op-ed in the LA Times accused him of running a "numbers racket." But last night, Silver triumphed: every one of his state-level presidential predictions proved true. "

cancel ×

576 comments

Sorry! There are no comments related to the filter you selected.

Math (5, Interesting)

Anonymous Coward | about a year and a half ago | (#41922747)

http://www.xkcd.org/1131/

Re:Math (5, Insightful)

YodasEvilTwin (2014446) | about a year and a half ago | (#41922835)

This. People need to learn that statistics and polling are sciences. Like all sciences they are inexact, with a margin for error; but the chances of the poll averages being wrong in this case were incredibly small.

Re:Math (5, Insightful)

alphatel (1450715) | about a year and a half ago | (#41923013)

This. People need to learn that statistics and polling are sciences. Like all sciences they are inexact, with a margin for error; but the chances of the poll averages being wrong in this case were incredibly small.

Or This. Pundits and Statisticians are about as far apart as Republicans and Climate Scientists.

Re:Math (5, Funny)

coldfarnorth (799174) | about a year and a half ago | (#41923211)

Or as a friend of mine said:

Nate Silver is to talking-head punditry what the National Weather Service is to "Hurricanes are caused by teh gays."

Re:Math (5, Insightful)

jhoegl (638955) | about a year and a half ago | (#41923231)

Pundits were creating the illusion of close races to drive up viewing.
They could care less if it adversely affected a vote or increased voter turnout.
The fact is they manipulate the information for their own personal gain. This time was it not only Fox News and MSNBC, but CNN.
This opens up an avenue for truth, which sheds light on what they are doing. Media outlets are calling him that because they are upset he is telling the actual truth and not making it cloudy to increase revenue.
Colbert Report made fun of this when he had him on the night before the election.

Re:Math (4, Insightful)

MightyMartian (840721) | about a year and a half ago | (#41923311)

Indeed. There was an illusion of a close race to sell advertising. People love drama, and having a contest where the media reported "Obama's got it, Romney's cause is hopeless" would not have had the sexy urgency necessary to cash in on.

We are seeing the OJ simpson freeway chase kind of reporting being applied to elections.

Re:Math (2, Insightful)

explosivejared (1186049) | about a year and a half ago | (#41923095)

; but the chances of the poll averages being wrong in this case were incredibly small.

I'm not sure that's exactly knowable. Sure, the numbers are way better than contradictory pundit guts, but for instance, we had no way of knowing if a "Bradley Effect" would have been in play. Response rates for polling firms consistently came in below 10%. Polling is getting harder and harder in an age where fewer people have landlines and polling cell phones is restricted. As of now, state polls are good guides. They will be right up until they aren't, and then the science will change.

I'm not saying that the probability of systematic error is large, just unknowable. It was a perfectly reasonable and scientific position for a Republican to say "Romney's chances are equal to the probability of error in the polls, and I hope that probability is large."

Re:Math (4, Insightful)

sjames (1099) | about a year and a half ago | (#41923185)

We can know the odds to a high precision. Just like I can know the odds of the flip of a weighted coin. That doesn't mean I can't be wrong when I guess my next coin toss, it just means I have a fair idea how likely I am to be wrong.

Re:Math (5, Interesting)

mcgrew (92797) | about a year and a half ago | (#41923215)

This topic isn't nearly as wild as the election the year I was born. From USA Today's In '52, huge computer called Univac changed election night: [usatoday.com]

In a few hours on Nov. 4, 1952, Univac altered politics, changed the world's perception of computers and upended the tech industry's status quo. Along the way, it embarrassed CBS long before Dan Rather could do that all by himself.

Computers were the stuff of science fiction and wide-eyed articles about "electric brains." Few people had actually seen one. Only a handful had been built, among them the first computer, ENIAC, created by J. Presper Eckert and John Mauchly at the University of Pennsylvania in the 1940s.

In summer 1952, a Remington Rand executive approached CBS News chief Sig Mickelson and said the Univac might be able to plot early election-night returns against past voting patterns and spit out a predicted winner. Mickelson and anchor Walter Cronkite thought the claim was a load of baloney but figured it would at least be entertaining to try it on the air.

On election night, the 16,000-pound Univac remained at its home in Philadelphia. In the TV studio, CBS set up a fake computer -- a panel embedded with blinking Christmas lights and a teletype machine. Cronkite sat next to it. Correspondent Charles Collingwood and a camera crew set up in front of the real Univac.

By 8:30 p.m. ET -- long before news organizations of the era knew national election outcomes -- Univac spit out a startling prediction. It said Eisenhower would get 438 electoral votes to Stevenson's 93 -- a landslide victory. Because every poll had said the race would be tight, CBS didn't believe the computer and refused to air the prediction.

Under pressure, Woodbury rejigged the algorithms. Univac then gave Eisenhower 8-to-7 odds over Stevenson. At 9:15 p.m., Cronkite reported that on the air. But Woodbury kept working and found he'd made a mistake. He ran the numbers again and got the original results -- an Eisenhower landslide.

Late that night, as actual results came in, CBS realized Univac had been right. Embarrassed, Collingwood came back on the air and confessed to millions of viewers that Univac had predicted the results hours earlier.

In fact, the official count ended up being 442 electoral votes for Eisenhower and 89 for Stevenson. Univac had been off by less than 1%. It had missed the popular vote results by only 3%. Considering that the Univac had 5,000 vacuum tubes that did 1,000 calculations per second, that's pretty impressive. A musical Hallmark card has more computing power.

That doesn't take away from Silver's math, though, considering that the polls all had Obama and Romney neck and neck and Obama won by a huge margin. It seems Woodbury did a far better job with an incredibly primitive computer than the modern polsters' statisticians did with today's high tech machines.

Re:Math (4, Informative)

halfEvilTech (1171369) | about a year and a half ago | (#41923337)

All things considered he was fairly accurate on the 2008 election as well. I think he got one state wrong if that as well.

So as far as I know he is 2 for 2.

Re:Math (5, Insightful)

MightyMartian (840721) | about a year and a half ago | (#41923341)

Part of the problem was an obsession with national level polling. Silver was analyzing the races per state, which is the only legitimate way to analyze it in an electoral college system. National polling is at best only an extremely crude indicator, and to my mind, in most modern presidential elections is likely useless.

Re:Math (-1)

Anonymous Coward | about a year and a half ago | (#41923229)

It's not quite as simple as that. In past years we have had good state polls and bad national polls (like this year), good national polls and bad state polls (like the conservatives hoped for this year) and of course both of the other variations. Election year polling is still both an art and a science driven by rapidly changing variables that aren't directly measurable. Until we see who of the electorate actually comes out and votes, nobody knows the outcome for sure.
 
This is another nice feather in Silver's cap, but it's incorrect to categorize his intelligent guesses and approximations as a certainty.

Good for him (-1)

Anonymous Coward | about a year and a half ago | (#41922753)

Too bad the electoral system in the USA is a joke and doesn't represent the vote of the people.

Re:Good for him (5, Interesting)

Anonymous Coward | about a year and a half ago | (#41922803)

Another possibility is that the Founding Fathers understood just what kind of dumbshits most of "the people" are, and built in some safeguards to allow civilization to survive the inevitable popular vote to shut it down.

Re:Good for him (2, Interesting)

Anonymous Coward | about a year and a half ago | (#41923055)

Another possibility is that the Founding Fathers understood just what kind of dumbshits most of "the people" are, and built in some safeguards to allow civilization to survive the inevitable popular vote to shut it down.

Not only that, but the restrictions on who could vote (basically white landowners) wasn't there because of some inherent prejudice that suggested women, free blacks or other demographics were lesser. The restrictions where there because white landowners had a very high chance of having a solid education. It morphed over time (both in society and in the "history" books) into something more racist / misogynistic. The only thing worse than pure "majority rules" is "uneducated majority rules".

Re:Good for him (4, Insightful)

FilmedInNoir (1392323) | about a year and a half ago | (#41922889)

Too bad the electoral system in the USA is a joke and doesn't represent the vote of the people.

So you wanted Al Gore to be president in 2000?

Re:Good for him (5, Insightful)

Ziggitz (2637281) | about a year and a half ago | (#41923073)

As opposed to the alternative? Jesus Fucking Christ, yes.

Re:Good for him (0)

Anonymous Coward | about a year and a half ago | (#41923123)

Too bad the electoral system in the USA is a joke and doesn't represent the vote of the people.

So you wanted Al Gore to be president in 2000?

Well he wouldn't have been any worse than the village idiot from Texas. Well okay, born in Connecticut, prep school in Massachusetts, Yale University, and then he went on to be the village idiot in Crawford, Texas.

Re:Good for him (0)

Anonymous Coward | about a year and a half ago | (#41923223)

Yes. 1000x more than Dubya.

Re:Good for him (2)

AK Marc (707885) | about a year and a half ago | (#41923441)

As bad as he is, he couldn't have been worse than Bush Jr.

Re:Good for him (4, Informative)

Latentius (2557506) | about a year and a half ago | (#41922891)

While the possibility of that is certainly true, it is false for this recent presidential election. Not all districts have reported in, but the most recent numbers show that Obama is ahead in the popular vote by a hair under 2.9 million votes.

http://www.google.com/elections/ed/us/results [google.com]

Re:Good for him (1)

Vermifax (3687) | about a year and a half ago | (#41922901)

It exactly represents the vote of the people state by state. As it is supposed to.

Re:Good for him (2)

ArcadeMan (2766669) | about a year and a half ago | (#41922989)

Does it?

If the votes in every state are 49% for Party A and 51% for Party B, that means Party B wins with "100% of the votes" country-wide yet 49% of the Americans would have voted for Party A.

How is that representative?

Re:Good for him (3, Insightful)

Tridus (79566) | about a year and a half ago | (#41923045)

There's only one President, and the guy with the most votes cast for him is there.

How is this NOT representative?

Re:Good for him (1)

ArcadeMan (2766669) | about a year and a half ago | (#41923265)

But he would have won with 51% of the votes, not 100%. The electoral results are not representative of the votes.

Re:Good for him (4, Insightful)

Tridus (79566) | about a year and a half ago | (#41923291)

The result of the vote is that he won. The result of the electoral vote is that he won.

This is pretty much the textbook definition of whining about something that doesn't matter in the slightest.

Re:Good for him (1)

alen (225700) | about a year and a half ago | (#41923087)

did that ever happen?
will it every happen? most likely not

people in every state have different beliefs on different issues which is the point behind the electoral college. Its supposed to make it hard to win an election by pandering and populism

Re:Good for him (1)

sjames (1099) | about a year and a half ago | (#41923335)

Because the candidate the majority voted for took office.

You could argue that a coalition government might have better represented the people's desires, but that's not an issue of the electoral college.

Re:Good for him (5, Insightful)

rhsanborn (773855) | about a year and a half ago | (#41923089)

It does tend to over-represent smaller states. This was somewhat intended, but as one analyst on Tuesday put it, "Should an electoral vote in Wyoming, that represents 130,000 people be equal to an electoral vote in California that represents over 600,000 people."

David Brooks gave an interesting response, he said that the electoral system forces the candidates to make an effort to play somewhat to the middle. Without the electoral system, Barack Obama would have campaigned heavily in California to get the liberal count up there. Mitt Romney would have campaigned heavily in Texas to instigate the conservative vote. The result was that they needed to go to places that weren't exactly on their side and try to convince them. They were forced to answer questions that both sides wanted to hear an answer to, rather than just their base. (I dare say most people haven't been pushing for real answers, but that's another issue altogether)

Re:Good for him (5, Insightful)

superdave80 (1226592) | about a year and a half ago | (#41922911)

I wish people would stop blaming the electoral college. The system is fine. It is the method in which the individual states assign the votes that is the problem. Florida in 2000 wouldn't have been such a big deal if they had distributed the electoral votes by district, rather than winner take all.

Win California by one vote? You get all 55 electoral votes! How stupid is that?

Re:Good for him (2, Insightful)

ArcadeMan (2766669) | about a year and a half ago | (#41923015)

It's as stupid as going with votes by district.

Win each district by one vote? You get all the votes, and hence all the states and the election! How stupid is that?

Re:Good for him (2)

superdave80 (1226592) | about a year and a half ago | (#41923299)

You get all the votes, and hence all the states and the election! How stupid is that?

So the person that gets more popular votes and electoral votes wins the election. I'm not sure I see what the problem is here.

Re:Good for him (1)

Anonymous Coward | about a year and a half ago | (#41922971)

Tuesday, the Pres. won the popular vote and the electoral vote. How could the nation have chosen more decisively?

Re:Good for him (1)

alen (225700) | about a year and a half ago | (#41922991)

i understand you live in flyover country, but there is a reason cows, chickens and corn can't vote

Re:Good for him (5, Insightful)

MightyMartian (840721) | about a year and a half ago | (#41923363)

The Electoral College system was never meant to represent "the will of the people". The House of Representatives is supposed to represent the will of the people. The presidency and the Senate had entirely different purposes and mandates.

Obligatory xkcd link (-1, Redundant)

drdread66 (1063396) | about a year and a half ago | (#41922755)

http://xkcd.com/1131/

Once in a while, Randall really nails it.

But when? (4, Insightful)

metrometro (1092237) | about a year and a half ago | (#41922759)

As Nate Silver pointed out, many times, over the course of the campaign, predicting what will happen one day before the election is easy. Very easy. Most everyone gets that right.

Predicting what will happen in June is hard. And much more interesting.

Re:But when? (4, Insightful)

Latentius (2557506) | about a year and a half ago | (#41922915)

And yet, even the very day of the election, there were still large numbers of pundits predicting a landslide victory for Romney. Guess the predictions aren't that easy, or perhaps it's just easier to ignore the numbers and resort to wishful thinking.

Pundits aren't there to provide accurate data. (5, Insightful)

mosb1000 (710161) | about a year and a half ago | (#41923017)

Pundits are there to draw people to a news organization, not provide accurate information. How many people would tune into the election if they said "Obama's got this one in the bag" (which we've known ever since Romney was nominated). Of course they will say you don't know who's going to win! Otherwise no one will watch their show.

Re:But when? (4, Insightful)

bluefoxlucid (723572) | about a year and a half ago | (#41923057)

Why do you think the early media reports were landslide for Romney? Why do you think it stayed so tense? Why do you think states were called before they were reporting, or that they were called for Romney at 74% reporting with 60% going to Obama? 97% chance of Obama victory. Obama's gonna win, Obama's winning, cool. Let's put in the Harry Potter DVDs and make some popcorn.

Don't watch Harry Potter, you faggots! Watch MSNBC and CSPAN so we can get ratings! Hey! HEY! HEY, CHECK IT PEOPLE, ROMNEY IS WINNING!

Re:But when? (5, Insightful)

rhsanborn (773855) | about a year and a half ago | (#41923117)

Large numbers of conservative pundits predicting a landslide for Romney. They also predict that the Earth is 6,000 years old, evolution isn't true, and illegitimate rape can't result in pregnancy. Let's call it wishful thinking instead of a prediction.

Re:But when? (2)

sjames (1099) | about a year and a half ago | (#41923403)

Most pundits aren't there to perform an insightful political analysis, they're there to tell a bunch of people what they want to hear so they can sell the soap.

Re:But when? (1)

explosivejared (1186049) | about a year and a half ago | (#41922931)

Exactly. Neither Nate nor any of the many other poll aggregators (Sam Wang, Drew Linzer, etc.) have found any way to conquer the inherit unpredictability of political events far into the future. Read Daniel Kahnemann. Experts, no matter how "scientific" their methods are consistently wrong and worse at predicting politics far into the future than the proverbial dart throwing monkey.

We care about Nate Silver and people that do what he does for two reasons: 1) They definitively point out that most pundits are full of crap and unwilling to realize that polling, not their guts, describe what's happening in the short run in the most accurate way currently possible. 2) For partisan reasons. Democrats love Silver because his numbers provided a security blanket to liberals afraid for Obama. To be fair, had the election turned out different and Nate's numbers called it for Romney, Republicans would be lionizing him as well, and we'd all be mocking whatever the Democratic version of "Unskewed Polls" had been. That popular media figures skew left really helps Silver's celebrity this time around.

Re:But when? (1)

scumdamn (82357) | about a year and a half ago | (#41923085)

This was the case in 2010. Many progressive blogs thought the turnout numbers would be much more Democratic than they were because party ID changed so much in two years. You have to trust the poll aggregates and I believe they have learned their lesson since. Let's see if the Republican pundits and publications do the same or if they declare 2014 a sea change and the beginning of a Republican ascendency just because typically Democratice constituencies don't turn out like Republican constituencies in off-years.

Re:But when? (2, Informative)

Anonymous Coward | about a year and a half ago | (#41923021)

Except that most of the pundits claimed that the race would either go in Romney's favour (Fox News), or would be a squeaker (pretty much every other pundit). In reality, Obama won pretty handily, and only those people who actually looked at the numbers (instead of their "guts") got it right. The National polls showed, in aggregate, by the end of the campaign, a 1-point advantage for Obama (which was pretty much right on the money). The State polls, except for Florida, were also very close to the end result. But the pundits decided that polls didn't matter, and what mattered most was their "narrative".

Well, facts matter. Numbers matter. And the post-modern view of most of the political class that only "narratives" matter is hokum.

Re:But when? (1)

Anonymous Coward | about a year and a half ago | (#41923069)

As Nate Silver pointed out, many times, over the course of the campaign, predicting what will happen one day before the election is easy. Very easy. Most everyone gets that right.

Predicting what will happen in June is hard. And much more interesting.

It is easy to make predictions the night before if you actually use numbers rather than your gut. But most pundits ignore numbers.

Look at the predictions from pundits right before the election. They were SHOCKINGLY inaccurate. CNN was pushing the narrative that it was a horserace that could go either way. They had a bunch of articles about how things might not be decided for days or even weeks. Fox News, of course, was heavily pushing the narrative that Romney was the prohibitive favorite and might even win in landslide fashion.

So the statement that "most everybody gets that right" is wrong if you are talking about the media. The media as a whole was never right. Of course, that's mostly because it was in their favor to be wrong (declaring things really close when it wasn't) than right. It is perfectly clear that being wrong and profitable is better than right and viewer starved when it comes to cable news.

Re:But when? (0)

Anonymous Coward | about a year and a half ago | (#41923093)

There's nothing interesting in "predicting" correctly in june, as it's nothing more than a lucky guess. At that point, much of the info hasn't even been made available (so much new dirt gets thrown in those last few months) that there's no way to possibly make an informed decision about how people will vote.

As far as predicting it the day before being easy, then please explain all the pundits on Monday predicting a landslide victory for Romney?

Seriously... (5, Insightful)

sunking2 (521698) | about a year and a half ago | (#41922761)

The only ones who believed the race was a 'virtual tie' were those who had gains to be had by it being so, namely the media.

Re:Seriously... (4, Informative)

explosivejared (1186049) | about a year and a half ago | (#41922957)

Romney's own internal numbers had Obama up by 5 in Ohio the weekend before the election even.

Re:Seriously... (1)

Anonymous Coward | about a year and a half ago | (#41923267)

It was a statistical tie, but because of the way the electoral college works, for Romney to have won he would have had to had to have had more of the tied States break for him than vice versa.

If you assume a really simple model, namely that Romney had to win Ohio, Virginia and Florida to win, and assume that he had a 50/50 shot at each, then his chances of winning were only about 1 in 8, or about 13%. Silver's model was obviously much more sophisticated and took into account every possible permutation and how likely they were.

And, that is not even taking into account the fact that Obama had a slight but insignificant edge in Ohio the entire time.

So basically, the only way that Romney was going to win was:

1) A systematic bias to the polling (which is what Republican pundits believed, but polling companies make money by being accurate, so probably not something to bank on).

2) A unlikely roll of the dice (hey, it happens in sports all the time).

Excellent (1)

seepho (1959226) | about a year and a half ago | (#41922765)

Are we going to see /. posts on the thousands of people whose predictions were false, or are we going full on confirmation bias here?

Re:Excellent (0)

Anonymous Coward | about a year and a half ago | (#41923079)

You can stream The Daily Show and Rachel Maddow from your browser and see clips of all the bad predictions and all the people who were surprised by how bad they lost.

Their predictions aren't nerd-news because they weren't dominated by math, but by belief.

Just to be clear, these are statistics. (4, Insightful)

mosb1000 (710161) | about a year and a half ago | (#41922771)

You can't really prove a probability wrong (unless it's 0% or 100%). While his most likely outcomes played out, it doesn't mean that he would have been wrong if a few of them hadn't.

Re:Just to be clear, these are statistics. (1)

Hatta (162192) | about a year and a half ago | (#41923001)

Sure you can, you just have to do it in aggregate. If when Nate Silver says there's an 80% chance of X happening, go back and see if X happens 80% of the time.

Re:Just to be clear, these are statistics. (1)

mosb1000 (710161) | about a year and a half ago | (#41923075)

These are one time events. You can't check to see of Obama will win Florida in 2012 6 times out of 10.

Re:Just to be clear, these are statistics. (2)

Colonel Korn (1258968) | about a year and a half ago | (#41923219)

We sort of can do an aggregate evaluation if we look at all of the state numbers and use all of the elections where Silver modeled the outcome based on polling and treat them as separate events. When we do so we see that Silver is most likely wrong...about his uncertainty. His predictions with 50-80% seem to be accurate something like >95% of the time, so he's probably overestimating the error in his model.

Re:Just to be clear, these are statistics. (1)

fermion (181285) | about a year and a half ago | (#41923269)

As has been mention Silver is not a celebrity because his model was the best or even remarkable. He is celebrity because he knows how to communicate, he is promoting himself for a book, and many people who think their model was better said his was bunk.

Notice I did not saw thought, or was, or was not. This is statistics. It is always a mistake to believe statistics will always be your friend. It will not. Relying on math alone is always a mistake. Models have a way of skewing from reality over time. Which is what happened to those that supported Romney and believed their numbers and algorithms predicted reality better. p. Nevertheless Silver's is a star created by those who opposed not based on observations or valid maths, but on personal belief. The belief that the non-white vote would disappear. The belief that most white people were fundamentally racist and therefore who vote against the black man when given an aristocratic white man.(while a majority can be shown to be have racist feeling, most now live or spent time in diverse community so they know a non-white can be, on occasion as good or better choice than a randomly selected white). The belief that jobs are the issue, when in fact, historically speaking, the job situation is relatively better for everyone except for the older white male. And ultimately, that a women should enjoy being raped and accept the baby to raise on her own.

Not how statistics works (3, Interesting)

GofG (1288820) | about a year and a half ago | (#41922793)

This actually shows that Silver is poorly calibrated. if he were accurately calibrated, 80% of his 80%-confidence predictions would come true, 50% of his 50%-confidence predictions would come true, etc. But 100% of his >50%-confidence predictions came true. In the future, he should be more sure of his predictions.

Re:Not how statistics works (1)

iluvcapra (782887) | about a year and a half ago | (#41922859)

Can't really draw any conclusions until he's done 5 or 10 elections.

Re:Not how statistics works (0)

Anonymous Coward | about a year and a half ago | (#41923155)

He already did 2 previous elections as far as I know with very good accuracy.

Re:Not how statistics works (5, Informative)

YodasEvilTwin (2014446) | about a year and a half ago | (#41922861)

...that's not how statistics work either. A result does not alter the underlying probabilities.

Re:Not how statistics works (0)

Anonymous Coward | about a year and a half ago | (#41923063)

No. If his confidence intervals were properly calculated, GPs description would be (approximately) true. The likelihood that he "accidentally" got all of his 50%+ predictions right is minuscule.

Re:Not how statistics works (0)

Anonymous Coward | about a year and a half ago | (#41923177)

It isn't so remarkable when you see that there were not that many 50%-confidence value predictions, only a few states (maybe only one) had confidence values near 50%. The rest has relatively high confidence values of 80%+.

Re:Not how statistics works (1)

GofG (1288820) | about a year and a half ago | (#41923303)

He predicted every state-level election, correct? Maybe 80 representatives, 33 senators or whatever?

He should have predicted at least one of them incorrectly. There is just no way he isn't undercalibrated. His predictions were *massively* correct, closer to 99% than 95% (100:1 vs 20:1).

Re:Not how statistics works (3, Insightful)

khallow (566160) | about a year and a half ago | (#41923043)

You have to remember that these are highly correlated events. A lot of the uncertainty goes all the same way. But I otherwise agree with the other poster. We'll need more of a track record to see how Silver does.

There are still warning signs. For example, he has issued predictions with three digits of precision. That's an obvious sign that something isn't right.

Re:Not how statistics works (3, Insightful)

VAElynx (2001046) | about a year and a half ago | (#41923179)

This actually shows that Silver is poorly calibrated. if he were accurately calibrated, 80% of his 80%-confidence predictions would come true, 50% of his 50%-confidence predictions would come true, etc. But 100% of his >50%-confidence predictions came true. In the future, he should be more sure of his predictions.

Congratulations, this is the stupidest thing I have read today.
The confidence rate of a prediction doesn't work like that. It's the probability with which the null hypothesis can be rejected given the data, basically, suppose that the relationship you are trying to prove isn't there, how likely it is that the data were generated by a statistical fluke? And much like any other statistics, a bunch of predictions with a 50% confidence interval doesn't mean that half of them must come right, especially in a single sample - all it means is that it's as likely for the theory to be true as for it to be false.

Re:Not how statistics works (5, Insightful)

radtea (464814) | about a year and a half ago | (#41923327)

This actually shows that Silver is poorly calibrated. if he were accurately calibrated, 80% of his 80%-confidence predictions would come true, 50% of his 50%-confidence predictions would come true, etc. But 100% of his >50%-confidence predictions came true. In the future, he should be more sure of his predictions.

Not necessarily. Most of the uncertainty in his predictions was due to the conditional probability of systematic bias in likely voter models. For example, Gallup was showing much better results for Romney and the Rac... err... Republicans across the board, which was probably due to how they screened people who responded.

Systematic error shows up as a conditional probability, so you are lumping together completely disjoint realities into your final result. In terms of discrete conditional probabilites, imagine that based on historical data you have three equally likley possible conditions: 5% Democratic bias, no bias, and 5% Republican bias. You run your simulations with each of these three biases, and you get a result that says senators D0, D1, D2, D3, D4, D5, D6, D7, D8 and D9 are all 80% likely to win. But 99% of that 20% chance they'll lose comes from the condition where there's a 5% Republican bias.

But remember: those three conditions describe completely disjoint realities. They are not sampling error, but statements of ignorance about the actual state of the world.

Now the world really is just one way (it may be ambiguous relative to some human categorization, but then that ambiguity is just part of the one unique way the world realy is.) So only one of the three conditions are true. If it happens that the no-bias case is the way the world really is, then 100% of those 80% chances will come true.

That said, in future elections Bayesian predictions of the kind Silver and everyone else in this space are making will lower the conditional probabilities of bias, because this election demonstrated good low-bias results, but so long as the ultimate uncertainty is dominated by the systematic error, Bayesian predictors will tend to appear either uncannily accurate or dismayingly inaccurate.

However, averaged over many, many election cycles (18 or more) you would expect to get statisics such that 80% of the 80% calls are correct, and so on. But within individual elections that use fixed likely-voter models that won't be the case.

Conditional probabilities are one of the most difficult things for humans to understand (the Monty Hall problem is a classic case where all the confusion comes from treating a conditional probability as if it was a total probability) so it's worth practicing the art of thinking carefully about these things, and the odds are still good I've said at least one confusing or incorrect thing in the above.

Re:Not how statistics works (1)

Anonymous Coward | about a year and a half ago | (#41923359)

Everyone in this room is now dumber for having [read your post]. I award you no points, and may God have mercy on your soul.

Re:Not how statistics works (1)

epine (68316) | about a year and a half ago | (#41923431)

But 100% of his >50%-confidence predictions came true. In the future, he should be more sure of his predictions.

I'm not sure that I agree with you a hundred percent on your policework there, Lou.

The national polls were showing strength for Romney, but when you drilled down, the models predicted victory for Obama. Whether the errors moved together (broad correlation) or not (independent precinct variation) determines whether you can apply "skill" with skill.

There were many pieces on Silver recently. The most cynical was this one:
Tarnished Silver: Assessing the new king of stats [macleans.ca]

Cosh is an erudite and formidable writer, which a background in French literature, and a redneck persona right out of the donut shop sketch on Canadian Air Farce. (I'd really like to see that sketch redone with Cosh vs Hitchens. Then the camera pulls back to show Pierre Trudeau and Margaret Atwood at the next table engaged in banal chitchat.)

In this case I'm not going to nod and slurp my double-double jumbo Timmies and cuckoo "you got that right" after every punchline.

Nate actually wrote up a postmortem piece on his bungled UK election call where he discusses the use of exit polls to apply a uniform swing (broad correlation) to the election as a whole, by which means the BBC outperformed his own predication. When you break it down, the R^2 values on vote shares weren't all that different. In that election, the best prediction came from a group which split the difference, applying swing to broad regions (such as Scotland, which behaved differently than the rest of the UK).

Silver goes on to argue that the UK should adopt more detailed exit polls as we have in America. I don't understand this, because I don't understand the incentive of people to A) participate, or B) tell the truth. If I have a vested interest in the outcome (let's just say I'm opposed to organized religions lacking the basic dignity to concoct their miracles *prior* to the invention of the printing press). Why shouldn't I just say or do whatever I believe serves my interest every step of the way? Every politician does this. It's his job to lie, and ours to tell the truth? Blow me.

The more exact psephology becomes, the easier it becomes for Joe Donut to play the game alongside the pols. This is not a feedback loop that converges to a good place.

In any case, Lou, I think you should sign up for a MOOC entitled Discussing "Skill" with Skill. This broadly means writing down the necessary preconditions (such as: x not equal to zero when dividing by x; x has a known sign when multiplying both sides of an inequality; infinite series converges; variables are i.i.d. in a variance summation).

Another potential source of nationwide "correlation" is electronic voting fraud. 90 to 95% predictions would be my personal ceiling in any American election until the last closed-source voting machine is on the downward escalator in some deep ocean trench.

Are you going to apply the i.i.d. assumption in a state government by Jeb Bush? If so, you're a brave man. May the votes be with you.

Sounds like Psychological Projection (-1)

Anonymous Coward | about a year and a half ago | (#41922813)

Joe Scarborough called someone else a "joke"?

Does MSNBC even cover Joe Scarborough's paycheck or do they let Starbucks foot the bill?

All? (1)

superdave80 (1226592) | about a year and a half ago | (#41922829)

I don't think Florida has been called yet (although Obama currently has the slight edge there). And I could have sworn that he had Florida leaning slightly towards Romney for quite some time, but now his website shows a slight lean towards Obama?

Not to say that he still didn't do a hell of a job predicting the outcome (just like last election), but to say 'all' at this point seems to be jumping the gun a bit.

Re:All? (1)

Gerald (9696) | about a year and a half ago | (#41922893)

The outstanding absentee ballots are from areas that lean Obama [electoral-vote.com] , which is why most places have at least unofficially called the state for Obama.

Re:All? (0)

Anonymous Coward | about a year and a half ago | (#41922953)

He changed florida to be for Obama before the election started. If I recall, he was the only one to do so. As he said he was always updating his numbers based on latest information.

Re:All? (3, Informative)

Missing.Matter (1845576) | about a year and a half ago | (#41922977)

Last minute polls on the last day of the race brought the state to 50 - 50 odds. Given the fact the state is still too close to call two days after the election, Nate seems to have called it no matter which who ultimately gets the electoral votes.

Re:All? (2)

ChrisGoodwin (24375) | about a year and a half ago | (#41923005)

About 2-3 days before the election he switched Florida from light pink to light blue.

Re:All? (1)

Daetrin (576516) | about a year and a half ago | (#41923377)

As others have pointed out, he switched Florida a couple days before the election. Also, he posted some exact numbers the day of the election [nytimes.com] :

"In the final pre-election forecast at FiveThirtyEight, the state of Florida was exceptionally close. Officially, Mr. Obama was projected to win 49.797 percent of the vote there, and Mr. Romney 49.775 percent, a difference of two-hundredths of a percentage point."

Yes, it'll be a few days probably before we know for sure if he called it correctly, but he was certainly 100% correct about how close it was going to be, which personally i think is good enough to call his prediction true at the moment.

Why Nate? (5, Interesting)

MyLongNickName (822545) | about a year and a half ago | (#41922847)

Why does Nate get so much attention, when other sites like electionprojection.com and electoral-vote.com do a similar service, are open on their methods and have had almost perfect results for the past two elections. This past election, those two sites only missed on Florida and that one was truly too close to be 90% confident on one way or another.

Re:Why Nate? (2)

d23tek (1208848) | about a year and a half ago | (#41922995)

Two reasons: Nate is a person with a face (and a catchy name). A generic site like electionprojection.com doesn't have a personality, doesn't do interviews, and can't be ridiculed or praised for personal characteristics. Secondly, it's precisely because other sites' methods are open. Once you see how the magic trick is done, it's not interesting anymore.

Re:Why Nate? (2, Informative)

Anonymous Coward | about a year and a half ago | (#41923011)

Probably because he was subject to personal attacks from some Very Loud People condemning him for predicting the result they didn't want to get.

Re:Why Nate? (1)

Anonymous Coward | about a year and a half ago | (#41923023)

Because he was targeted by analysts who disputed his results.

Re:Why Nate? (1)

Missing.Matter (1845576) | about a year and a half ago | (#41923047)

He made a name for himself in 2008, and is now on one of the most viewed news sites out there. electionprojection.com isn't exactly the New York Times. Because of this he got appearances on the Daily Show, Morning Joe, Real Time, and other TV shows; now he's kind of a personality, especially among uber nerds who identify with his nerdy demeanor.

Re:Why Nate? (2)

mewsenews (251487) | about a year and a half ago | (#41923083)

I'm guessing the likely explanation is that Big Media only cares about other Big Media and this Nate Silver guy has the NYT as a patron

Re:Why Nate? (1)

mattie_p (2512046) | about a year and a half ago | (#41923109)

Nate gets so much attention because he is right. A lot. In 2008 he got 49 of of 50 states correctly as well as all 35 senate races. In 2012 he got 50 out of 50 states right (he did, in fact, call Florida). As an exercise to the reader, I leave it to you to figure out how many senate races he correctly called.

Re:Why Nate? (0)

Anonymous Coward | about a year and a half ago | (#41923163)

Apparently your reading skills are weak. OP indicated that the two sites are also very accurate and have a good track record.

Re:Why Nate? (5, Insightful)

scumdamn (82357) | about a year and a half ago | (#41923119)

It's because of what he offers in addition to the model. The color commentary, explanation of why things change from day to day and analysis of individual polling firms after the fact. I was paying attention to him from the start (2008 primaries on Daily Kos) and he does a great job of explaining things to the layman.

Re:Why Nate? (0)

Anonymous Coward | about a year and a half ago | (#41923165)

All Silver was doing what aggregating the polls of others. His "predictions" were the same as realclearpolitics with the exception of Florida. And Silver's prediction of Florida was made at the very last minute. His projection was basically 50-50 and he chose Obama based on minor movement in the national polls in Obama's favor. It won't matter who won Florida now. Silver's defenders won't care even if it turns out he was wrong.

Re:Why Nate? (0)

Anonymous Coward | about a year and a half ago | (#41923331)

I've never heard of the guy before a few days ago and I haven't understood why all the ridiculous praise since the election, either. I mean, he predicted the outcome of the election; it's not like he *caused* the outcome of the election. Further, I don't think everyone would be jerking each other off on twitter and everywhere else, with Nate Silver comments every five minutes since election night, if he had predicted a win for Romney that had come reality (and, no, I don't favor Romne even in the slightest --- but let's be frank here -- most of his popularity is because he reassured a win for the left and that proved out).

It kind of strikes me like those people who only pay attention to politics for about two weeks every four years. Or who suddenly start tweeting and facebooking about a soccer player or football player -- by name -- during the championships, when they don't otherwise follow or know anything about said sport or that person.

At any rate, I find the skill to predict the outcome of an election to be utterly useless. I'm sure it serves a purpose for someone, but for 99.99% of us . . . it hardly matters. Unless you're wagering on it, I guess.

The pundits' derision backfired (0)

Anonymous Coward | about a year and a half ago | (#41923407)

Nate was the most visible, to the point where his predictions were most vociferously derided by the pundits (well, mainly those who didn't like his conclusions.)

Just looked at the real world and called it (2)

Attack DAWWG (997171) | about a year and a half ago | (#41922873)

Some of the things people said about him were nuttier than that. This [examiner.com] guy called him "a man of very small stature, a thin and effeminate man with a soft-sounding voice that sounds almost exactly like the “Mr. New Castrati” voice used by Rush Limbaugh on his program." So that is supposed to have some sort of effect on his analysis?

All he did was take polls that already existed--lots of them--and do statistical analysis on them. Just ran the numbers, and didn't speculate. Contrast that with sites like this [unskewedpolls.com] one, which, although quite pompous, was stuck in its own alternate reality and ended up being quite wrong.

Re:Just looked at the real world and called it (3, Funny)

MyLongNickName (822545) | about a year and a half ago | (#41923053)

I've never seen the unskewedpolls site before, but it is funny. The take a Rasmussen poll which have consistently skewed toward Republicans in the past two elections, accuses it of liberal bias and then insists on changing the Rasmussen result by 3 points in favor of Romney. Only then can it say that Rasmussen has predicted a Romney win. This would be like the KKK having a bias in favor of black men.

I wonder if I can buy the domain unskewedpolls.com (0)

Anonymous Coward | about a year and a half ago | (#41923107)

Should go for a song now that its faith-based math was proven to be ineffective. At least they should get something out of the time and money they spent on the site. Sadly, I guess we never will know Romney's plan to lower taxes by 20% AND lower the deficit. He seems to be taking the details of *that* plan back to Utah with him without revealing them,

PECOTA didn't help the Yankees (0)

alen (225700) | about a year and a half ago | (#41922945)

Baseball Prospectus had the Yankees and Nationals as PECOTA favorites to win almost every game they lost. even after the Yankees lost Jeter.

Giants vs Tigers you didn't need PECOTA to predict that SF would crush the kitties

i was actually expecting Romney to win

How can he be right? (1)

Dionysus (12737) | about a year and a half ago | (#41922949)

But, but, according to conservative pundits, he's too "effeminate" to be right [examiner.com]

Facts never trump belief and/or tradition (1)

erroneus (253617) | about a year and a half ago | (#41922955)

There are ample examples of how provable and testable facts are unable to trump belief and tradition. Belief and tradition are parts of who people are. Facts are independant of people and no one owns them. To have a fact conflict with belief is literally to loose a part of one's identity and most are far too egotistical to allow any part of themselves to be lost or cast into doubt.

Re:Facts never trump belief and/or tradition (2)

ChrisGoodwin (24375) | about a year and a half ago | (#41923037)

Then they should get used to being wrong.

They are used to being wrong all the time (1)

bussdriver (620565) | about a year and a half ago | (#41923153)

Pundits are entertainers they need no skill or brainpower; when they fuck up they just have to be entertaining to keep their job. They are just another form of a reality TV "star" and things have been so bad for so long an airhead pundit type character ran for VP in 2008... and is now with her people, as an airhead pundit.

I know a dittohead and he is a master at rationalization either from being wrong all the time or because his talent allowed him to live in a bubble without knowing it.

Failure to defraud (1)

sir_eccles (1235902) | about a year and a half ago | (#41922959)

One wonders if it is possible to draw the conclusion that the because his model was so accurate then the majority of voter suppression efforts must have been ineffective? Or the corollary, if his model had been wrong would that have indicated large scale successful voter suppression?

Re:Failure to defraud (1)

paiute (550198) | about a year and a half ago | (#41923113)

One wonders if it is possible to draw the conclusion that the because his model was so accurate then the majority of voter suppression efforts must have been ineffective?

In a close election such as most of the last few have been, you don't need a lot of voter suppression to win - you just need a tad in the right places.

Big data PROVED 2000 and 2004 elections stolen too (1, Interesting)

leftie (667677) | about a year and a half ago | (#41923277)

The opinion polls entering election day and exit polls proved the Karl Rove and the GOP stole both the 2000 and 2004 presidential elections for George W Bush.

Not so good in the Senate (0)

Anonymous Coward | about a year and a half ago | (#41923283)

He missed 2 in the Senate (Montana and North Dakota) Both were projected to go Republican but ended up going Democrat. By the colors it looks like ND was 90% probability.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>