Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Math Indicates Pollster Is Forging Results

Soulskill posted more than 4 years ago | from the lies-damned-lies-and-statistics dept.

Math 319

An anonymous reader writes "Nate Silver suggests the political pollster Strategic Vision is 'cooking the books. And whoever is doing so is doing a pretty sloppy job.' Silver crunched five years worth of their polling data, and found their reported results followed a suspicious pattern which traditionally suggests fraud. The five-year distribution of the numbers 'is not random. It's not close to random.' The polling firm had already been reprimanded by the American Association for Public Opinion Research for failing to disclose their methodology, though the firm argues they did comply with the organization's request. Their response to Silver's accusation? 'We have a call in to our attorney on this and fully intend to take action that will vindicate us.'"

cancel ×

319 comments

Sorry! There are no comments related to the filter you selected.

Affordability (-1, Offtopic)

postmortem (906676) | more than 4 years ago | (#29545537)

USA can't afford space program anymore; but China and few others can. Welcome to 21st century, USA is not being number one in anything.

Evolution in Action (0, Offtopic)

Bald_Earthling (948761) | more than 4 years ago | (#29545547)

As much as I don't like to see NASA die, what is Bad News for NASA is probably Good News for the private launch industry. Go go SpaceX and Armadillo!

Re:Evolution in Action (1)

musefrog (1471169) | more than 4 years ago | (#29545555)

I like that in your enthusiasm for all things private launch industry, you didn't notice that you'd loaded the wrong story to comment on... :)

Re:Evolution in Action (1, Offtopic)

WindBourne (631190) | more than 4 years ago | (#29545591)

The nasa story is not working. I guess at this time, NASA stands for "Need Another Slashdot Area".

Re:Evolution in Action (3, Insightful)

Anonymous Coward | more than 4 years ago | (#29545693)

BREAKING NEWS:
The AP is reporting a major fuckup at Slashdot. The web site cannot even do the most basic task essential to its operation, allows readers to leave comments on articles. No comments were available from anyone employed by the web site. Phones rang and rang and rang. Several other Sourceforge properties had their numbers disconnected due to non-payment.

It is apparent no one in charge of the place gives even a sliver of a fuck, or even reads the front page after articles are posted, as it is 2009 and there are 50 fucking ways to notify the readership of the nature of the problem and the expected timeline for resolution. And that 50 is just from a fucking cell phone. If a person had an actual computer and an internet connection, even a netbook at a Starbucks, the number rises into the 1000s.

Long gone are the days when the popular geek web site devoted to technology actually worked. Long gone are the days when there were actual technical explanations of outages. Instead its more stories about politicians arguing over traffic ticket revenue posted as "Your Rights Online", iPhone slashvertisements, slashvertisements masquerading as book reviews, and links to people's blogs about blogs about news stories, and/or tweets about tweets about press conference summaries.

Re:Evolution in Action (5, Funny)

NoYob (1630681) | more than 4 years ago | (#29545623)

I've been experiencing weird things too with Slashdot and stories not loading and seeing things that don't make sense. I don't get it.

Anyway, back to the topic of Windows 98 being released today. I wonder if the Clinton Administration will continue with the anti trust investigations into M$.

Re:Evolution in Action (1)

popeye44 (929152) | more than 4 years ago | (#29545791)

You meant to post this in the thread for the new game with time travel right? :-]

Re:Evolution in Action (3, Funny)

Arthur Grumbine (1086397) | more than 4 years ago | (#29546051)

I know this might be slightly off-topic, but I think that the issues Slashdot has been having are due to an unexpected spike in traffic after they posted the story of how 3D Realms was switching over to Epic's Unreal Engine for the upcoming Duke Nukem Forever. I'm pretty stoked about this and am saving up to be able to afford a Voodoo2 - DNF is gonna be da bomb!!

Re:Evolution in Action (1)

clarkkent09 (1104833) | more than 4 years ago | (#29545895)

Seems like NASA story comments are appearing in here. Tragically, GP might have been modded off topic and now mocked through no fault of his own. There is no justice in this world *shakes fist at the gods*

Re:Evolution in Action (1)

Shadow of Eternity (795165) | more than 4 years ago | (#29545565)

what exactly do these guys have to do with NASA?

major fcukup at slashdot (5, Informative)

postmortem (906676) | more than 4 years ago | (#29545559)

a. you can't post
b. if you do manage to post, post goes to wrong topic!

Re:major fcukup at slashdot (3, Informative)

multisync (218450) | more than 4 years ago | (#29545649)

Yeah, it's been like that off and on all day.

To those with mod points: use them on something worthwhile. Noting that your posts are turning up in the wrong topic is on topic. Modding postmortem's post Off Topic is a mis-use of your mod points.

Re:major fcukup at slashdot (0)

Anonymous Coward | more than 4 years ago | (#29545655)

yep, strange, never seen this to happen

Re:major fcukup at slashdot (0)

Anonymous Coward | more than 4 years ago | (#29545669)

I guess that's what passes for improvement.

Re:major fcukup at slashdot (1)

woodrad (1091201) | more than 4 years ago | (#29545997)

I feel bad that this post is 0, Offtopic. If I had mod points, I would mod you up. Why? Because I give you the benefit of the doubt and believe you were meaning to post in a story that was about that problem.

Why should I care? (1, Flamebait)

Just Some Guy (3352) | more than 4 years ago | (#29545561)

Pretend I know nothing about Pollster (which happens to be true). Why should I care whether they've faked results? By that, I mean: do they research options of favorite flavors of cotton candy, or public support for health care reform, or the best style of car, or...? In other words, do they do stuff that actually matters?

Re:Why should I care? (4, Insightful)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#29545577)

From TFA, it looks like they handle a fair variety of sundry topics in American politics. Not a giant deal, I've certainly never heard of this particular outfit before; but I find it extraordinarily hard to believe that anything which increases the amount of false-but-plausible-looking noise in the world is a good thing.

On important topics such is more dangerous than on less important ones; but its mere existence makes the world a less knowable place either way. Either you have people believing false data, or you have people falling into the essentially nescient "all data are just source biases" position.

Re:Why should I care? (4, Insightful)

multisync (218450) | more than 4 years ago | (#29545749)

its mere existence makes the world a less knowable place either way

Well said.

I find it disturbing, too, that the media just reports the polling companies' results, without reporting things like what questions were asked, in what order, how the poll was conducted or who commissioned it, all of which can have a big effect on the results. A lot of "push polling" goes on, especially when the polls are commissioned by special interest groups, business associations, unions or political parties themselves.

I'm not in the US, so I don't know this polling company, but I've had a municipal, provincial and federal election in the past 12 months (with another possible federal election imminent) and I think polling and radio call in shows have a great deal of effect on people's opinions these days, more so than traditional newspaper and television newscasts.

If Strategic Vision was conducting fraudulent poles, I would be looking at their client list and going after whoever paid for them as well.

Re:Why should I care? (3, Informative)

maxume (22995) | more than 4 years ago | (#29545849)

NBC always reports on the NBC/Wall Street Journal poll. I think they commission it. They seem to do a decent job of describing how they do it:

http://online.wsj.com/article/SB124527518023424769.html [wsj.com]

(that link works when clicked on from a Google search, but given that the WSJ has a mighty paywall, I don't know if it will work otherwise)

So maybe you need to talk about a more nuanced group than 'the media' (I wouldn't be particularly shocked if other major outfits were at least approximately as responsible).

Re:Why should I care? (3, Insightful)

Brian Gordon (987471) | more than 4 years ago | (#29545965)

If the vote is to reflect public opinion, people should vote their own opinion. They don't need to try to help the system by guessing the most popular option.

Re:Why should I care? (1)

Unoriginal_Nickname (1248894) | more than 4 years ago | (#29546023)

Voting, being a proactive decision, automatically introduces selection bias into the poll. i.e. the sample group represents those who are likely to volunteer their opinions, rather than a cross-section of the general population,

Re:Why should I care? (1, Flamebait)

HornWumpus (783565) | more than 4 years ago | (#29545581)

They do polling so obviously: No they don't do stuff that matters.

Nobody believes them anyhow. Any of them.

Re:Why should I care? (0)

Anonymous Coward | more than 4 years ago | (#29545817)

Nobody believes them anyhow. Any of them.

I wish that was true. But people not only believe terrible polls, they will even lie about who they voted for. Fake polls are known to change opinion.

That said, Nate's method is screwed up.

Re:Why should I care? (3, Insightful)

Spy der Mann (805235) | more than 4 years ago | (#29545587)

Pretend I know nothing about Pollster (which happens to be true). Why should I care whether they've faked results? By that, I mean: do they research options of favorite flavors of cotton candy, or public support for health care reform, or the best style of car, or...? In other words, do they do stuff that actually matters?

Faked polls = astroturfing.

Need I say more?

Re:Why should I care? (1)

mrmeval (662166) | more than 4 years ago | (#29545657)

If the fake polls do not benefit a company are they astroturfing? If it benefits the current regime is it astroturfing?

Re:Why should I care? (1, Insightful)

Tontoman (737489) | more than 4 years ago | (#29545671)

Depends on who is commissioning the poll.

Republicans are (-1)

Anonymous Coward | more than 4 years ago | (#29545899)

Republicans are commissioning the polls from Strategic Vision. Look it up on Google. They often come up with results that favor the Republicans, more so than other polling organizations. Hmm . . .

Re:Why should I care? (5, Informative)

Attack DAWWG (997171) | more than 4 years ago | (#29545975)

They are a partisan, Republican-oriented polling company. They have gotten into trouble in the recent past for their questionable results.

Re:Why should I care? (2, Funny)

MobileTatsu-NJG (946591) | more than 4 years ago | (#29545865)

Pretend I know nothing about Pollster (which happens to be true). Why should I care whether they've faked results? By that, I mean: do they research options of favorite flavors of cotton candy, or public support for health care reform, or the best style of car, or...? In other words, do they do stuff that actually matters?

Faked polls = astroturfing.

Need I say more?

Well, you might need to explain what astroturfing is. Most people here think that astroturfing is when you are satisfied with a mass-market product.

Re:Why should I care? (1)

Unoriginal_Nickname (1248894) | more than 4 years ago | (#29546027)

Or dissatisfied with an unpopular company's products.

Re:Why should I care? (1)

skine (1524819) | more than 4 years ago | (#29545599)

Try visiting the website before asking questions about its essence.

Re:Why should I care? (4, Insightful)

interkin3tic (1469267) | more than 4 years ago | (#29545621)

First of all, I don't think "What do I care" is anything but flamebaiting. Who cares if you don't care?

Second, if they're the same "strategic vision" that the article is talking about, their webpage says
"Strategic Vision has worldwide experience developing tools to measure decision-making, human behavior, attitudes and perceptions. Its globally relevant, comprehensive theory of human behavior creates the most effective strategies addressing decision-making in product development and communications in the widest variety of fields, including automotive, customer service, government and politics, medicine and healthcare, organizational and jury, travel and leisure, food and beverages, and education." So they probably report on anything you will pay them to poll on, or rather, anything you will pay them to make a graph from nothing.

Their self-reported client list [strategicvision.com] . Granted, they may have just made that list up as well.

Lastly, a quote in TFA by the company gives you plenty of reason to care:

[W]e categorically deny them and will refute them. We have a call into our attorney on this and fully intend to take action that will vindicate us...he has attempted to do severe damage to our reputation and what is he going to do when we disprove him just say I am sorry. That isn't enough at this point.

There you go: the company is mad about being uncovered and is doing the next step any stupid assholes do when their misdeeds come to light: sue in a vain attempt to keep the information from becoming well known. Therefore, -everyone- should know they're faking the results. I'm tempted to e-mail all their clients with a link to the article. If they go out of buisiness, maybe other shitty companies will finally realize you don't sue people who expose you as charlatans.

Bwhahahah, sometimes I say ridiculous things.

Re:Why should I care? (0)

Anonymous Coward | more than 4 years ago | (#29545759)

Thats Strategic Vision Inc, the wrong one, this is the right one, Strategic Vision LLC.
http://www.strategicvision.biz/political/index.html

Re:Why should I care? (5, Informative)

bfields (66644) | more than 4 years ago | (#29545833)

if they're the same "strategic vision" that the article is talking about, their webpage says "Strategic Vision has worldwide experience developing tools to measure decision-making, human behavior, attitudes and perceptions....

Nope, you're looking at the webpage of a different company! See Nate's previous article [fivethirtyeight.com] :

Why would you pick the name "Strategic Vision, LLC" for your company when the name "Strategic Vision, Inc." was already in use by an extremely well regarded, San Diego-based research firm that has been in business for more than 30 years? Are you deliberately trying to confuse your potential clients and leverage Strategic Vision, Inc.'s much stronger brand name?

Re:Why should I care? (4, Informative)

quantaman (517394) | more than 4 years ago | (#29545855)

Second, if they're the same "strategic vision" that the article is talking about

They're not, from another helpful article from FiveThirtyEight [fivethirtyeight.com]

Why would you pick the name "Strategic Vision, LLC" for your company when the name "Strategic Vision, Inc." was already in use by an extremely well regarded, San Diego-based research firm that has been in business for more than 30 years? Are you deliberately trying to confuse your potential clients and leverage Strategic Vision, Inc.'s much stronger brand name?

You're looking at the page from the well regarded Strategic Vision, Inc. Funny that SV LLC seems to be so happy to sue Nate Silver, it would seem that SV Inc has a far stronger case against SV LLC.

Could be an interesting intersection of Trademark/Slander laws...

Who cares if you don't care that he doesn't care? (1)

ClosedSource (238333) | more than 4 years ago | (#29545869)

It's "Who cares?"(s) all the way down.

Re:Why should I care? (1)

zero0ne (1309517) | more than 4 years ago | (#29545901)

If that client list is true, how many of those clients do you think would at least help finance the guy making these claims?

If it comes down to lawyer fees, I would imagine tons of those companies would toss some money to this guy. If he ends up being right that they have fraudulent results, wouldn't that mean Strategic Vision's clients can sue?

Re:Why should I care? (4, Informative)

(startx) (37027) | more than 4 years ago | (#29545911)

Except you've linked to the wrong company. Strategic Vision, Inc. [strategicvision.com] is a well respected 30-year old polling firm in California. Strategic Vision, LLC [strategicvision.biz] is the shady 5-year old GOP shill corp with questionable poll results and no real office (or polling results allegedly). Careful with those links, you don't want to slander the wrong company here. I think SV Inc. may have a trademark case on their hands if their feeling litigeous.

Re:Why should I care? (0)

Anonymous Coward | more than 4 years ago | (#29546007)

Their = something they possess - They're = they are.

They're feeling litigious.
Their lawyers are feeling litigious.

Re:Why should I care? (0)

Anonymous Coward | more than 4 years ago | (#29545961)

PLEASE NOTE:

There are two companies named Stragetic Vision

Stragetic Vision Inc, a respectable California-based polling firm. This is the one you have linked. http://www.strategicvision.com/

and

Stragetic Vision LLC, the sketchy Atlanta-based polling firm in question.
http://www.e-strategicvision.com/

In a previous post, Silver has asked:

Why would you pick the name "Strategic Vision, LLC" for your company when the name "Strategic Vision, Inc." was already in use by an extremely well regarded, San Diego-based research firm that has been in business for more than 30 years? Are you deliberately trying to confuse your potential c

http://www.fivethirtyeight.com/2009/09/few-more-questions-for-sketchy-pollster.html

Re:Why should I care? (3, Funny)

schon (31600) | more than 4 years ago | (#29545967)

I don't think "What do I care" is anything but flamebaiting. Who cares if you don't care?

According to a poll I just saw by Strategic Visions LLC, 68% of Americans care!

Strategic Visions Inc. != Strategic Visions, LLC (1)

Andorion (526481) | more than 4 years ago | (#29545995)

FYI:

Good: Strategic Visions Inc. @ http://www.strategicvisionsinc.com/ [strategicvisionsinc.com]
Suspect: Strategic Visions, LLC @ http://www.strategicvision.biz/ [strategicvision.biz]

See: http://www.fivethirtyeight.com/2009/09/few-more-questions-for-sketchy-pollster.html [fivethirtyeight.com]

Re:Strategic Visions Inc. != Strategic Visions, LL (1)

Andorion (526481) | more than 4 years ago | (#29545999)

Wow, I even screwed it up... they're both "Strategic Vision" without the s at the end, and the Good is at http://www.strategicvision.com/ [strategicvision.com]

Re:Why should I care? (5, Informative)

TeethWhitener (1625259) | more than 4 years ago | (#29545627)

In other words, do they do stuff that actually matters?

In a word, yes. Nate Silver manages the blog FiveThirtyEight [fivethirtyeight.com] and is well-known as a statistical analyst from the 2008 US election (among other things). Strategic Vision has released quite a few polls. In Silver's words,

...Strategic Vision's polls cover a wide array of topics: Presidential horse race numbers in any of a dozen or so states, senate and gubernatorial polling, primary polling, approval ratings of various kinds, polling on issues like the war in Iraq, and more abstract questions such as whether voters think that 'experience' or 'change' is the more important quality in a Presidential candidate.

So yes, this is pretty big news, should it turn out that Strategic Vision's behavior is in fact illicit. They're influential enough that news agencies may pick up their polling results. This is bad enough, but when you factor in the fact that polling results can be very effective propaganda in something like a presidential race, fraudulent polling can have significant consequences.

Re:Why should I care? (1)

turkeydance (1266624) | more than 4 years ago | (#29545715)

you care because: 1. your eleven-year-old daughter cares. deeply. this kind of stuff is middle-school CNN. she will make SURE you care. 2. fake? says who? and, more importantly, why? 3. finally, to be the man, you have to beat the man. in other words, disprove it if you can, since the man posted first.

Re:Why should I care? (1)

Dachannien (617929) | more than 4 years ago | (#29545823)

That's actually a really good point. On a related note, what I'm interested to know is whether the allegedly faulty data diverges from other firms' polling data on particular questions. In other words, are they pushing an agenda of some sort? Are they just faking data so they have something to sell? Is Nate Silver full of shit?

More interesting info about Strategic Vision (1)

Attack DAWWG (997171) | more than 4 years ago | (#29545831)

Here is some more info [ajc.com] about them. According to the article they are a "Republican-oriented polling firm based in Atlanta."

Re:Why should I care? (5, Interesting)

plague911 (1292006) | more than 4 years ago | (#29545837)

Strategic Vision is a Republican pollster. Meaning when a Republican politician wants a poll about a particular set of data they give Strategic Vision some money and they do a poll. This can be for either internal polling to give them and idea how the "battle" is going or for general consumption. And yes Strategic Vision is big enough to matter, but they are just the tip of the iceberg how misleading "R" pollsters

In general there are some Republican some Independent and some Democrat pollsters however all of their results are supposed to be scientific the idea is dose a poll for internal consumption really help if tells you that you are going to win easily on election day only to have to be a landslide against you?The answer is no.

The reason why this is dangerous is multi fold. 1) Due to the supposed scientific nature it has been used to make public policy decisions 2) It can influence peoples opinions. 3) It can influence a senator's or some other politicians choices while they are in power.

Here is a perfect example of this. A certain Republican senator from Maine is considering if she should support a public option, so she wants to see what the citizens of her state think about the topic. She hires Strategic Vision to do a poll for her. Strategic Vision comes back and says 60% of your state's citizens are against it. She gose "Wow I guess im not supporting that bill" In reality its 60% the other way. From this the senator decides to not support the bill and it dose not pass.

I will be as blunt as possible. I am accusing Rasmussen, Strategic Vision and other Republican pollsters of deliberately lying to the American people in order to alter the public debate. If you follow the math they have been consistently off for years. If you want to just look at the last election cycle Rasmussen etc all had the results a lot tighter than the results on election day. This could just be poor polling on their part but I will offer exhibit B

Since health care reform has been a topic in the news the difference between the several Republican pollsters and "everyone else" has been steadily growing. I firmly believe that the insurance industry has been paying these pollsters to lower their numbers for the democrats to push them to drop health care reform.

Yes the Democrats poll numbers have been sliding somewhat across the board. However if you look at the data from the Republican sources. They have the numbers significantly different than those of the "Independent and Democratic" pollsters.

Over all I want to say this "dishonest polling" helps no one. It may help push a certain agenda temporarily but It can also cause those who support it to loose elections..... Look at the results from 2008 the REPUBLICAN PARTY IS BEING MISLEAD BY ITS OWN POLLSTERS AND IT IS COSTING THEM ELECTIONS

Re:Why should I care? (-1, Troll)

Anonymous Coward | more than 4 years ago | (#29545873)

I will be as blunt as possible. I am accusing Rasmussen, Strategic Vision and other Republican pollsters of deliberately lying to the American people

Go ahead and be as blunt as you like, no one gives a fuck about you.

Honest people do (0)

Anonymous Coward | more than 4 years ago | (#29545939)

Honest people do "give a fuck." This is a sleezy Republican pollster that is inventing results, and it does matter.

Horseshit (3, Funny)

Saint Stephen (19450) | more than 4 years ago | (#29545597)

I call total, 100%, biased, fuck me up the ass horseshit on this inane accusation. Lies, damn lies, and statistics.

Re:Horseshit (1)

Frosty Piss (770223) | more than 4 years ago | (#29545859)

'We have a call in to our attorney on this and fully intend to take action that will vindicate us.'

Looks like they plan on a retraction from the author...

Re:Horseshit (1)

Keebler71 (520908) | more than 4 years ago | (#29545955)

Maybe their lawyers are better mathematicians than the guys who did the polling and thats what they mean by "vindicate us" ... oh nevermind...you're right.

NASA story here (-1, Offtopic)

WindBourne (631190) | more than 4 years ago | (#29545605)

The nasa story does not accept postings right now.

Bush and the neo-cons did a hatchet job on NASA's X-33 and then continually dropped NASA's funding over the entire 8 years. If the dems (and hopefully the entire west) are smart, they will spend a bit today, and get the private space industry going. It is on the verge of being internet 1992. Most importantly, the world needs to start mining in space. The reason is that China is running around buying as much mineral right as they can, taking it off the market (either shutting down the mine, or only allowing the resources back to China), while allowing their own abundant resources to be sold only in China. Basically, if we want to avoid a future world war, then we MUST shoot for space and mining.

Hopefully, Obama, the Dems, EU, Canada, Russia, etc have enough foresight to see this.

META-MODERS and NASA POSTINGS (-1, Offtopic)

WindBourne (631190) | more than 4 years ago | (#29545647)

Meta-moders, There are posts on here that are from the NASA story. The reason is that /. is currently broke (as in a bug, not as in USA debt). Some came here automatically. Mine was posted because I was following the other posters. Please nuke the modders that hit these as off-topic. It should be obvious that they KNEW that /. had issues and some of the posts came here (a post from in front of here told them so). For these modders to be nuking ppl for posting in this area is ridiculous considering the situation.

Re:META-MODERS and NASA POSTINGS (1)

bcmm (768152) | more than 4 years ago | (#29545741)

Most M2s aren't going to read the entire discussion at -1 before M2ing. I know I don't. Sorry.

By the way, the unambiguous word you're looking for is "broken". And please don't say "ppl". This is Slashdot.

Re:META-MODERS and NASA POSTINGS (0)

Anonymous Coward | more than 4 years ago | (#29545795)

First, I DO look at discussions before metamodding. I prefer to understand the context of what is going on.
Second, I PICKED the word broke, I was not searching. ZING.
Third, I am not going to change the ppl to people just because somebody wants to be a nazi. THIS IS SLASHDOT.

Re:META-MODERS and NASA POSTINGS (0, Offtopic)

BitterOak (537666) | more than 4 years ago | (#29545793)

Meta-moders, There are posts on here that are from the NASA story. The reason is that /. is currently broke (as in a bug, not as in USA debt). Some came here automatically. Mine was posted because I was following the other posters. Please nuke the modders that hit these as off-topic. It should be obvious that they KNEW that /. had issues and some of the posts came here (a post from in front of here told them so). For these modders to be nuking ppl for posting in this area is ridiculous considering the situation.

I disagree. If a posting ends up linked to the wrong story it is off-topic. It doesn't matter whether it is the fault of the poster or a bug in the Slashdot code, off-topic is off-topic, and should be modded as such. The purpose of modding is for the benefit of the readers, not the posters!

Re:META-MODERS and NASA POSTINGS (0)

Anonymous Coward | more than 4 years ago | (#29545827)

Mod parent up.

What he said.

MegaDittosâ.

"You're a great American."

Re:META-MODERS and NASA POSTINGS (0)

Anonymous Coward | more than 4 years ago | (#29545949)

Oh yeah,
I am must learn to quit replying to self,
Sincerely
BitterOak.

Re:META-MODERS and NASA POSTINGS (0)

Anonymous Coward | more than 4 years ago | (#29545973)

But replying to an offtopic post does not make one offtopic. (Posting AC because I think I'm the only one who believes in this idea).

Ah ha! (2, Funny)

NoYob (1630681) | more than 4 years ago | (#29545609)

"[W]e categorically deny them and will refute them.

So, which category do they deny? The category of truth or the category of lying?

Re:Ah ha! (2, Informative)

etymxris (121288) | more than 4 years ago | (#29545697)

Not sure if you're trying to make a pun, but "categorical" in this case means "without exception." For example, Kant talks about categorical and hypothetical imperatives. Categorical imperatives you do always without exception (such as never lying, according to Kant anyway). Hypothetical imperatives are what you do based on the situation (CPR is appropriate only when someone is not breathing, for example).

Not statistically significant (0)

pikine (771084) | more than 4 years ago | (#29545611)

Even in the alleged truthful poll, the percentage difference between highest tally and lowest tally is 20%. So what if the Pollster tallies have 57% difference? I feel that their threshold between "random" and "totally not random" is simply arbitrary. Who says that 20% is okay, and 57% isn't?

Re:Not statistically significant (2)

plague911 (1292006) | more than 4 years ago | (#29545887)

When you are making decisions based on public opinion and the differance between 52% and 48% makes the difference between whether you keep your elected position. Imagine what the difference would be between 60% and 40%. I'm not sure of the exact reasoning for these kinds of polls 20% seems to be about the stranded margin of error. I imagine it has some aspect of what the state of the art is in scientific statistic estimation theory is. In which case 57% to 20% difference would like using 1850's technology compared to the technology we will have in 2010. At the very least Strategic Vision is run by idiots. If not they are intentionally misleading the public.

Re:Not statistically significant (5, Informative)

ceoyoyo (59147) | more than 4 years ago | (#29546021)

First, the example he gives where he looks at polls from ALL sources is an example of a plausible distribution of real results because, assuming the majority of pollsters are not cooking their data, the data should be dominated by randomness. He then looks at this particular pollster and finds a much greater disparity in trailing digit frequency. The question is, is it significant, or just chance?

Given the numbers, it's not particularly hard to figure out. You can calculate the likelihood of any particular result given a theoretical distribution using a G test of goodness of fit. Technically for numbers this small you could use an exact test but I don't know of a web version and I'm too lazy to write one up. But here's a description of, and an excel spreadsheet that performs, the G test of goodness of fit: http://udel.edu/~mcdonald/statgtestgof.html [udel.edu]

Basically, you plug in the distribution you see and compare it with the one you expected. What you get is the probability of that distribution occurring by chance. So if we plug in the observed data for all the pollsters and assume equal likelihood for all trailing digits we get a p=0.006. Whoops, looks like our assumption isn't quite correct. As the blog author notes, the observed distribution is humped a little, favouring the middle numbers. He also gives a possible explanation. For giggles, the probability of the Strategic Vision results given equally probable trailing digits is absolutely microscopic: p=1.44x10^-17. Together those tell us that our assumption of equal digit distribution is probably not quite right, but the Strategic Vision data still looks mighty funny.

Okay, so assume instead that most pollsters aren't making up their numbers. Not that their numbers are necessarily accurate, but that they're at least not making them up off the top of their heads. So using the data from all pollsters as a template, how likely is the Strategic Vision distribution? That's a G test of independence: http://udel.edu/~mcdonald/statgtestind.html [udel.edu] . We could use Fisher's exact test, but I can't find one that will do a 2x10 table.

Plugging in the data, we get G=43.068, d.f.=9, which gives p=2.09x10^-6. The blog author was actually a little careless when he said the chances of Strategic Vision's results are millions to one against. If you insist on the equal-probability theory then the odds are 70 quadrillion to one against Strategic Vision and 166 to one against the industry as a whole. Taking the more realistic approach that the industry average is a better representation of the actual probability, the odds against Strategic Vision's results are about half a million to one against. Not millions to one, but close enough.

So what? (4, Funny)

msauve (701917) | more than 4 years ago | (#29545617)

Polls show that 78.6% of all statistics are made up on the spot.

you can't believe anything anymore (2, Funny)

commodoresloat (172735) | more than 4 years ago | (#29545677)

Agreed. Who is this Math guy anyway? Perhaps it's Math who faked the results, and Pollster is beyond reproach!

Too many 7s and 8s? (0)

etymxris (121288) | more than 4 years ago | (#29545683)

This "respected blogger" seems to be doing numerology. Take any data set and you'll find patterns that are statistically impossible. This is because you're asking the questions after you have the data at hand. It's like placing a bet at the Kentucky Derby after the race rather than before.

Re:Too many 7s and 8s? (3, Informative)

HornWumpus (783565) | more than 4 years ago | (#29545757)

Take any data set and you'll find patterns that are statistically impossible.

Not if you understand statistics.

Also note: If you understand statistics you would _never_ use the phrase 'statistically impossible'

Re:Too many 7s and 8s? (1)

etymxris (121288) | more than 4 years ago | (#29545987)

I've done more statistics than you give me credit for. And "statistically impossible" is a manner of speech. Anyway, here's what I'm talking about:

What is the probability that the first result will be a1, the second a2, the third a3, and so on for some given set of constants a1...an? Well, the probability that the results measured will be those exact values is pretty much impossible, statistically speaking. Well, it is if you're asking the question before you actually do your experiments. But if you have your experimental results at hand, you can ask that question, having the benefit of having the results right in front of you, and make the results seem implausible. Of course, that example is very simplistic. In any set of data, there is some pattern that, a priori, is nearly impossible to obtain. But there are an uncountable number of such patterns. So for every data set, you can find some pattern that shouldn't happen.

Re:Too many 7s and 8s? (2, Informative)

blueskies (525815) | more than 4 years ago | (#29545779)

If i take any data set (say one with a standard distribution), how many of those data sets would i have to sample on average before i found one that looked like the ones he is talking about? If the expected number of data sets i would have to look at is in the millions, you are correct in that i might find it in my first sample, but the chances are incredibly tiny.

Re:Too many 7s and 8s? (1)

ceoyoyo (59147) | more than 4 years ago | (#29546031)

If you take one with a uniform distribution then you would expect to find one with a greater or equal disparity to the one observed once every 70 quadrillion. If you take a distribution corresponding to the industry average, you'd get a result disparity greater than or equal to strategic vision's, on average, one time in about half a million.

I worked it out above. ;)

Re:Too many 7s and 8s? (4, Informative)

evanbd (210358) | more than 4 years ago | (#29545811)

Fortunately, there are corrections you can do for that. And he took a fairly normal statistical test on the numbers, which is equivalent to saying he didn't perform that many comparisons. To very rough approximation, you need to correct your p-value for all the less weird analyses you might have performed on the data instead. It's a bit hard to pin down an exact p-value for the analysis he did (the underlying data isn't expected to be flat; it's also not expected to be that bizarrely lumpy), but I promise that Nate Silver has an understanding of this issue (which you'd see, if you'd read the post).

Re:Too many 7s and 8s? (1)

ceoyoyo (59147) | more than 4 years ago | (#29546039)

If you accept his initial theory that the digits should be equally probable then it's a multinomial exact test or a G test of goodness of fit. If you observe, as he did, that the industry average supports a slightly different distribution then you can compare SV's results with the industry average using a Fisher's exact or G test of goodness of fit. They're simple tests, and no corrections are necessary unless you do multiple comparisons, which is not the case here.

Re:Too many 7s and 8s? (0)

Anonymous Coward | more than 4 years ago | (#29545921)

Nate Silver is a statistician. It's his job to know statistics, and he knows them far better then you do. He knows what is "placing your bet after the race" and what is "proof of foul play".

What's wrong with this data? (2, Interesting)

klapaucjusz (1167407) | more than 4 years ago | (#29545709)

I'm not sure I understand what Silver is claiming about the data.

He shows that the distribution of second digits in the results of Pollster's polls doesn't follow a uniform distribution -- and from that he somehow deduces it's not random.

If you look at the figure in the second article, it looks to my untrained eyes like a gaussian curve with maximum around 8 -- since when are gaussians not random?

Re:What's wrong with this data? (1)

klapaucjusz (1167407) | more than 4 years ago | (#29545727)

Just to be clear -- you cannot expect second digits in what are two-digit results to be uniform. You can expect fourth digits to be uniform, but that data is not available.

Re:What's wrong with this data? (0)

Anonymous Coward | more than 4 years ago | (#29545897)

Can you be more clear on which distribution 2nd digits should have? So far you said
- uniform: no
- gaussian centered at 8: ok

This is a bit vague.

Re:What's wrong with this data? (1)

klapaucjusz (1167407) | more than 4 years ago | (#29546005)

It is reasonable to assume that non-significant digits will be uniform, given a sufficiently large sample. On the other hand, it is not reasonable to expect that mere second digits will be uniform in data that is as highly biased as poll results.

In other words, I don't expect any particular distribution, but I don't believe that the mere presence of a non-uniform distribution is enough to prove wrong-doing.

Re:What's wrong with this data? (0)

Anonymous Coward | more than 4 years ago | (#29545933)

Exactly my thougt, followed by realizing he doesn't quite grasp the meaning of the data, nor statistics itself. He is following a conclusion from a probability. I don't quite understand how that works...

Re:What's wrong with this data? (2, Informative)

Johnny Loves Linux (1147635) | more than 4 years ago | (#29546003)

> since when are gaussians not random?

That's exactly the problem he's pointing out. The second digit should be a UNIFORM distribution if it came from real data. If the digits are gaussian that indicates that either

  • there's some process accounting for a gaussian distribution that he doesn't know about (and he does consider that possibility) or
  • the numbers are cooked by a human being who has a preference for 8's over other digits.

improbable (3, Interesting)

drDugan (219551) | more than 4 years ago | (#29545713)

Reading TFA, Nate's analysis implies that there is a systematic bias toward some last digits in the overall poll percentages aggregated over many disparate topics.

What seems so improbable (to me) is that if someone really were grossly "cooking the books" like this - literally not doing the poll, or tallying any numbers at all, but instead simply reporting fake results for press ... is that they would be so stupid to make up the results manually instead of using a computer in some way. What, some guy in an office reading other polls and saying "gee I think the number will be 45%."

If this kind of bias really has been introduced by manually creating and publishing the results (as the analysis seems to imply), then it will be easy to track down and prove with further digging into the data, interviewing people who made the calls or took the data, etc. However, accepting such an explanation would requires a level of stupid on the part of the principals in this company that is so extreme that I find such a scenario an improbable explanation for the results presented.

Re:improbable (2, Insightful)

cptdondo (59460) | more than 4 years ago | (#29545843)

But here's the deal:

You do the poll. You have to; you can't just make up the numbers. Sooner or later someone would figure out you don't have a phone bank.

But the poll numbers come up as 46 for, 43 against, and the rest undecided.

Now you can't go and say, 98 for, 1 against, and 1 undecided; that's what the communists do and everyone knows they're lying.

But you report it as 47 for, 42 against, and the rest undecided. Now you've falsified your data, but you think in a way that's hard to catch. You bump the numbers one or two or three points in favor of your position.

However, I'm unconvinced that this is some sort of smoking gun; Silver needs to really run this sort of simplistic analysis on a lot of other polls and see if there in fact is a bias towards a 47 - 43 split with 10% undecided. That actually sounds about right for a lot of the polls I remember in the last election.

Re:improbable (2, Informative)

Silentknyght (1042778) | more than 4 years ago | (#29545905)

However, I'm unconvinced that this is some sort of smoking gun; Silver needs to really run this sort of simplistic analysis on a lot of other polls and see if there in fact is a bias towards a 47 - 43 split with 10% undecided. That actually sounds about right for a lot of the polls I remember in the last election.

If you read the TFA, Nate addresses this. He states that his data--SV LLC's polling results--are selected from a wide, wide, wide variety of topics, not just necessarily the highly divisive ones where there may be a relatively even split between two choices.

Moreover, (as Nate states) over enough data, even the effect of the undecided percentage on the trailing digit should be random.

Re:improbable (1)

klapaucjusz (1167407) | more than 4 years ago | (#29545979)

Moreover, (as Nate states) over enough data, even the effect of the undecided percentage on the trailing digit should be random.

Except that in this case, the trailing digit is merely the second digit. A bias in the second digit of what is after all highly biased data (you don't have a lot of 98-2 results in polls) is not unlikely, even in samples much larger than what he's using. Not saying that the company is honest, but Silver's argument is not sufficient to condemn them.

Re:improbable (1)

timmarhy (659436) | more than 4 years ago | (#29545917)

all i get out of reading the article is that silver has a bee in his bonnet and doesn't like the firm in question. anyone who's done a statistics course knows numbers can be twisted and played with to come out with just about any answer. i'd be very suprised if ALL pollsters do this.

Re:improbable (1)

ethergear (1130483) | more than 4 years ago | (#29545957)

The first graph is prefaced by "...Here, for instance, is what I get if I run the numbers for all Senate and Presidential polls -- more than 3,000 (!) of them -- in my 2008 database:" Nate might have been more explicit about the difference in datasets, but I think this indicates that he analyzed similar data from other pollsters.

Re:improbable (1)

ceoyoyo (59147) | more than 4 years ago | (#29546045)

It is a PR firm.

Handwaving math. (3, Informative)

Gorobei (127755) | more than 4 years ago | (#29545725)

Nate Silver does great analysis at the first order multiple-linear-regression level -- he outperformed all the other polls/predictors in 2008 iirc.

He sucks at meta-analysis though, in that he just doesn't understand the math. His 2008 monte-carlo stuff gave good results, but was just a bad reinvention of averaging. His recent foray into analyzing stock returns was interesting but 0-information (i.e. useless.)

Now he's mentioning Benford's law, but playing with trailing digits. Then he handwaves a non-normal result with an appeal to "it looks wrong." Come on, give us some real math here!

That said, he's probably right, but he's given us no math to support his claim.

Re:Handwaving math. (1)

ceoyoyo (59147) | more than 4 years ago | (#29546053)

I did it: http://news.slashdot.org/comments.pl?sid=1382853&cid=29546021 [slashdot.org]

It's not quite as unlikely as he says, (half a million to one instead of millions to one) but Strategic Vision is almost certainly sampling something that is not what the rest of the industry is sampling.

There are lies, damn lies... (1)

gmuslera (3436) | more than 4 years ago | (#29545731)

... and pollster's statistics

Use stats, not laws (2, Informative)

unlametheweak (1102159) | more than 4 years ago | (#29545773)

Their response to Silver's accusation? 'We have a call in to our attorney on this and fully intend to take action that will vindicate us.'"

Generally, I would expect a logical course of action from an honest and transparent firm would be to hire a statistician to vindicate themselves. Lawyers don't make a reputable firm appear any less reputable.

Re:Use stats, not laws (2, Insightful)

Frosty Piss (770223) | more than 4 years ago | (#29545883)

Lawyers don't make a reputable firm appear any less reputable.

Lawyers don't make a reputable firm appear any more reputable.

Strategic Vision, LLC. *not* Strategic Vision, Inc (1)

dave-on-the-dot (734701) | more than 4 years ago | (#29545857)

I think it would have been nice for the poster to indicate that this refers to Strategic Vision, LLC., and *not* Strategic Vision, Inc., since Nate Silver specifically suggests that they may be trying to play off the credibility of the latter.

The Dishonest Casino and Hidden Markov Models (1)

hugg (22953) | more than 4 years ago | (#29545877)

There's not enough eigenvectors in this thread...

http://controls.engin.umich.edu/wiki/index.php/Occasionally_dishonest_casino:_crimes_or_just_noise%3F

Sue Everybody! (1)

Tehrasha (624164) | more than 4 years ago | (#29545919)

'We have a call in to our attorney on this and fully intend to take action that will vindicate us.'

By all means, dont back up your numbers with mathmatical proof of your own.
I cant wait to see how they try to sue mathmatical laws and formulae.

Protip: Winning a lawsuite doesnt make you any less a liar.

The other possibility... (0)

Anonymous Coward | more than 4 years ago | (#29545985)

Is that the accusing organizations are cooking their books to make their polls come out to statistical dead heats to encourage their clients to buy more subsequent polls...

The closer poll results are, the more polls are taken, plain and simple.

If Gallup polls and finds McCain 45, Obama 44, undecided 11, then you can be you're butt that both sides are going to request a new poll the next day.

If Gallup polls and finds McCain 48, Obama 37, undecided 15, then chances are it'll be a few days before they pay for another poll..

So, it is in the Pollsters' best interest to have bunching around 4 and 5 as described in TFA, and therefore they are the ones who are probably cooking the books.

What I'd like to see (1)

rcolbert (1631881) | more than 4 years ago | (#29546019)

I'd like to see the trailing digit distribution for three or more other polling firms for the same period and with a comparable amount of data points. At first glance, non-random distribution looks pretty bad. I'm curious to see if other firms demonstrate random distribution as expected.

Push Polls? (1)

oljanx (1318801) | more than 4 years ago | (#29546055)

It's not uncommon for polls to be conducted in a less than objective fashion. For example, a pollster might play a series of carefully selected audio clips from a political debate, which are designed to make one candidate look better than the other, and then ask the subject their opinion of the candidates. The goal is to "push" an opinion on the subject rather than collect information. If a push poll is successful, the data is going to be skewed. And after applying Benford's Law you're probably going to see a lot of 7s.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>