Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Web Geniuses Or Web Dimwits?

Zonk posted more than 7 years ago | from the nothing-but-geniuses-here dept.

164

ScribeCity writes "The Washington Post has a provocative piece about online experiments at identifying experts. One wonders when someone will come up with a truly effective formula for measuring human intelligence — or take a stab at doing so — that exploits all the stuff people are publishing online." From the article: "This wisdom of the crowd could be outsmarted by what Michael Arrington, editor of the TechCrunch blog, recently dubbed the 'wisdom of the few.' Sites like PicksPal rely on input from the masses chiefly as a venue for auditioning prospective experts, on the theory that these virtuosos could provide even more accurate information and predictions than the crowd. 'If you figure out which ones did the best and get rid of the ones who have no idea, you'd do even better. Distill it down to the people who really know,' Arrington said."

cancel ×

164 comments

Or... (5, Funny)

jo42 (227475) | more than 7 years ago | (#16506525)

Just get a chimp to throw darts at the wall...

Re:Or... (4, Insightful)

UbuntuDupe (970646) | more than 7 years ago | (#16506821)

If that chimp picked stocks that way, it would beat something like 80% of mutual fund managers (simply by virtue of not charging for his essentially random results).

(I know, I know, "but not my mutual fund!")

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16507425)

it would beat something like 80% of mutual fund managers
I think you meant 50% of managers.

Re:Or... (1)

UbuntuDupe (970646) | more than 7 years ago | (#16507561)

No, I meant 80%. They actually do worse than randomness once you subtract their fees. Well, even before that.

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16507963)

Do you have anything to back up that claim, or do you actually have a chimp throw darts at the roughly 9000 companies on the NYSE and NASDAQ alone? What about OTC? That would be a big dart board.

Re:Or... (1)

UbuntuDupe (970646) | more than 7 years ago | (#16508127)

The Index Funds Advisors [ifa.com] site pretty much sums it up.

Re:Or... (1)

Canthros (5769) | more than 7 years ago | (#16508209)

In fairness, I'd think that's partly because the Index Funds Advisors would like for you to invest in their index funds, instead of somebody else's mutual funds.

(But, yes, mutual funds need to be picked very, very carefully.)

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16508473)

It makes sense to get info from a seller of Index funds about how bad other funds compare.

Re:Or... (2, Interesting)

UbuntuDupe (970646) | more than 7 years ago | (#16508661)

I didn't send you there expecting to trust their assertions. I sent you there because they compile all the research there and source it, which you can independently verify, such as the Malkiel's A Random Walk Down Wall Street. I'm really not asserting a bold thesis here; any independent source on the matter not after your money (consumer advisor Clark Howard, the various academic researchers they list) and some that aren't independent (Warren Buffet and Peter Lynch) will tell you the same thing.

Or, go to MarketWatch [marketwatch.com] and run your favorite mutual fund against the S&P 500 for its life.

Yes, IFA explains the superiority of index funds, but their arguments apply to any index funds, not just theirs.

The person who you shouldn't trust is the John Edward-type mutual fund salesman who will erase the fund family's misses, show you the hits, and say, "See the ones that outperformed? That was because of great management, really! Now, cough up the cash."

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16508733)

run your favorite mutual fund against the S&P 500 for its life
Now we're getting down to it. You didnt say "mutual fund vs. S&P", you said "stock picks vs. mutual funds". There is a BIG difference. S&P is large cap and diversified. Random stock picks are exactly that: completely random. When you throw half truths out there you are essentially spreading lies.

Re:Or... (1)

UbuntuDupe (970646) | more than 7 years ago | (#16508797)

No, I didn't lie; the S&P example I gave was just to show the superiority of passive management. This page I gave you covers documented experiments where they compared active manager performace to that of random stock picking, and it doesn't bode well for active managers.

Now, do you want to keep trying to get a tiny face-saving victory, or actually learn something?

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16508863)

I'm still trying to get where you found the 80% number. That would learn me but GOOD!

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16509001)

Is this [investopedia.com] maybe where you got it?

Re:Or... (1)

UbuntuDupe (970646) | more than 7 years ago | (#16509207)

Dude, how about instead of roaming all over google and heckling me, you maybe start reading the 12 steps [ifa.com] thing?

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16509309)

OK, thanks. I will.

Re:Or... (4, Insightful)

OakDragon (885217) | more than 7 years ago | (#16508231)

The Wall Street Journal has (had?) a "Dartboard" feature [informit.com] in which they did just this, then compared the picks to choices made by analysts. Depending on the time-frame you're looking at, just random choices seem to give the analysts a run for their money, as it were.

Re:Or... (1)

FesterDaFelcher (651853) | more than 7 years ago | (#16508597)

Thanks OakDragon. UbuntuDupe: Read the last paragraph, the DJIA, (an INDEX) pounds the darts and the analysts.

Re:Or... (2, Insightful)

UbuntuDupe (970646) | more than 7 years ago | (#16508841)

Okay, so an index beats darts and analysts.

Now, remind me, how does that contradict my claim that darts beat analysts (or rather the "experts" at mutual fund management)?

And where did I disparage index funds?

Oh right -- that face-saving victory you were looking for.

Better indicator... (3, Insightful)

R2.0 (532027) | more than 7 years ago | (#16507683)

If they are posting stuff on the Internet and sound like an expert, they are most likely NOT an expert.

Re:Or... (0, Offtopic)

nickos (91443) | more than 7 years ago | (#16509123)

see also digg [digg.com]

Why didn't they test Slashdot? (5, Funny)

garcia (6573) | more than 7 years ago | (#16506555)

In order to effectively determine the rate of experts vs. everyone else, you could simply scan through all previous Slashdot posts (while removing those prefaced by IANAL) and easily determine those that are experts.

Make sure you are browsing at -1, *those* people are the real experts ;)

IANAL, but (4, Funny)

RealProgrammer (723725) | more than 7 years ago | (#16506641)

... you sound like you could be an expert.

Re:Why didn't they test Slashdot? (2, Insightful)

Quiet_Desperation (858215) | more than 7 years ago | (#16506643)

Make sure you are browsing at -1, *those* people are the real experts ;)

You may joke, but these days anyone who questions the current pseudoscience-dogma-of-the-month tends to get modded -1 when they interject facts into the discussion, so you're not that far off.

Re:Why didn't they test Slashdot? (0)

morgan_greywolf (835522) | more than 7 years ago | (#16506963)

You may joke, but these days anyone who questions the current pseudoscience-dogma-of-the-month tends to get modded -1 when they interject facts into the discussion, so you're not that far off.


Hi! Welcome to Slashdot! You must be new here! :-)

Re:Why didn't they test Slashdot? (0)

Anonymous Coward | more than 7 years ago | (#16508921)

No, I'm new here. ...

I miss that guy. =/

Re:Why didn't they test Slashdot? (2, Interesting)

Mr. Underbridge (666784) | more than 7 years ago | (#16507879)

You may joke, but these days anyone who questions the current pseudoscience-dogma-of-the-month tends to get modded -1 when they interject facts into the discussion, so you're not that far off.

You know, that used to be more true. Then Digg came along and took away all the morons. Go check out some of their flamebait stories (politics would be a good start). If you don't echo the group view, you will be modded into oblivion. However, statements like "Bush is teh stupid!" will actually get modded up. Modding is definitely done by sentiment more than any actual insight.

So thanks Digg, for making slashdot better!

Re:Why didn't they test Slashdot? (1)

Cyclometh (629276) | more than 7 years ago | (#16506813)

Maybe there's some way of harnessing /. moderation (and meta-moderation) data to get a candidate of likely experts.

Re:Why didn't they test Slashdot? (3, Insightful)

buswolley (591500) | more than 7 years ago | (#16507781)

NO..What you'd be selecting for are those who find the first possible place to post..

Moderation(negative or positive) depends mainly on its position in the discussion. If you manage to post near the beginning of the page, you will get moderated.

Re:Why didn't they test Slashdot? (2, Interesting)

cultrhetor (961872) | more than 7 years ago | (#16506835)

Places like /. are the basis for these sorts of developments - user-moderated, information-recommendation boards that rank opinion and content based on a number of criteria. Although a number of boards like this one fail or become shills (ePinions), those that survive are models for social recommendation researchers. Discourse analysis is a peculiar human trait, one that computers cannot (yet) accurately perform because our communicative practices are situated in unique, perspective-based contexts, so I'll be interested to see what develops.

Re:Why didn't they test Slashdot? (0)

Anonymous Coward | more than 7 years ago | (#16507205)

In reality there are always going to be 11 groups.

Those who know, those who guess, and those who don't get it.

Never happen (4, Insightful)

Quiet_Desperation (858215) | more than 7 years ago | (#16506581)

One wonders when someone will come up with a truly effective formula for measuring human intelligence

It won't happen, not because it's not possible, but because some group or another will have a lower mean score, and the cries of racism, sexism, ageism, redbluestateism, culturalism, OSism, haircolorism, footsizeism, dicksizeism, or whateverism will drown out the truth.

You know... the way it is right now.

Re:Never happen (2)

ClosedSource (238333) | more than 7 years ago | (#16506689)

I'm sure real scientists wish they could blame the failure of their theories on political correctness too.

Re:Never happen (0, Offtopic)

nine-times (778537) | more than 7 years ago | (#16506867)

Also, it certainly won't happen until we come to a common idea of what "intelligence" is. I've met plenty of people who are good at math who can't string a sentence together. I've known people who are good at math and writing, and just do the stupidest things. So what's intelligence made up of? Quick computer-like operations? Clear thinking and deep analysis? Good judgement?

It seems to me that many people have various levels of all sorts of different mental capabilities, all of which we lump into "intelligence", without really distinguishing. Any test you come up with will arbitrarily choose a subset of these capabilities and rate them to an arbitrary level of importance, and that's the best-case scenario, assuming everything is accurate. And what about people who are lacking specifically in the mental capabilities that make a person a good test-taker?

Different "intelligences" (4, Interesting)

blueZ3 (744446) | more than 7 years ago | (#16507277)

My mother is a professor of education at a college in California and this is something that educators have been talking about for a long while. Google for "multiple intelligences" for a lot more information, but basically there's a theory that says that "intelligence" can be divided up into a number of categories and that people tend to excell in one or two of these areas, but few are outstanding in all of them.

The standard breakdown is something like:

  • Linguistic intelligence
  • Logical-mathematical intelligence
  • Spatial intelligence
  • Bodily-Kinesthetic intelligence
  • Musical intelligence
  • Interpersonal intelligence
  • Intrapersonal intelligence
  • Naturalist intelligence

Atheletes tend to excell at Spatial and Kinesthetic, while the stereotypical geek is strongest in Logical-mathematical and weakest in Interpersonal.

I'm not sure I completely agree that this is the end-all-be-all for understanding intelligence, but it does provide an interesting look into ways to classify people who might not be "book smart." For instance, a terrific ballerina might not have excellent Interpersonal or Linguistic intelligence, but she certainly has some special "intelligence" that allows her to excell in an area where I would certainly be an abject failure.

I encourage anyone interested in this idea of multiple intelligences to poke around and do some research. Again, it may not be the final answer, but it provides an interesting framework for thinking about the topic.

Re:Different "intelligences" (3, Interesting)

porcupine8 (816071) | more than 7 years ago | (#16507395)

The problem is, this theory has been around for over 20 years (Howard Gardner's "Frames of Mind" in the mid-80s) but in that time, there has been zero empirical evidence for it. Whereas theories based on "g", or the general factor of intelligence that IQ tests *try* to measure (but do so with varying levels of accuracy) has about a century of pretty solid data behind it.

Re:Different "intelligences" (4, Interesting)

Anonymous Coward | more than 7 years ago | (#16508409)

The biggest problem with this categorization is that it's simply politically correct nonsense. Unfortunately, some people are overly swayed by the positive connotations of the word "intelligence" to the point that they think you can have no human value if you're not "intelligent". Gifted athletically, even if you're not smart? You must be "bodily-kinesthetically intelligent". Talented musican or artist? You must be "musically intelligent" or "chromo-visio-spatially intelligent". The word "intelligence" gets smeared around so many different categories that it loses any meaning beyond more general words like "talent" or "ability".

The problem here isn't that intelligence tests don't let musicians score well. IQ tests measure what they measure just fine. The problem is the value system of people that want so badly for everyone to be "intelligent" that they have to change the definition of the word.

The correct response is to realize that intelligence simply isn't all that. (I know this point will be a tough sell on Slashdot, where intelligence is particularly valued.) It's perfectly ok not to be "intelligent", and play to your other strengths.

Identifying those strengths and weaknesses is important in being able to choose activities in which you'll be successful, or at the very least in being realistic about the extra hurdles you're going to face. There's a reason I didn't become a musician or an NBA star, and it wasn't lack of "intelligence". That, I manage just fine, but I'm sadly lacking in other talents that people value. However, the self-esteem-uber-alles crowd picked the word "intelligence" to glorify, rather than say "athleticism", which is why people are trying to force-fit abilities into names like "bodily-kinesthetic intelligence" instead of pounding the round peg into the square hole and trying to console me that I have "mental athleticism" or "cerebral coordination".

If you call his tail a leg, how many legs does a three-legged dog have?

Re:Different "intelligences" (2, Insightful)

jackbird (721605) | more than 7 years ago | (#16508647)

No, the problem is in defining "intelligence" at all (who's smarter, Shakespeare or Newton?), particularly in any way that isn't heavily reliant on context or culture. As an example, one of the subtests in the Wechsler IQ test (the most widely used one) involves providing verbal definitions for a series of cards with pictures on them. In an early version of the test, "helicopter" was scored as a fairly advanced card. Once the Vietnam war brought images of helicopters into the news on a daily basis, the validity suffered.

Re:Different "intelligences" (1)

t0rkm3 (666910) | more than 7 years ago | (#16508625)

I would hesitate to categorize athletic capability as "intelligence". As a highly ranked athlete in several disciplines, it takes more of a "lack of intelligence". You have to be able to switch off the analytical mind and allow your body to naturally control itself.

Body awareness and movement memory is not a conscious controllable aspect of the nervous system. Thus categorizing it as intelligence seems a bit of a leap. Do we categorize felines as kinesthetic geniuses because of their sense of balance?

Re:Never happen (3, Insightful)

porcupine8 (816071) | more than 7 years ago | (#16507347)

See, people have actually thought about these issues, you just don't hear about it in the media. (Except Gardner's Multiple Intelligences theory, which has zero data to back it up.) You should work into the work of John Carroll, and his hierarchical model of intelligence.

Re:Never happen (1)

nine-times (778537) | more than 7 years ago | (#16507843)

No, I know, but these things are still a bit problematic. Let's say I come up with a theory where there are 4 intelligences and you come up with another where there are 4, but the 4 are different. Or let's say another guy comes up with 5. How do you know you're capturing all the different ways of being "intelligent". How do you know for sure that your groupings/divisions are the best? And finally, how do you compare a score on interpersonal skills to a score on a mathematical skill? I think these decisions are going to be fairly arbitrary no matter what, even if there's *some* good sense or science behind it.

Re:Never happen (1)

porcupine8 (816071) | more than 7 years ago | (#16509075)

I doubt it'll ever be truly settled in a "hard" manner until we can put you in a scanner and read out some numbers as to your ability to do X, Y, and Z - but I think the field is a bit more mature than you give it credit for. As I said, it just doesn't get out into the mainstream media that much, because if anyone were to think that IQ tests actually have some *gasp* predictive validity, then all of society will come crashing down around us. Or so many people would seem to have you think.

Re:Never happen (0)

Anonymous Coward | more than 7 years ago | (#16507257)

It won't happen, not because it's not possible, but because some group or another will have a lower mean score, and the cries of racism, sexism, ageism, redbluestateism, culturalism, OSism, haircolorism, footsizeism, dicksizeism, or whateverism will drown out the truth.

Actually, like it or not those groups are basically correct. Any measure of "intelligence" that isn't based on physiology (e.g. a blood test, brain scan, midi-chlorian levels, etc) is automatically biased by those who are establishing the baseline. Which is a better measure of intelligence, the ability to cite the birth city of Bach, or to know how to determine if a small body of water is safe to drink? Depends on the context.

Re:Never happen (1)

jamboarder (620309) | more than 7 years ago | (#16507451)

One wonders when someone will come up with a truly effective formula for measuring human intelligence

It won't happen, not because it's not possible, but because some group or another will have a lower mean score, and the cries of racism, sexism, ageism, redbluestateism, culturalism, OSism, haircolorism, footsizeism, dicksizeism, or whateverism will drown out the truth. You know... the way it is right now.

Cries of those "isms" come less from political correctness and more from arbitrarily quantifiable definitions of 'intelligence'. May as well embark on the quest for a truly effective formula to measure "prettiness" or "goodness".

Re:Never happen (1)

COMON$ (806135) | more than 7 years ago | (#16508255)

Dont know, experts exchange does a pretty good job of weeding people out. Then again it is kind of like a MMoG in that you can rack up points pretty fast by just farming all day.

An Exploration of Truthiness... (0)

Anonymous Coward | more than 7 years ago | (#16506601)

Stephen Colbert would be proud.

Perverse Incentives (2, Insightful)

Tackhead (54550) | more than 7 years ago | (#16506613)

> 'If you figure out which ones did the best and get rid of the ones who have no idea, you'd do even better. Distill it down to the people who really know,'

...and then disappear everyone who knows what they're doing, so you can hire clueless sycophants whose loyalty can be guaranteed.

A sword cuts both ways, after all. I fear this tech.

Re:Perverse Incentives (3, Funny)

morgan_greywolf (835522) | more than 7 years ago | (#16507187)

...and then disappear everyone who knows what they're doing, so you can hire clueless sycophants whose loyalty can be guaranteed.


Ooh! Just like U.S. Federal Government! Good idea!

Simple (4, Insightful)

CrazyJim1 (809850) | more than 7 years ago | (#16506623)

Make a group based moderation system, where you moderate in groups.

Step1: Lets say Democrat/Republican. When a Rep mods something up, all other Reps see it modded up. If a Dem mods something down, no other Reps see it modded down.

Step2: Identify posters who say stuff that gets modded up past a certain point. Lets say you get a point for the top 10 posts of each day. Then the posters with the most points are dubbed experts in their field.

Its simple, and I'm suprised no one has done it before. It's like Digg in some ways, but vastly superior as groups don't bicker over what they declare as news, and it identifies experts.... maybe even political candidates.

Re:Simple (4, Funny)

karlto (883425) | more than 7 years ago | (#16507197)

and it identifies experts.... maybe even political candidates

I wasn't aware that it was possible to be both of those

Re:Simple (2, Interesting)

rthille (8526) | more than 7 years ago | (#16508983)


Doesn't that just lead to group-think?

Part of why I read slashdot is for the (slightly) alternate viewpoints.

:o\ (2, Insightful)

TubeSteak (669689) | more than 7 years ago | (#16506631)

I'm not an 'expert' in anything, yet I read far and wide enough to pick up lots of random & indepth tidbits that 'experts' have not heard about.

Ever heard the joke about the phd professor who studied more and more about less and less, until he knew everything about nothing? Yea, many people would consider that professor an expert.

Re::o\ (3, Funny)

supersnail (106701) | more than 7 years ago | (#16506945)

If about nothing he everything knew then truly the force would be with him!

Not a Bad Idea (3, Interesting)

AKAImBatman (238306) | more than 7 years ago | (#16506633)

There is a certain logic to this. How many times have "experts" told us screwy nonsense, and had lousy track records [dvorak.org] , and yet the public at large retained them as experts? Sometimes, the untrained may be able to see things that the supposedly well-trained can't.

Or to put it another way, it all becomes a set of probabilities. If person X has guessed the outcome of something (say, a football game) correctly 80% of the time, then you're safer betting on his predictions than you are betting on expert Y who is only correct 30% of the time. If you aggregate the probabilities and successes, you should be able to develop a model with a high probability of being correct. You'll never be able to gain 100% accuracy, but that's just the nature of the Universe [wikipedia.org] . ;)

Re:Not a Bad Idea (1)

aafiske (243836) | more than 7 years ago | (#16507159)

Wrong.

Suppose we have everyone in the world guess the outcome of a 100 coin flips. Some fraction would be 100% correct. Repeat a few times. We've now winnowed down a pool of people who are excellent at guessing coin flips, right? It's safer to go with their guess than the guess of someone else, right?

The fact is, _someone_ was going to be mostly right. However, there's nothing special about that person, they just happened to get lucky. Their previous luck does not affect their current predictive powers, which are zero.

It happens in the financial markets a lot. Some analyst gets stuff right for 4 out of 5 years. He's the new star. But ... given all the analysts out there, it'd be shocking if there weren't _someone_ who got those odds, even if they just chose randomly.

Re:Not a Bad Idea (1)

AKAImBatman (238306) | more than 7 years ago | (#16507505)

Suppose we have everyone in the world guess the outcome of a 100 coin flips. Some fraction would be 100% correct. Repeat a few times. We've now winnowed down a pool of people who are excellent at guessing coin flips, right? It's safer to go with their guess than the guess of someone else, right?

Except that you're trying to predict a random event that can't be predicted. When a good baseball team defeats a bad baseball team, there's nothing random about it. When one candidate is elected over another, the answer already exists; it only needs to be tabulated. When a company makes a move in the marketplace, it does so based on the options it has open to it.

These are not random events. Someone who is good at understanding the nuances behind them, CAN predict their outcome. The problem with situations like rising Stock Brokers is that they often rely on abstract models (which may or may not have any meaning) and/or the contacts through which they were getting their info are not as potent as they once were.

Re:Not a Bad Idea (1)

Control Group (105494) | more than 7 years ago | (#16509087)

The problem with your argument is that it denies the very existence of expertise. You could apply the same argument to surgeons performing triple bypasses, and thereby "prove" that you're just as well off asking your mechanic to operate on you as you are going to Johns Hopkins.

All you've really demonstrated is that it's possible to come up with false positives when determining expertise. This is not a surprising answer.

Re:Not a Bad Idea (1)

morgan_greywolf (835522) | more than 7 years ago | (#16507375)

Or to put it another way, it all becomes a set of probabilities. If person X has guessed the outcome of something (say, a football game) correctly 80% of the time, then you're safer betting on his predictions than you are betting on expert Y who is only correct 30% of the time.


Hmmm...seems like the outcome of a football game could be predicted correctly at least 50% of the time if the predictions were chosen at random.

Re:Not a Bad Idea (2, Insightful)

Gospodin (547743) | more than 7 years ago | (#16507909)

Sure, but the picks may not be random. If you accept that someone can be right 80% of the time, then they can be wrong 80% of the time and hence right only 20% of the time. Ironically, a sports gambler who is right only 15% of the time is more valuable than one who is right 75% of the time. You simply hear his picks then do the opposite. It's the one who picks 50% who is informationally worthless.

Re:Not a Bad Idea (0)

Anonymous Coward | more than 7 years ago | (#16508399)

that's why we give all our money to professor pigskin!

Apply the same filtering to government elections (3, Interesting)

c0d3h4x0r (604141) | more than 7 years ago | (#16506653)

'If you figure out which ones did the best and get rid of the ones who have no idea, you'd do even better. Distill it down to the people who really know,' Arrington said.

I've always said that elections should qualify each voter's ballot to make sure the decision is made by the people who are best equipped to decide. The first page of a voting ballot should be a questionnaire that asks simple unbiased questions that require the voter to demonstrate knowledge of who or what they are voting on. "What does candidate X say their stance is on abortion?" "When did you first hear about initiative I-456?" "Please specify which political party each candidate below belongs to", etc. The score a voter gets on their questionnaire would then be used as a "weight" factor when counting their ballot, so that people who know the candidates and the issues better get more of a say, which is clearly how things ought to be.

Re:Apply the same filtering to government election (2, Insightful)

popo (107611) | more than 7 years ago | (#16506737)

Yes, but who decides which issues make the questionnaire?

The questionnaire's authors would in-effect be defining the criteria for election.

Maybe I vote for someone based upon whether or not they annoy the crap out of me.
That's my prerogative.

Re:Apply the same filtering to government election (1)

c0d3h4x0r (604141) | more than 7 years ago | (#16507143)

Yes, but who decides which issues make the questionnaire?

The questions are limited to those that are objective and factual in nature (that's what unbiased means), so it really doesn't matter.

Maybe I vote for someone based upon whether or not they annoy the crap out of me. That's my prerogative.

It shouldn't be, because it's a ridiculous and unsafe basis for choosing a candidate.

A candidate should win because they are the most qualified for the job and best represent the public's views on the issues. They shouldn't win because they are the best looking or the hippest. Therefore your vote should only count if you are voting for a legitimate reason.

Re:Apply the same filtering to government election (1)

Jussi K. Kojootti (646145) | more than 7 years ago | (#16507333)

The questions are limited to those that are objective and factual in nature...

Bollocks. Let's say we have two questions that both are answered correctly by the same percentage of people. It's pretty much guaranteed that different groups of people get those questions wrong. By choosing the right questions you can weigh the elections, even if the chosen questions are "objective and factual"...

Re:Apply the same filtering to government election (1)

c0d3h4x0r (604141) | more than 7 years ago | (#16507693)

Let's say we have two questions that both are answered correctly by the same percentage of people. It's pretty much guaranteed that different groups of people get those questions wrong.

There would have to be a positive correlation between "the group of people who didn't know the answer to question #1" and some other classification (such as "the group of people who are homosexuals"). It would be extremely unlikely as well as extremely difficult to intentionally rig.

Besides, you can solve it completely by requiring that each choice on the ballot get the same number of questions about them in the questionnaire. So if a question asks, "Where does candidate X stand on this issue?", then you'd have to ask the same question for each of the other candidates.

Sure, people are more likely to know where their own candidate stands than the others, but that's fine -- the more questions you answer correctly, the more your vote counts. So if you only know about your own candidate, your vote counts more than someone who doesn't know anything about any candidates, but your vote counts less than someone who knows about their own candidate and the other candidates too. Everything works out as it rightly should.

I'm sure some people will read this and ask, "But doesn't this discriminate against people who are less educated, or who don't have the time to learn about the issues or candidates?" The answer is yes, of course it does. It's just acknowledging the reality that you can't make a good decision without being informed. It's no different than requiring someone to have eyesight to drive a car -- that policy is intentionally discriminatory against the blind, but who's complaining about that?

Finally, even if there are imperfections with the system I'm suggesting, it would still be light-years better than how the process currently works. Nothing is ever perfect, and you can poke holes in anything, but the system in use now has such obvious problems that it's easy to think up ways to radically improve it.

Re:Apply the same filtering to government election (2, Insightful)

Mister Whirly (964219) | more than 7 years ago | (#16508029)

They have tried this before http://en.wikipedia.org/wiki/Literacy_test [wikipedia.org] and it was flawed.
"The theoretical basis for them was that illiterate persons were not sufficiently informed about the candidates and issues involved to be able to make a truly informed decision. In practice, however, the literacy requirement was often used to prevent those determined by the ruling class to be undesirable, such as the poor, racial and ethnic minorities, and other groups that it wished to see disenfranchised, from voting."
Never trust a system rife with potential for abuse not to be abused.

And how can you have one definitive correct answer to a question like "Where does candidate X stand on this issue?" - it is impossible. Have you seen how long current voting takes at some voting places? Hours long lines and that is with just having to prove who you are, and not how smart you are...

Re:Apply the same filtering to government election (1)

RKBA (622932) | more than 7 years ago | (#16508495)

On the other hand, if they asked factual questions like:
1. What is the hexadecimal opcode for the IBM 360 "Jump" instruction?

I would be perfectly happy with the idea. ;-)

Re:Apply the same filtering to government election (1)

LunaticTippy (872397) | more than 7 years ago | (#16507605)

It'd be easy to tilt the questions. Ask questions that are bullet-points for Focus on the Family (such as candidates' position on "family values") and you give that bloc more weight. Ask Sierra Club questions (position on environmental issues) and that bloc is favored. Ask trigger questions like abortion, gun control, immigration policy and you mainly get extremists on both sides who know these positions best.

It'd be nice to think you're selecting for interested, educated people without bias. I think it'd be difficult to maintain a bias-free question set.

I always thought that it should be sort of hard to register to vote. Take a written and driving test and have to renew every so often. But, there are problems with that approach too. My version of the test would be blatantly unfair, disqualifying any voter who thinks they talk with a diety that has opinions on political policy.

Re:Apply the same filtering to government election (1)

Original Replica (908688) | more than 7 years ago | (#16507919)

The original eligiblity requirements for voting in the US were intended be the Founding Fathers to do just that. Unfortunately they where racist,sexist,and classist. I agree that it is odd that you need a test to be allowed to drive but not a test to vote. Maybe the best would be an unbiased video of a debate between all the people on the ballot that would be required viewing before going to the voting booths.

That's why I read at +5 and use friends (4, Interesting)

davidwr (791652) | more than 7 years ago | (#16506691)

1) Read old threads at +5, new threads at +2

2) If a person has a lot of insightful/informative posts, check their posting history

3) If they are consistently +3/4/5 informative/insightful, add them as a friend

4) add points to friend's posts so they start out +2.

OK, seriously, I don't do that but if I did, I'd see posts of "wise ones" and ignore posts from those that don't make the cut.

Re:That's why I read at +5 and use friends (1)

borawjm (747876) | more than 7 years ago | (#16507447)

The problem inherent to all sites that have a rating or modding feature is that these "groups of friends" form and get more attention and priority than some lone guy (who might actually be well versed in the subject). He won't get heard because these "wise ones" push him off the chart so to speak.



Possible logical fallacy (2, Insightful)

Shimmer (3036) | more than 7 years ago | (#16506771)

Let's say you have a pool of 10,000 prognosticators. You ask each one to pick the winner of 10 football games. The odds of getting all 10 correct are 1 in 2^10=1024. So out of the pool of 10,000 people, by random chance alone you're likely to get about 10,000/1024 = 10 people who pick all 10 games correctly. Are these people "geniuses"? No, they just got lucky during this particular trial. The odds of them getting game #11 correct are just 50-50.

BTW, this can be used as the basis of a scam against the "geniuses" if you can convince them that they have special powers as a result of the trial.

Moral of the story: Be very careful with statistics.

Re:Possible logical fallacy (2, Interesting)

richg74 (650636) | more than 7 years ago | (#16507597)

This is actually extremely similar to the coin-flipping contest described in an appendix to one of my favourite books about the stock market (and other financial markets), Fred Schwed's Where are the Customers' Yachts?. Of the contestants who flip ten heads in a row, he writes, "they are the true experts, the ones who can't miss. They have their biographies written."

Notice also the similarity to the fallacy underlying the articles one sees occasionally, along the lines of "Man Wins Lottery Twice Against Astronomical Odds". (The odds against a particular person winning twice are not the same as the odds against anyone winning twice.)

There is a correct way to do this forecasting, combining multiple inputs. One needs a measure of the track record of each forecaster, and a measure of the degree to which the input forecasts, and their accuracies, are correlated. (The statistical method is formally similar to Generalized Least Squares.) It's a useful technique, which I have seen used successfully in real financial markets. But there are no fancy "experts" and no fawning newspaper articles.

Blast! (1)

Phoenix666 (184391) | more than 7 years ago | (#16507875)

Foiled by statistics again!

You're as bad as the guy who takes issue with the statement, "We won't stop until all children are above average!"

You're as bad as the guy who worked at MegaHuge Hedge Fund in the late 90's. His boss walked into the office one day all excited about a new way to measure risk, called "Downside Risk Quotient." He asked the guy how often the stocks in their portfolio were below their mean price, or what their "Downside Risk" was. The guy foolishly answered, "50% of the time."

Re:Possible logical fallacy (0)

Anonymous Coward | more than 7 years ago | (#16507903)

Let's say you have a pool of 10,000 prognosticators. You ask each one to pick the winner of 10 football games. The odds of getting all 10 correct are 1 in 2^10=1024. So out of the pool of 10,000 people, by random chance alone you're likely to get about 10,000/1024 = 10 people who pick all 10 games correctly. Are these people "geniuses"? No, they just got lucky during this particular trial. The odds of them getting game #11 correct are just 50-50.

Make that picking against the point spread and you'd be more accurate. The outcome of football games isn't random. Suppose the games were the New England Patriots vs. the Oakland Raiders and 9 other equally lopsided games. The probability of getting all 10 correct would be 50-50 at worst if the prognosticators had even casual knowledge of football.

Infinite Monkeys on the infinite keyboards... (1)

jmagar.com (67146) | more than 7 years ago | (#16506773)

This should be pretty simple. Which few of the monkeys are genius? Find the ones that produced Hamlet, Romeo and Juliet, the Tempest, etc.

Any monkey able to produce a component of Shakespeare's collective works [wikipedia.org] should qualify.

It seems to me that Google [google.ca] could easily be used to find genius...

talking heads (1)

Aaarrrggghhh (987643) | more than 7 years ago | (#16506827)

I would love to input the data spewed out by all of the talking heads who make their political predictions (and other useless nonsense predictions) and are so often wrong. Then we could post disclaimers at the bottom of the screen when they talk:
(...Mr. Speak-for-my-Party has only been correct 25% of the time and he was on both sides of the issue for 75% of his correct predictions).

Re:talking heads (1, Informative)

Anonymous Coward | more than 7 years ago | (#16507049)

See the book by Philip E. Tetlock called "Expert Political Judgment: How Good Is It? How Can We Know?" by Princeton University Press. He collected the data you are looking for.

Obligatory Lazarus Long Quotes (1)

spun (1352) | more than 7 years ago | (#16506881)

"Democracy is based on the assumption that a million men are wiser than one man. How's that again? I missed something.
Autocracy is based on the assumption that one man is wiser than a million men. Let's play that over again, too. Who decides?"

And...

"Always listen to experts. They'll tell you what can't be done, and why. Then do it."

Courtroom Whores (1)

loose electron (699583) | more than 7 years ago | (#16507017)

The common parlance term for the "expert witness" among lawyers is "the courtroom whore" -- Lots of fancy sounding credentials, gotta have the doctroal degree, and willing to say anything for a price. Totally worthless idiots in most cases.

Besides:

"If you ask enought experts, you can confirm any opinion or theory."

Not sure who said it, but it's valid IMHO.

Let me get this straight... (0)

Anonymous Coward | more than 7 years ago | (#16507097)

So predicting the outcome of a sporting event or a stock's performance makes one a genius or expert?

Lord help us and save us. The idiocy of the media and general public never ceases to amaze me.

Doesn't work (1)

mlwmohawk (801821) | more than 7 years ago | (#16507191)

The problem with using public opinion about "experts" is that "experts" must then spout public opinion to be recognized.

True experts often have opinions contrary to public opinion. Just look at Slashdot. Insight is sometimes modded as flamebait. Counter intuitive opinions or assertions get derided and the author insulted.

Sorry, but "the masses" are generally stupid and would rather burn experts at the stake than question their tiny little world.

In Massachusetts, in our governor's race, one of the candidates is an attorney who was a public defender and he defended a couple despicable people. What does the opponent do? Leak private information about his brother in-law's criminal record, and accuse him of being "weak on crime" because he defended obviously guilty people.

The educated in MA, almost unanimously, call the ads appauling because it isn't attacking merely the candidate, but the whole justice system in our country. The "stupid" say things like "How could he defend that person." Never once thinking about "innocent until proven guilty." (Which remains true through appeal.)

So, when people try to parse the nonsensical ravings of the masses for reasoned information, I recall the old computer addage: "Garbage in, garbage out."

Re:Doesn't work (1)

Intron (870560) | more than 7 years ago | (#16508149)

That's desperation. Deval Patrick is going to clobber Kerry Healey. The masses aren't that stupid. Healey's latest stunt of suggesting to end turnpike tolls isn't going to work, either. We all know they'll change their mind right after the election.

I served on a jury a few years ago. It was an eye-opening experience. The jury was housewives or people like me too stupid to get out of jury duty. As a group, the jury noticed everything going on during the trial and came to a fair decision very quickly. Much better, I think, than if a single judge had been simultaneously presiding over and deciding the case.

I have more respect now then I did before serving on the "wisdom of the masses".

Re:Doesn't work (1)

rblum (211213) | more than 7 years ago | (#16508155)

One hopes that the truly educated in MA would call the ads "appalling", not "appauling" ;)

Re:Doesn't work (1)

mlwmohawk (801821) | more than 7 years ago | (#16508333)

Being educated does not, in fact, seem to affect one's ability to spell.

Survivorship Bias (1)

aldheorte (162967) | more than 7 years ago | (#16507335)

Explains everything observed by these sites. I predict that a lack of science and math education will continue to result in people fruitlessly attempting to use past performances in predicting chance-based events (either because of true randomness or sufficient complexity to thwart casual analysis) to denote exports that will at some point start getting it wrong.

Nice scam (3, Insightful)

DaveJay (133437) | more than 7 years ago | (#16507361)

Isn't this just that same old thing, where for each sporting event, you send a mailer to 50% of the people picking one team, and 50% picking the other, and whoever wins, that 50% of your original audience gets split between the two possible winners in the next mailing? Eventually you end up with a small audience, but they're CONVINCED you have a flawless sports betting "system" and pay you to learn it.

Here, by pretending you're figuring out who the "experts" are, you're not diluting your audience with each round of guessing; instead, you're diluting your potential pool of "experts" (or systems), and eventually everyone decides that person X is always right, when really odds were that at least one person in a large pool of guessers would guess right 100% of the time.

Past performance is no guarantee of future performance, people.

Re:Nice scam (1)

LargeWu (766266) | more than 7 years ago | (#16508073)

Indeed. It's exactly the same.

Notice that this guy started playing back in March, but they only evaluate over a 5 week period. How did he do the rest of the year before then? How are his picks since he was deemed "expert" doing by themselves? How long does he stay an expert once his personal picks start approaching the mean?

I'm not saying these guys aren't better than the general population. And the consensus picks might be better than any one individual. But take any one of these guys, and have them pick every game, and I think you would be doing VERY well to be successful 56% of the time over, say, 5 years.

Wisdom of the crowds (4, Insightful)

Col. Klink (retired) (11632) | more than 7 years ago | (#16507431)

Weird. The phrase Wisdom of the crowds [worldcat.org] was coined by James Surowiecki as the title of his book (see also wikipedia [wikipedia.org] ). The premise was that crowds, on average, can do better than a committee of experts. It's not that there is someone always in the middle, it's actually the highs and the lows aggregated that make sense in the wisdom of the crowd.

This sounds like the old scam. Pick 1000 people. On day 1, send 500 of them a prediction that stock A will go up and send the other half a prediction that the stock will go down.

On day 2, the stock either went up or down. Either way, you made a correct prediction to 500 people. Split the 500 and send two more predictions on an all new stock.

Keep repeating this. On the fifth day, you'll have 75 people who have seen you make 5 perfect predictions in a row. Now ask each of them for $10,000 to invest in your next prediction...

Just because one person happens to have hit the mean each time doesn't mean he's got "the knack". Statistically, there's sure to be someone whose guesses approach the mean. But that doesn't mean that their next prediction is any more likely to be accurate.

Stick with the aggregated mass knowledge.

Re:Wisdom of the crowds (2, Informative)

Lurker2288 (995635) | more than 7 years ago | (#16507599)

Yeah, I sort of thought the whole idea behind 'wisdom of crowds' is the fact that you aggregate enough data to cancel out the individual biases and result in a relatively accurate conclusion. If you pull out all the "experts" who make the correct call in one trial, don't you lose the correcting power of the group? How big and intellectually diverse does a 'crowd' have to be?

Re:Wisdom of the crowds (3, Interesting)

Sean0michael (923458) | more than 7 years ago | (#16508243)

Exactly. Having a pool of experts does not mean you will have a better outcome. The odds of any one expert being correct every time are very slim. But the odds of a group being correct every time are actually better. For those that watch the football announcers (our experts) make predictions about who will win today, they disagree and none have perfect records. But when it comes to predicting spreads, the guys controlling how many points one team will win by are much more accurate because a large number of people, all with private knowledge and information plus sharing some general knowledge, all weigh in on the outcome. This diverse group (which includes experts) generally gets the correct spread (if they don't, the sports gambler in charge is losing lots of money).

Experts are great, and their knowledge is valuable. But in making certain kinds of decisions, it is better to tap into the Wisdom of Crowds.

It's all about presentation (3, Insightful)

not already in use (972294) | more than 7 years ago | (#16507587)

One thing you have to remember: Perception > Reality. Speaking intellegently and writing intellegently is usually enough to convince someone that you actually know what you're talking about, if you're audience is ignorant or naive. That makes for a lot of percieved experts in the field of technology. Take the example of an internet born initiative to ban dihydrogen oxide in some county California http://www.dhmo.org/facts.html [dhmo.org] . Read this. If you haven't already heard of this, well, dihydrogen oxide is water. See how easy it is to convince a bunch of soccer moms they need to ban water? (Or that apple needs to abandon hardware... hehe)

Finding internet experts on Slashdot? Try 4chan. (1)

Peterus7 (607982) | more than 7 years ago | (#16507877)

And now, everybody feels pressured to post. Great job- post an article about finding internet experts on slashdot.

I thought Slashdot solved this problem already? (2, Funny)

Cherita Chen (936355) | more than 7 years ago | (#16508001)

Didn't Slashdot solved this problem with their moderation scheme? Oh, wait, nevermind... that would mean "Karma Whores" would qualify as experts. Nevermind.

The Emperor (2, Interesting)

entropy123 (660150) | more than 7 years ago | (#16508013)

This quote, headlined today on google, is instructive: It has always been the prerogative of children and half-wits to point out that the emperor has no clothes. But the half-wit remains a half-wit, and the emperor remains an emperor. - Neil Gaiman I find that 'Experts' are largely chosen based on qualities other than their expertise. Usually they have good personalities and make friends easily, especially with leadership. The leadership prefers to pick and talk with experts who generally agree with their views and look good. So, it is very difficult for me to look at a given expert and think 'Hrmmm...this guy must really know something about X and that is why he is on CNN'. More likely he was friends with a CNN producer.

Vox Populi (1)

xactuary (746078) | more than 7 years ago | (#16508063)

So basically you could have a massive system to guage the stock market, based on each quote player's unquote picks. You could call it the New York Stock Exchange, for example... Must go now. It's time for my random walk.

I am not an expert..... (1)

AMDfreak (976903) | more than 7 years ago | (#16508167)

....but I did stay at a Holiday Inn Express last night!

Re:I am not an expert..... (1)

systemeng (998953) | more than 7 years ago | (#16509397)

Experts when peroperly applied do form a usable concensus. The Rand Corporation studied using groups of experts to predict enemy attacks in the 1940's. They came up with the Delphi Method of estimation seen at http://en.wikipedia.org/wiki/Delphi_method [wikipedia.org] In tasks such as project management, the Delphi Method has been shown to be quite effective at predicting things like the true completion time of a complex project. While not a be-all and end-all the Delphi Method is one of the best uses of experts in prediction.

Erm (1)

valkabo (840034) | more than 7 years ago | (#16508327)

Maybe I'm a bit dumb but:

If user.slashdot.average > 4 Then user.intilligence = 1
If user.slashdot.average 4 Then user.intilligence = 0
If user.slashdot.average = -1 Then user.breeding = 0

I think that might work..(please rate this up or I won't be getting laid according to my own work)

Algorithms for rivaling the best expert (1, Interesting)

Anonymous Coward | more than 7 years ago | (#16508933)

Given a collection of experts, there are algorithms for judging the reliability of individual experts on the basis of a set of trial tests, and combining the expert opinions so your error rate is guaranteed to approach that of the best-performing expert (roughly 2x worse, worst-case, asymptotically). This is related to the statistical learning technique of boosting [wikipedia.org] . See the discussion here [dabacon.org] for more technical information, and a link to a paper.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...