Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Journal Article On Precognition Sparks Outrage

CmdrTaco posted more than 3 years ago | from the but-magic-is-fun dept.

Science 319

thomst writes "The New York Times has an article (cookies and free subscription required) about the protests generated by The Journal of Personality and Social Psychology's decision to accept for publication later this year an article (PDF format) on precognition (the Times erroneously calls it ESP). Complaints center around the peer reviewers, none of whom is an expert in statistical analysis."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered


Prediction (5, Funny)

Anonymous Coward | more than 3 years ago | (#34775490)

I predicted this would happen.

AGW (-1)

Anonymous Coward | more than 3 years ago | (#34775760)

But, none of the AGW peer reviewers were Statistics guys, in a "science" that is built on statisticss.

In fact, it took a Stat guy to find errors in their models.

oblig (0)

Anonymous Coward | more than 3 years ago | (#34775500)

The author of the article should have seen this coming really ..

Re:oblig (2)

krou (1027572) | more than 3 years ago | (#34775924)

The reason they published this is because they wanted to study the effect of negative reinforcement on ESP ability, but as you can see, that's just pissing people off.

The problem with this article is it will cause (3, Funny)

jolyonr (560227) | more than 3 years ago | (#34776308)

Mass Hysteria, Dogs and Cats living together, etc.

Why Is It Wrong to Call This ESP? (4, Informative)

eldavojohn (898314) | more than 3 years ago | (#34775502)

on precognition (the Times erroneously calls it ESP).

Why is that erroneous? Precognition and premonition are two facets of Extrasensory Perception. From its wikipedia article [wikipedia.org]:

Extrasensory perception (ESP) involves reception of information not gained through the recognized physical senses but sensed with the mind. The term was coined by Sir Richard Burton,[citation needed] and adopted by Duke University psychologist J. B. Rhine to denote psychic abilities such as telepathy and clairvoyance, and their trans-temporal operation as precognition or retrocognition.

So if you were dealing with anything of the above or anything external to our normal senses, I think that qualifies as ESP and calling it ESP. Sure that acronym has a lot of baggage but from the study itself:

... this is an experiment that tests for ESP (Extrasensory Perception).

That's what the tests subjects were told and I don't think the article is erroneous.

Re:Why Is It Wrong to Call This ESP? (0)

Smallpond (221300) | more than 3 years ago | (#34775572)

It's ESP if the cause is believed to be a currently unknown sense or special power. If they are doing a scientific study on whether people can perceive the future with no conclusions about the cause, then it's not really about ESP.

Re:Why Is It Wrong to Call This ESP? (1)

MoonBuggy (611105) | more than 3 years ago | (#34775608)

Well we don't know any current mechanism by which people can perceive the future (assuming that everyday extrapolation is not included, I guess - there's no mystery about the fact that I can tell a flying ball will soon hit the ground), so wouldn't that mean that if evidence is found for an ability to sense the future, that implies ESP under your definition?

Re:Why Is It Wrong to Call This ESP? (1)

Smallpond (221300) | more than 3 years ago | (#34775762)

Why do you remember the past and not the future? "Hasn't happened yet" is a perception, not a physical characteristic of the world. The world just exists in 4 dimensions; it doesn't change. Living things perceive a direction of time.

Re:Why Is It Wrong to Call This ESP? (1)

TheLink (130905) | more than 3 years ago | (#34775830)

The world just exists in 4 dimensions; it doesn't change.

What's the evidence for this?

Re:Why Is It Wrong to Call This ESP? (2)

gweihir (88907) | more than 3 years ago | (#34775980)

The world just exists in 4 dimensions; it doesn't change.

What's the evidence for this?

None at all. It is a neat model, and physics seems to be completely explained by it ("seems", as there are still fundamental gaps in the current models), but it does not take into account human beings. It is quite possible that directionality of time and impossibility to predict the future is actually something brought into this universe by human beings or life itself.

The 4-dimensional model does only cover dead matter and it is just a model, not reality.

Re:Why Is It Wrong to Call This ESP? (1)

MoonBuggy (611105) | more than 3 years ago | (#34775862)

Firstly, that bears little relation to what I said. Even taking your post at face value, the fact is that we "living things" still do perceive the past but not the future with the senses currently known by science - if it is discovered that we can also perceive the future, it will be above and beyond our current sensory experience, hence ESP. But anyway, we're really just arguing semantics here.

The world just exists in 4 dimensions; it doesn't change.

Secondly, entropy begs to differ.

Re:Why Is It Wrong to Call This ESP? (5, Funny)

prionic6 (858109) | more than 3 years ago | (#34775888)

I'd say it is more a case of earth having 4 corners in some kind of a simultaneous 4-day... In only 24 hours of rotation, there are 4 corner days, cubes for a quad earth. No 1 Day God.

Re:Why Is It Wrong to Call This ESP? (1)

gardyloo (512791) | more than 3 years ago | (#34775978)

Perhaps you should learn a bit about causality and entropy, before (whi you wander in and spout gobbeldygook.

I agree (3, Funny)

Anonymous Coward | more than 3 years ago | (#34775644)

There are many types of ESP, covering different aspects of life. Each is differentiated by a letter.

For example, the 14th type of ESP covers sports. It is called ESPN.

Re:Why Is It Wrong to Call This ESP? (2)

zero.kalvin (1231372) | more than 3 years ago | (#34775800)

I don't understand this. If the researchers did a proper experiment, respected the rules, followed proper procedures, and did a proper analysis of the data they collected, in a scientific way. Why is it a problem to publish ? So now we should bar publication that don't agree with our general conception ? If it was done considering the specific guidelines set by the scientific community of how to do things, screw them. Science is not a democratic process, nor it should be politically correct. Science is science.

Re:Why Is It Wrong to Call This ESP? (5, Informative)

AlecC (512609) | more than 3 years ago | (#34775898)

From the summary, the implication is that the data analysis was not proper - or at least, not shown to be proper. Since the claimed effect is a fairly small artifact only detectable by sophistcated statistics, it seems reasonable that the reviewers should include those who have a deep understanding of such statistics - which, it is claimed, they did not.

Re:Why Is It Wrong to Call This ESP? (0)

Anonymous Coward | more than 3 years ago | (#34776016)

Science is not a democratic process, .... Science is science.

That peer review is the gold standard for publication implies that science *IS* a democratic process at some level. It's just that only the other scientists get the chance to vote.

Re:Why Is It Wrong to Call This ESP? (3, Insightful)

ledow (319597) | more than 3 years ago | (#34776190)

They failed that one of your hurdles - they didn't do a proper analysis on their data. Basically, their data conclusively shows that the chances of pre-cog existed COMPARED TO it not existing is extremely minimal (actually quite strongly in favour of it not existing). But they specifically chose only certain analyses to conclude that it *did* exist.

There are many rebuttals at the moment, most linked to in these comments, that you can read but basically - to remove all statistical jargon - they didn't bother to take account of how probable their data was by pure chance. Their "error margin" is actually vastly larger than their data could even escape, so they can't really make any firm conclusions and certainly NOT in the direction they did. Statistics is a dangerous field, and whoever wrote and reviewed that paper didn't have a DEEP grasp of it, just a passing one.

If you calculate the *chance* that their paper is correct versus their paper being absolute nonsense, not even taking into account anything to do with their methods or that their data might be biased, their data can ONLY mathematically support a vague conclusion that their paper is nonsense. To do the test properly and get a statistically significant result (not even a *conclusive* result, just one that people will go "Oh, that's odd") they would have to do 20 times as many experiments (and then prove that they were fair, unbiased etc.).

It's like rolling three sixes on a die and concluding that the particular die you rolled can only possibly roll a six. It's nearly as bas as claiming that so can every other die on the planet.

Re:Why Is It Wrong to Call This ESP? (5, Insightful)

jason.sweet (1272826) | more than 3 years ago | (#34776282)


In one sense, it is a historically familiar pattern. For more than a century, researchers have conducted hundreds of tests to detect ESP, telekinesis and other such things, and when such studies have surfaced, skeptics have been quick to shoot holes in them.

I always thought the hole-shooting was an essential step in the scientific process. If you can't patch up the holes, you aren't really doing "science."

In science, you tell a story. Then everyone says, "no, that's wrong because..." Then you say, "I'm afraid I'm right, because..." It is an imperfect process because people have biases that are hard to overcome. But, if the empirical evidence is strong enough, you will overcome these imperfections. In the end, you build a consensus by presenting enough evidence that no one can argue with. I'm not sure what is more democratic than that.

Obligatory (0)

Anonymous Coward | more than 3 years ago | (#34775540)

Didn't see that one coming.

Re:Obligatory (1)

Anonymous Coward | more than 3 years ago | (#34775578)



Filter error: Don't use so many caps. It's like YELLING.

Great response paper (5, Informative)

246o1 (914193) | more than 3 years ago | (#34775554)

For those who haven't seen it, here's a pretty sharp takedown of this paper, as well as some notes on statistical significance in social sciences in general: www.ruudwetzels.com/articles/Wagenmakersetal_subm.pdf

Re:Great response paper (5, Informative)

Anonymous Coward | more than 3 years ago | (#34775672)

And for those who still haven't seen it, here's a proper link [ruudwetzels.com].

Re:Great response paper (1)

paiute (550198) | more than 3 years ago | (#34775842)

And for those who still haven't seen it, here's a proper link [ruudwetzels.com].

The linked paper is a good read, even for someone like myself with only an AP level knowledge of statistics. Good illustrative examples.

not just social sciences (-1)

Anonymous Coward | more than 3 years ago | (#34775826)

Climate warming relies pretty heavily on statistical significance and does very little in the way of having statistics experts involved in peer review.

Statistics have even replaced contemporary eye-witness accounts of historical periods. Decades of reports of rivers freezing over every year does not show that temperature was under 32 degrees F since tree rings say otherwise.

I know, "local" vs "global", I'm cherry picking, and on and on. Hopefully you can at least admit that statistics are
1. heavily used
2. not regularly peer-reviewed by statistical experts (they claim one needs to be a "climate expert" to criticize their work. no stat or modeling or computing or other experts allowed)

Re:Great response paper (5, Insightful)

ledow (319597) | more than 3 years ago | (#34775992)

Ouch. Taken down by two Bayesian tests on whether it's more likely that the paper is true or not. They didn't even need to get out of bed or dig out a big maths book to basically disprove the entire premise of the original paper using its own data.

As they hint at in that rebuttal - As a mathematician and someone of a scientific mind, I would just like to see *ONE* good test that conclusively shuts people up. Trouble is, no good test will report a false result and thus you'll never get the psychic / UFO / religion factions to even participate, let alone agree on the method of testing because they would have to accept its findings.

Never dabble in statistics - the experts will roundly berate you and correct you even if you *THINK* you're doing everything right. When PhD's can't even work out things like the Monty Hall Problem properly, you just know it's not something you can throw an amateur towards.

Re:Great response paper (4, Interesting)

wonkavader (605434) | more than 3 years ago | (#34776292)

Dabbling is fine when the results are good. He had 53%. If he'd had 65%, dabbling would have worked. But dabbling is just the start, and that's not just the nature of peer review, it's the nature of collaboration and a University setting. You find something neat by dabbling, and you walk down the hall to visit someone with more stats experience to get some clarity before you publish.

He had 53%. He knew that if he walked down the hall, he'd get told he had squat. So he didn't walk down the hall.

There's dabble initially, and that's fine. And there's dabbling (ONLY) and calling it done. That's not.

Seems like the paper was written by a dabbler, then reviewed by a respected team of dabblers. And not one of them looked at 53% and walked down the hall. Bubbleheads.

Re:Great response paper (4, Insightful)

Burnhard (1031106) | more than 3 years ago | (#34776212)

But isn't that simply applying the maxim: extraordinary claims require extraordinary evidence? I.e. it is not asserting the evidence doesn't exist, only that the evidence isn't strong enough to make the assertion that the hypothesis is true with any confidence. There's a subtle difference.

Re:Great response paper (0)

Anonymous Coward | more than 3 years ago | (#34776216)

From the abstract: "We reanalyze Bem’s data using a default Bayesian t-test and show that the evidence for psi is weak to nonexistent".

Since it all derives to thought through quantum effects, thought realizes it cannot be recursive and thus is correcting for itself. It is the quantum state manifesting itself in the macro world.

Research Funding (4, Informative)

MoonBuggy (611105) | more than 3 years ago | (#34775558)

I haven't yet had a chance to read the paper fully (it's 50 or so pages), but if they are actually that confident in their evidence that precognition has been found, the James Randi Foundation has a million dollars [randi.org] waiting for them.

Re:Research Funding (1)

TFAFalcon (1839122) | more than 3 years ago | (#34775612)

Why settle for just 1M? Play the lottery for a few weeks and you don't even have to bother writing an article to be rich.

Erotic Pictures a Necessary Element (5, Interesting)

eldavojohn (898314) | more than 3 years ago | (#34775868)

Why settle for just 1M? Play the lottery for a few weeks and you don't even have to bother writing an article to be rich.

So I realize a lot of people aren't going to read the article but here's the meaty parts for you statistics snobs (and really, the Bayes folks are going to be all over this one):

Across all 100 sessions, participants correctly identified the future position of the erotic pictures significantly more frequently than the 50% hit rate expected by chance: 53.1%, t(99) = 2.51, p = .01, d = 0.25.3 In contrast, their hit rate on the nonerotic pictures did not differ significantly from chance: 49.8%, t(99) = -0.15, p = .56. This was true across all types of nonerotic pictures: neutral pictures, 49.6%; negative pictures, 51.3%; positive pictures, 49.4%; and romantic but nonerotic pictures, 50.2%. (All t values < 1.) The difference between erotic and nonerotic trials was itself significant, tdiff(99) = 1.85, p = .031, d = 0.19.

There's a lot more about eliminating random number generators (by using this little guy [araneus.fi]) leading to prediction as well as running more tests where they are asked to pick a preference of two identical images. The most interesting part is that these results seemed to hinge on pornography. The individuals only exhibited this "precognition or premonition" when they were picking erotic images or rewarded with erotic images (albeit from the International Affective Picture System).

The skeptic in me is very pleased and excited about this part of the paper:

Accordingly, the experiments have been designed to be as simple and transparent as possible, drawing participants from the general population, requiring no instrumentation beyond a desktop computer, taking fewer than 30 minutes per session, and requiring statistical analyses no more complex than a t test across sessions or participants.

Grad students across the country: get to work!

But you would have to have the lottery involve some sort of erotic pictures containing the known numbers in order for this edge to be garnered. Which would be impossible unless the lotteries changed how they worked. Maybe play blackjack with a set of playboy cards? :-)

Re:Erotic Pictures a Necessary Element (1, Offtopic)

TaoPhoenix (980487) | more than 3 years ago | (#34776160)

There are only seventeen types of pornographic troll comments here in the recent history of Slashdot, and most of them are goatse. Therefore I predict that the third pornographic troll comment to appear seven stories from now will be a goatse. Further, there seems to be a rise in a goatse bot that can't spell which is supplanting classical troll phrasing.*

* Statistics not properly peer reviewed. However, I predict that it is too much work for anyone to properly disprove this comment. Therefore I will get some kind of Heterogenous Mod including one flamebait and one funny.

** If this is true, it would then deserve an Informative Mod. If this post does not then receive an Informative Mod, that would indicate non-random negative reinforcement applied deliberately in disregard of Mod guidelines.

Re:Erotic Pictures a Necessary Element (1)

Peeteriz (821290) | more than 3 years ago | (#34776238)

Haven't read the paper, don't know the sample size involved and experiment details, and don't have the time to run the math right now, but it looks like the numbers shown in your post would still support the null hypothesis that there is no precognition effect, so entirely opposite conclusions should be drawn from the same data.

Re:Research Funding (1, Insightful)

TheGratefulNet (143330) | more than 3 years ago | (#34775928)

something to the effect of 'since there are no rich fortune-tellers, we have to assume that they are all fakes.'

the answer is simple, really. they are all fakes.

life isn't all complex. humans being fakers and liars is one constant we can count on in the universe.

Re:Research Funding (4, Insightful)

Rob Kaper (5960) | more than 3 years ago | (#34775674)

Any evidence for precognition instantly takes precognition into the scientific realm and out of the paranormal, supernatural and occult.

Re:Research Funding (3, Informative)

MoonBuggy (611105) | more than 3 years ago | (#34775804)

Oh, absolutely, but the rules state:

Webster’s Online Dictionary defines “paranormal” as “not scientifically explainable; supernatural.”

Within the Challenge, this means that at the time your application is submitted and approved, your claim will be considered paranormal for the duration. If, after testing, it is decided that your ability is either scientifically explainable or will be someday, you needn’t worry. If the JREF has agreed to test you, then your claim is paranormal.

I'm sure that if a bunch of scientists came along and said "we have statistically significant evidence of precognition, and not a damn clue how it works", the Randi foundation would jump at the chance to test them.

I don't believe for a second that these people actually do have any legit evidence, but on the off chance that they are for real then this will be a massive breakthrough. Of course, it will be explainable by science in time, and perhaps "supernatural" is a poor choice of word, but if you read through the entire FAQ [randi.org] you'll see that the foundation sound entirely reasonable, and I don't doubt that they would be willing to test something on the basis that it runs quite counter to currently accepted theory.

Their aim (and one that I applaud) seems to be to either disprove paranormal claims, or to prove them in a scientific manner. Sure, doing so will, by definition, destroy their 'paranormal' status, but it could also revolutionise scientific thinking. As I said though, it's probably a moot point, since I see no reason to believe this paper any more than the thousands that came before it.

Re:Research Funding (1)

nedlohs (1335013) | more than 3 years ago | (#34775748)

That'd be remarkably stupid, since they could just visit casinos and pull in a million dollars every couple of days. At least until the casinos run out of money.

Re:Research Funding (1)

TheCarp (96830) | more than 3 years ago | (#34776044)

That assumes perfect precognition. The effects that I saw claimed were more like, 3% better than random.

For a number of casino games, with "perfect play" (perfect not including black jack card counting, even though it should) the casino advantage is in that range usually. In fact, I believe the payout on slot machines is often close to 98%, again depending on the play. (some of the really pathalogically bad bets give the house much better odds)

So, maybe these people DO hit up casinos, and... don't even know it. They just seem to have a bit more luck than everyone else, and come out just a bit up. Of course, one claim that I didn't see (have seen other articles on this) is that individuals have precognition. Perhaps everybody has just a tiny bit of it.

Now, more likely... these are statistical anomalies. Frankly, I wouldn't doubt that he knows that and is trying to make a point.

Re:Research Funding (1)

mangu (126918) | more than 3 years ago | (#34776294)

The effects that I saw claimed were more like, 3% better than random.

Which should be more than good enough to make a fortune.

At European casinos, if you play red vs. black in the roulette, you have a 18/37 chance, that is 48.64%, of winning double your bet.

A 3% better than random precognition rate would let you get rich in a few hours, while still being random enough to be considered pure luck, so you could get filthy rich before they banned you from all the casinos in the world.

Re:Research Funding (0)

Anonymous Coward | more than 3 years ago | (#34776074)

Sorry, I think that should read: "until the casinos ban them for life, on their first day, well before they've managed to turn much of a profit on the trip". Casinos don't like winners and they all share information with each other to ban them.

Re:Research Funding (3, Insightful)

dkleinsc (563838) | more than 3 years ago | (#34775768)

I'm not sure they're that confident in their evidence. Nor should they be - they did a study, they publish their findings, lots of other scientists either put down rebuttals (as has already happened), or repeat the study and see if it's accurate enough to be true. That's the way science is supposed to work.

What's not supposed to happen is "Scientist A does an apparently sound study that appears to demonstrate something that scientists B,C, and D consider silly, and scientists B, C, and D stop scientist A's work from ever seeing the light of day."

Re:Research Funding (2)

246o1 (914193) | more than 3 years ago | (#34775870)

I'm not sure they're that confident in their evidence. Nor should they be - they did a study, they publish their findings, lots of other scientists either put down rebuttals (as has already happened), or repeat the study and see if it's accurate enough to be true. That's the way science is supposed to work.

What's not supposed to happen is "Scientist A does an apparently sound study that appears to demonstrate something that scientists B,C, and D consider silly, and scientists B, C, and D stop scientist A's work from ever seeing the light of day."

They shouldn't be confident in their evidence - you are right about the way science should be done, but I think in this case the rebuttals are easy to find because this paper seems to be the result of significance-chasing, with enough simultaneous 'experiments' going on within each individual experiment (i.e., the 'experiment' to determine whether men were affected by Thing Type A was a subset of the experiment of whether anyone was affected by Things Type A,B,C, or D) that it would be surprising if significant deviations didn't occur.

This exact problem exists, with added moral hazard, in drug companies doing trials, which is why we should always require that the data of ALL drug trials be made public - otherwise, a drug company desperate for a win could run 20 different studies, and usually get a 95% significant result above placebo on even a placebo! It's important to consider the number of chances someone has to make their case, in random trials, not just whether one study or way of slicing the data seems to make the case.

Re:Research Funding (1)

sycodon (149926) | more than 3 years ago | (#34775778)

I believe that they have to tell him what is in an envelope locked in a safe to get the money. Unless he's doubled down and made additional challeneges.

Re:Research Funding (1)

MoonBuggy (611105) | more than 3 years ago | (#34775932)

The challenge I linked allows anyone with a claim of "paranormal ability" to design their own scientific test, to be agreed upon by the foundation. Once agreement is reached, the person is tested under supervision by a panel of impartial experts. Theoretically, they then show an astonishing ability, get a nice bundle of cash, and science is greatly advanced by studying what they can do. So far, though, the result has invariably been that the participant looks slightly sheepish when their 'powers' fail to function in the preliminary test.

I always think it's kind of a shame, actually - sceptical as I am, I'd quite like to see some amazing new phenomenon that we can't yet explain.

Re:Research Funding (4, Funny)

elrous0 (869638) | more than 3 years ago | (#34775836)

If they exist, I want them arrested immediately for aiding and abetting terrorists on September 10, 2001. Obviously, they all willfully stayed silent.

Extraordinary claims... (1)

sdo1 (213835) | more than 3 years ago | (#34775566)

"Extraordinary claims require extraordinary proof." - Marcello Truzzi

Re:Extraordinary claims... (1)

paiute (550198) | more than 3 years ago | (#34775860)

"Extraordinary claims require extraordinary proof." - Marcello Truzzi

"Extraordinary claims require a shiny object in my left hand to distract you from my right." - S. A. Scoggin

You guys are all late to the game... (0)

Anonymous Coward | more than 3 years ago | (#34775618)

All of the people qualified to have this argument already had it yesterday.

Save the click (0)

Anonymous Coward | more than 3 years ago | (#34775632)

I already read it.

I can predict the future! (1)

antifoidulus (807088) | more than 3 years ago | (#34775654)

I predict this article will spark a bunch of kvetching about having to register and obligatory links to bugmenot like sites. Do I win?

Re:I can predict the future! (1)

erroneus (253617) | more than 3 years ago | (#34775824)

I agree with you completely except for the kvetching part. I find it difficult to agree because I don't know what it means. I suspect it is not an English word.

Reminds me of this hoax... (3, Informative)

LordNacho (1909280) | more than 3 years ago | (#34775662)

http://en.wikipedia.org/wiki/Sokal_affair [wikipedia.org]

Basically, a physicist made up some BS and got it published in a journal called Social Text about postmodern cultural studies. He then came out later and revealed the hoax, embarrassing the reviewers and the journal. Lack of intellectual rigour seemed to be the target. This time, it seems to be more specifically aimed at the lack of understanding of statistics in certain subjects.

Re:Reminds me of this hoax... (1)

fadethepolice (689344) | more than 3 years ago | (#34776302)

Similar reasoning would also invalidate the worldwide banking system because bernie madoff and ponzi were able to pull off a financial hoax.

Peer review wouldn't matter (0)

Anonymous Coward | more than 3 years ago | (#34775712)

I've been to grad school, and much of the research being done out there in many disciplines, especially in the social sciences (but even the real sciences as well), is a lot of rubbish anyways, and having a statistical expert or a statistically driven paper only serves the purpose of legitimizing the invalidity of the rubbish. Professors publish papers as part of their job description, and the more papers a university publishes, the more funding it gets. Universities have become businesses pumping out lots of garbage to make lots of cash, just like normal American businesses have become: 1) produce garbage, 2) take people's money. Why do you think tuition goes up 5-10% every year? Follow the money trail.

Peer review only provides weak prescreening (2, Interesting)

Anonymous Coward | more than 3 years ago | (#34775724)

I'm a Ph.D. candidate in EE, and I'm sometimes invited to review papers for IEEE journals.

I always read the paper carefully at least 3 times, read the important parts of references that are new to me, check all the math and sometimes even reproduce some simpler simulations.

Most reviewers aren't this careful. They either don't have the time or don't have the expertise to find some flaws. Keep in mind that reviewers aren't paid, and are anonymous. Also, the best reviewers are the best researchers, who are usually busy with their own projects.

I often find serious flaws that my fellow reviewers completely overlook. Fortunately in these cases, the editors have always used my reviews to override the other two (the review decision is not a majority vote). Given the quality of the reviewers out there, some papers are accepted simply because the editor invited 3 incompetent reviewers, which is not very unusual. And we're talking about IEEE journals, which should be the best in the field.

So in practice, peer review is only a weak pre-screening process that often rejects good papers and accepts bad work. Science progresses because once something is published, other people attempt to reproduce it. If the idea works, then it's incorporated into other work and becomes famous. Otherwise, people just ignore it.

Re:Peer review only provides weak prescreening (5, Interesting)

gweihir (88907) | more than 3 years ago | (#34776134)

I have a PhD in CS.

You can also have good science rejected by getting three incompetent reviewers. Happened to me several times, the worst one when the program committee attached a note that showed they had not read or understood their own call for papers. I suspect a direct lie to keep me out. Published it later unchanged somewhere else and those people were surprised it got rejected earlier.

In addition to incompetent reviewers, there are also those that are envious or want to steal your ideas. Peer-review is fundamentally broken. One friend who has a PhD in a different CS area thinks 70% of researchers are corrupt, reviewing things positively when they know the authors, no matter the quality and negatively otherwise. Lying in application to research grants is also quite common. The final result is that good researchers have trouble working and often leave research altogether, which may be an explanation for how glacially slow some fields move.

If you think you may be psychic... (1)

steak (145650) | more than 3 years ago | (#34775734)

maybe you are.

Harness your unique abilities and take advantage of the many Federal benefits available for psychic citizens.

Would you like to know more ?

Re:If you think you may be psychic... (1)

Compaqt (1758360) | more than 3 years ago | (#34776006)

Your ideas intrigue me, and I would like to subscribe to your newsletter.

Bad language (1)

gmuslera (3436) | more than 3 years ago | (#34775758)

The word precognition should fall into the same kind of internal contradiction as the word almighty, at least for the future events on which you can change affect whether it happens or not. To predict the future is ok, but really knowing the future should violate some physics rules as going faster than the speed of light.

I believe (1)

NuKe_MoNgOoSe (1941452) | more than 3 years ago | (#34775764)

Why not? The world is more interesting believing in shit like this.. same with aliens and ghost and the after life, god and jesus.. The world is just more colorful if you lose logic and adopt a ideology that anything we know may be wrong, and anything we dont know can certainly be possible. Most people are so trapped in the confines of logic and science that the magical or the fantastic are ludacris concepts that are best kept in comic books. For those of us able to step outside this ignorant box the world is a much more varied place full of wonder and mystery even more than science itself can offer a explanation to. Electrical activity shows we only use a small percentage of our brain at any given time and there are regions of the brain that medical science has no idea what they are for. It is not so far fetched to believe that the logical rules of this world cannot be bent or even broken. We are just to stubborn to let go of the linear and hard-angled concepts, the rules that allow us to explain and categorize everything in our life.. See the future, move objects with your mind, call on reseves of extraordinary strength in dire circumstances.. countless, countless claims of these fantastic feats are all over the internet and in papers and span generations and every country... so I guess everyone is a liar?

Re:I believe (1)

ScentCone (795499) | more than 3 years ago | (#34775958)

so I guess everyone is a liar?

Yup, that pretty much covers it. Liars, or people who are too dim to understand how their own physiology can mess with their perceptions of what's happening to them, near them, or in incorrectly remembered past moment.

I was a Precog (1)

Herkum01 (592704) | more than 3 years ago | (#34775766)

When I 10 years old, there was a commercial that said that if you roll a 6-sided die 60 times and you correctly guessed the results more than 10 times you were precognitive. Well I guess correctly 11 times, so there are your scientific results. Now I move into my new career as a stock market analyst.

Precognition? No way... (1, Funny)

mswhippingboy (754599) | more than 3 years ago | (#34775772)

Most people I encounter even have trouble with "postcognition". Yea, I'm looking at you Sarah Palin!

Re:Precognition? No way... (1)

ScentCone (795499) | more than 3 years ago | (#34776072)

Most people I encounter even have trouble with "postcognition". Yea, I'm looking at you Sarah Palin!

I was thinking, actually, that all of the people (including Palin) who predicted that Obama would turn out to be less or different or worse than the imaginary character that millions of people thought they were votiing for ... that that shows a certain level of cognition that a whole lot of post-election-coginition is allowing other people to realize they'd done something silly.

And that's skepticism, everybody (1)

JamesP (688957) | more than 3 years ago | (#34775806)

Make an article with a "forbidden word" or a "forbidden topic" or even something a little bit different from something "that everybody knows" (e.g. gastric ulcera) and it's immediately wrong.

Remember guys, next time a crackpot says "oh but they laughed at Einstein as well", IT'S YOUR FAULT

Reminds me of a recent Slashdot article (1)

bigsexyjoe (581721) | more than 3 years ago | (#34775810)

http://science.slashdot.org/story/11/01/02/1244210/Why-Published-Research-Findings-Are-Often-False [slashdot.org] If you read this piece it talks about a "decline effect." Basically research that gets published has positive results to report and conforms to established opinion. However, further study shows that the effects aren't as strong as before. It talks about research showing pre-cognition before, but later be disproven. I think it's just fine to publish work in journals on pre-cognition, if the work was done in a scientific way. The only reason not to is because it doesn't conform to established opinion. If it's a false positive, it'll not be replicated and we'll forget about it. If it stands the test of time then we'll realize that there is pre-cognition of some sort but a mundane explanation will be found. After all, your brain is a sort of pre-cognition machine. I have a pretty good idea of what is going to happen in my life today. Don't you?

Fuck NY times and their cookie wall (0)

Anonymous Coward | more than 3 years ago | (#34775822)

Don't expect to see me on the other side...

Why do so many e-publications insist on killing themselves?

Good for JPSP (1)

Anonymous Coward | more than 3 years ago | (#34775834)

When I was a PhD student at one of the top 3 psychology programs, I made it a hobby to study the statistics used in psych papers that made it into Science and Nature. Without fail, each was a huge mess, well below any reasonable standard of science, let alone honesty.

If JPSP is going to accept poor methodology, it might as well accept this paper. Otherwise it's using the prior probability of the conclusion as a proxy for actual statistical analysis. The paper posted by 246o1 essentially points out that once the prior is taken into account, the evidence in the paper is extremely weak. That's a good point, but it's valid for nearly all psychological papers, so instead of making an ad hoc point here, why not raise the standards of evidence to include explicitly modeling the prior?

Somewhat amusingly, the papers that make it into the highest level journals (e.g. Science and Nature) are those that make the most surprising claims (babies have an innate sense of morality?!) and hence those advocating hypotheses with lower prior probability. So essentially the evidence for the claims in the more prestigious journals is weaker.

Re:Good for JPSP (1)

symes (835608) | more than 3 years ago | (#34776124)

Sadly, I agree. Psychology is dying. Not because there's not a lot of potentially interesting research to be done, more because psychologists are increasingly both nepotistic and unable to analyse their way out of a card board box. Yet at the same time we should perhaps be thankful for the science of psychology. Imagine what would happen if all those psychologists tried to do something useful in the world! We'd have a lot more Wakefield/Lancet fiascos, for sure. Returning to the point at hand, my advice has always been to research and accurately describe methods and analytic strategy way before any data collection has begun - instead psychology has become the science of finding the test that provides the significant result, damn the consequences. That said, surprising results with scant evidence can be very useful at times as they can motivate better designed studies that really do find something of interest. So long as people are honest about their findings and are not trying to fudge the issue, we are ok. There's nothing fundamentally wrong with announcing that chewing the bark of some tree cures headaches in a sample of three.

maybe... (3, Interesting)

TheCarp (96830) | more than 3 years ago | (#34775852)

Maybe this is a hack. They say he has a sense of humor.... think of this... he did design his studies well, at least the ones that I have read about. The effects of this "time leaking" are fairly small. Perhaps the entire point...is to make a point about statistics.

Added bonus? Put the ESP issue to bed. Him doing this, and specifically doing it so publicly and getting it passed peer review and publication, ENSURES that these studies are going to be replicated by numerous people, for the next several years. That, in and of itself, could produce enough evidence against ESP to really put the issue to bed :)

Say what you want about his paper, the effects reported are as large as many "well accepted" study results. Which may be the scariest part of all.

That said, I am no ESP believer (that may be obvious) but, some of the statements that are made against it are ridiculous too. "Why aren't people winning the lottory with their perfect precognition". The effects he is talking about here are on the order of a few percentage points better than random... which is more than the house advantage at many casino games (assuming optimal play)


Something like this must happen from time to time (2)

pentadecagon (1926186) | more than 3 years ago | (#34775880)

Across all 100 sessions, participants correctly identified the future position of the erotic pictures significantly more frequently than the 50% hit rate expected by chance: 53.1%

It's pretty easy to come up with significant results in this field: Just do a sufficiently large number of experiments, and you will inevitably come across some significant results. This works for any definition of significance, though of course it's easier for low standards.

Re:Something like this must happen from time to ti (2)

Hatta (162192) | more than 3 years ago | (#34776198)

Exactly. One out of every twenty "statistically significant" effects (P value =.05) is due to random chance.

Precog beats statistics analyst? (1)

Yaddoshi (997885) | more than 3 years ago | (#34775914)

They don't NEED to be a statistical analysis expert if they already know what the results are going to be ... in advance.

Not a surprise... (1)

njvack (646524) | more than 3 years ago | (#34775950)

Disclaimer: I have not read the original paper.

This is really not a big surprise. The researcher has probably run many, many experiments. Many of them doubtless turned up nothing. But, by chance alone, an experiment has a 5% chance of showing an effect with 95% confidence -- that's what p < .05 means. It's like rolling a natural 20. If he's run, say, 100 experiments over the years, he should have something around 5 rather convincing results to show for his efforts.

Next, hundreds of other researchers request his materials and run the same experiments and... shock! A bunch of them show the exact same effect! Now, not only has one research lab demonstrated evidence for ESP, but the study has been successfully replicated by researchers around the world!

All of this, of course, is just by chance.

And really, you see this all the time in other fields (at least, in the fields I work in). It's just very vey very very very easy to convince yourself that the reason your earlier experiments didn't work out was that you made a mistake, and the reason this one did was that you did everything perfectly.

A few years ago, there was an excellent essay in PLOS One Why Most Published Research Findings Are False [plosmedicine.org] about these (and more insidious) effects. Should be required reading for scientists.

Journal Article On Precognition Sparks Outrage... (5, Funny)

knewter (62953) | more than 3 years ago | (#34776058)

The headline SO should have been:

Journal Article On Precognition Sparks Outrage BEFORE IT'S PUBLISHED.

That is all. I expect more out of an editor.

Lack of statistical expertise you say? (0)

Burnhard (1031106) | more than 3 years ago | (#34776084)

It's a shame the same people don't apply their enormous wits to Climate Science, especially if the complaint is a lack of statistics expertise in the review process. Please mod me troll. Thanks.

No subscription with this link (0)

Anonymous Coward | more than 3 years ago | (#34776214)

Strip the "http://www.nytimes.com/glogin?URI=" part from the link, and voila, you can pee(r) through the registerwall.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account