Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Is Something Wrong with the Scientific Method?

pickens (49171) writes | more than 3 years ago

Science 6

Hugh Pickens writes "Jonah Lehrer has an interesting article in the New Yorker reporting that all sorts of well-established, multiply confirmed findings in science have started to look increasingly uncertain as they cannot be replicated. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology and in the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants. “One of my mentors told me that my real mistake was trying to replicate my work," says researcher Jonathon Schooler. "He told me doing that was just setting myself up for disappointment.” For many scientists, the effect is especially troubling because of what it exposes about the scientific process. "If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved?" writes Lehrer. "Which results should we believe?" Francis Bacon, the early-modern philosopher and pioneer of the scientific method, once declared that experiments were essential, because they allowed us to “put nature to the question" but it now appears that nature often gives us different answers. According to John Ioannidis, author of “Why Most Published Research Findings Are False," the main problem is that too many researchers engage in what he calls “significance chasing,” or finding ways to interpret the data so that it passes the statistical test of significance—the ninety-five-per-cent boundary invented by Ronald Fisher. "The scientists are so eager to pass this magical test that they start playing around with the numbers, trying to find anything that seems worthy,”"

cancel ×

6 comments

Sorry! There are no comments related to the filter you selected.

Just statistical incompetence (1)

AliasMarlowe (1042386) | more than 3 years ago | (#34731070)

I come across this all the time, but there's no need to panic. Basic physics and engineering are not threatened, just the justifications for the fad-du-jour in medicine or sociology or other semi-empirical studies. Alcohol is good for you; no, it's bad for you; no, it's very good for you in moderation; no, any amount is bad; and so forth. All of the conclusions on alcohol vs health are at 95% confidence, too, given the appalling misapplication of statistics to the relevant data (typically from inadequate studies, anyway).

Medical scientists are far worse than engineers or physicists for abuse of statistical methods. None of these groups plumbs the same depths as MBA candidates, of course. I speak as one who has taught grad students in Physics, Engineering, Business, and Medicine (actually Medical Physics). There is a distinct gradient in the extent to which they will manipulate or interpret data so a statistics black box will produce the desired result. The business students produce the worst reactions when shown the error of their ways ("You say the data are inconclusive, but I need to draw a conclusion from it. The conclusion does not have to be right, but it must be persuasive, so I have to get the right statistical result...").

Re:Just statistical incompetence (1)

russotto (537200) | more than 3 years ago | (#34731152)

In medical science especially, the fashion for meta-analyses can't help. Aggregating data from different studies results in all sorts of artifacts appearing.

There's nothing wrong with the scientific method (1)

surveyork (1505897) | more than 3 years ago | (#34731874)

There's something wrong with some scientists. TFS cites problems in psychology, ecology and medicine. That's no surprise. As it has been pointed in the above comment , there's a quite broad misuse of statistics, so in the end, the saying "lies, damn lies and statistics" comes true, since statistics are interpreted the way the scientist wants to better fit his hypothesis. In cosmology and physics there's also something going on: evidence for a pre-Big-Bang universe, evidence of multiverses, dark matter signatures, proton shrinking 4%, string theories... The scientific method is valid, you just have to use it properly.

Re:There's nothing wrong with the scientific metho (1)

Spazmania (174582) | more than 3 years ago | (#34733240)

Many if not most published climate-change papers describe some interesting results and then go on to draw broad conclusions that don't follow from the data, that are at best hypothesis the data suggests experimenting with. What percentage of scientists in a field have to get it wrong before the problem is with the methodologies and behavior accepted during peer review rather than the specific individuals?

Pet theories and the need to publish... (1)

Knightman (142928) | more than 3 years ago | (#34735194)

In other words the "scientists" are sloppy and they produce results that fit their pet theory... Nothing new really, been going on for a long long time.

It's troubling though that it's so widespread but I guess most of them need to produce results so they can justify their need for money/tenure etc. and that usually means taking shortcuts and fudge the data.

Scientific method is fine (1)

airfoobar (1853132) | more than 3 years ago | (#34736950)

The problem is that many scientists, just like many politicians, can be bought and sold for the right price. The problem is that we live in a world that cares more about money than about science or culture or the public or the environment...
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>