Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Science and the Shortcomings of Statistics

samzenpus posted more than 4 years ago | from the 14%-of-people-know-that-statistics-can-prove-anything dept.

Math 429

Kilrah_il writes "The linked article provides a short summary of the problems scientists have with statistics. As an intern, I see it many times: Doctors do lots of research but don't have a clue when it comes to statistics — and in the social science area, it's even worse. From the article: 'Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous, and tests of medical dangers or treatments are often contradictory and confusing.'"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered


Lies, Damned Lies, and Statistics. (5, Informative)

Shadow of Eternity (795165) | more than 4 years ago | (#31518344)

In other news math may not lie but people still can, all the honesty and good statistics in the world doesnt help end-user stupidity, and there are statistically two popes per square kilometer in the vatican.

Re:Lies, Damned Lies, and Statistics. (3, Informative)

jeckled (1716002) | more than 4 years ago | (#31518400)

Also, statistics are often manipulated to suggest correlations where there are none.

Re:Lies, Damned Lies, and Statistics. (2, Insightful)

Michael Kristopeit (1751814) | more than 4 years ago | (#31518558)

valid correlations are often manipulated to suggest causation where there is none.

in the end, it's only a problem if the person listening is an idiot...

Re:Lies, Damned Lies, and Statistics. (5, Funny)

Cryacin (657549) | more than 4 years ago | (#31518596)

Exactly. I would never believe a statistic that I did not make up myself!

Re:Lies, Damned Lies, and Statistics. (4, Funny)

dwarfsoft (461760) | more than 4 years ago | (#31518566)

As with everything, xkcd delivers [xkcd.com]. My personal favorite :)

People often get caught assuming that Correlation == Causation.

Re:Lies, Damned Lies, and Statistics. (0)

Anonymous Coward | more than 4 years ago | (#31518754)

People often get caught assuming that Correlation == Causation.

Well that's certainly a new one to me!

As you can tell this is my first time on slashdot.

Re:Lies, Damned Lies, and Statistics. (1)

Entrope (68843) | more than 4 years ago | (#31518836)

That error at least something that is debunked fairly often. It's harder to explain to most people (those without sufficient background in mathematics or statistics) that you're going to have results that falsely appear to be "statistically significant" if you repeat a random trial often enough. For example, you should expect to find a wrong p=0.05 result within 15 trials. For whatever reason, people hear jargon like "statistically significant" and forget the truism about a stopped clock.

The problem is statisticians (5, Insightful)

BrokenHalo (565198) | more than 4 years ago | (#31518544)

In other news math may not lie but people still can...

Usually (in science at least) it's not even a matter of lying. Part of the problem is that the multi-headed monster that statistics has become has a tendency to lead people to over-use numerical "answers" vomited up by stats packages, without really understanding what they are for, or how to interpret them.

Statistics are very useful for predicting certain things, but all too often they are submitted as "proof" of a given condition, which is dangerous. Sometimes we need to throw away statistics and start applying common sense.

Re:The problem is statisticians (4, Interesting)

caerwyn (38056) | more than 4 years ago | (#31518792)

Actually, one of the most dangerous uses of statistics is exactly predicting with them inappropriately. Curve fitting is especially prone to this error- attempting to make any predictions outside of the central mass of the points used to *produce* the curve is completely bogus, and yet people do it all the time.

Re:Lies, Damned Lies, and Statistics. (1)

Opportunist (166417) | more than 4 years ago | (#31518776)

While we're at it, stay away from hospitals! Most people in civilised countries die there rather than anywhere else!

Re:Lies, Damned Lies, and Statistics. (0)

Anonymous Coward | more than 4 years ago | (#31518812)

The problem with viewing statistics in medicine is that statistics are also used by politicians. And since politicians are frequently liars, there is a strong correlation that using medicine will make you a liar. Therefore, it is highly likely that 75% of politicians have used medicine solely to make themselves liars, with a margin of error of 2.5%.

Re:Lies, Damned Lies, and Statistics. (1)

Lars T. (470328) | more than 4 years ago | (#31518850)

and there are statistically two popes per square kilometer in the vatican.

But the expected value of popes per Vatican City is still one.

Its common knowledge (0, Redundant)

Johnny Fusion (658094) | more than 4 years ago | (#31518348)

That 77.28% of all statistics are made up.

Re:Its common knowledge (2, Funny)

snl2587 (1177409) | more than 4 years ago | (#31518390)

How do you figure that? My latest calculations placed it at 70% [Note: Error +/- 10%].

Re:Its common knowledge (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31518512)

That makes sense.

I read that the percentage of niggers who have herpes was around 50%, but I think it's more around 110% with a 10% margin of error. As I recall, there was an additional complication involving the methods of that study -- the standard deviation was difficult to quantify because all niggers are deviants -- and the graphs looked more like a uniform distribution than a standard bell-curve.

Re:Its common knowledge (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31518710)

That 77.28% of all statistics are made up.

Yes, however in Soviet Russia, 77.28 percent of car analogies are made about YOU! First post!

Re:Its common knowledge (3, Insightful)

Opportunist (166417) | more than 4 years ago | (#31518760)

And 77.335% of all statistics claim more accuracy than their expected deviation warrants.

Re:Its common knowledge (0)

Anonymous Coward | more than 4 years ago | (#31518864)

Luckily, only 34.48% of the public ever pays attention to statistics. Only 54.13% of which can properly understand what they mean.

The world of the average Joe is mean.

Attention White People!! (-1)

Anonymous Coward | more than 4 years ago | (#31518352)

I implore all my fellow race-aware white brethen to stop using Windoze. Join the land of Linux where we have nigger-free code. You don't want your computer, wife, girlfriend or daughters to be tainted by M$ code that isn't nigger-free do you?

Re:Attention White People!! (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#31518410)

I'm Linux Torvalds and I approve of this message.

Summery? (4, Funny)

sincewhen (640526) | more than 4 years ago | (#31518394)

It's not just statistics that people have a problem with...

Re:Summery? (3, Funny)

oGMo (379) | more than 4 years ago | (#31518672)

From your sig:

-- Braden's law of data: All data spends some of it's lifetime in an excel spreadsheet.

What's that law about spelling/grammar corrections inevitably having spelling or grammar mistakes in them?

Re:Summery? (0)

Anonymous Coward | more than 4 years ago | (#31518688)

What's that law about spelling/grammar corrections inevitably having spelling or grammar mistakes in them?


Long winded troll (0)

TapeCutter (624760) | more than 4 years ago | (#31518396)

The entire article can be summed up by the tiresome cliche "correlation != causation". To make matters worse they quote an economic historian who does not understand that science is not in the bussiness of proof... "“That test itself is neither necessary nor sufficient for proving a scientific result,” asserts Stephen Ziliak, an economic historian at Roosevelt University in Chicago."

Re:Long winded troll (1, Insightful)

Anonymous Coward | more than 4 years ago | (#31518444)

Statistics is terrible for proving things, but rather good at disproving them.

Re:Long winded troll (1)

khallow (566160) | more than 4 years ago | (#31518482)

The entire article can be summed up by the tiresome cliche "correlation != causation".

That misses a lot of the problem. For example, observer bias through poor statistical design of the experiment or throwing out data can cause the appearance of correlation or causation in data that isn't so.

Re:Long winded troll (1)

Nefarious Wheel (628136) | more than 4 years ago | (#31518490)

The entire article can be summed up by the tiresome cliche "correlation != causation"...

The logical fallacy is called "post hoc, ergo propter hoc" - "after this, therefore because of this".

Sort of like - I get a headache every time someone turns on the television, therefore headaches are caused by the television.

Oh, hang on...

Re:Long winded troll (1)

Homburg (213427) | more than 4 years ago | (#31518526)

science is not in the bussiness of proof

So what is it in the business of?

Re:Long winded troll (1, Interesting)

Anonymous Coward | more than 4 years ago | (#31518632)

Evidence. Big difference.

Re:Long winded troll (1)

obliv!on (1160633) | more than 4 years ago | (#31518640)

Science is in the business of probably knowledge. So they really need to improve their probability and statistics knowledge.

Re:Long winded troll (0, Insightful)

Anonymous Coward | more than 4 years ago | (#31518580)

No it can't. The article does a fairly good job at summarizing the systematic conceptual mistake of misinterpreting a p-value as representing a probability that the hypothesis is not true, among other things, and a somewhat less good job at introducing Bayesian statistics. These are subtler issues than the true-but-trivial—and tiresome—cliché you refer to.

Re:Long winded troll (1)

TapeCutter (624760) | more than 4 years ago | (#31518786)

No it can't, what?

"These are subtler issues than the true-but-trivial—and tiresome—cliché you refer to."

Actually the subtler issue here has nothing to do with statistics, they are implying peer-review does not work.

What it actually said (5, Informative)

williamhb (758070) | more than 4 years ago | (#31518818)

Contrary to the parent poster's claim, the article does not focus on correlation vs causation. It focuses on people getting the correlation wrong in the first place. It lists several common mistakes scientists make when writing up research studies. (Not all scientists are very good at stats). These include:
  • If you run enough studies you are almost certain to find a difference that appears statistically significant at the p<0.05 level through chance alone. (It is incredibly unlikely that you will win the lottery; but across the whole pool of tickets someone wins it most weeks.) That makes studies that bulk analyze large amounts of data against many different factors, actively hunting for something that is significantly different, erroneous.
  • "p < 0.05" does not mean there is a 95% chance of your result being "true"; it just means that someone else rolling dice has a 5% chance of achieving the same result through chance alone.
  • Tests are often combined in ways that are mathematically inconsistent
  • Finding a statistical effect does not mean it is a strong effect
  • You cannot simply compare effect sizes between two studies because the results of their control groups may differ ("effect size analysis" is usually wrong)
  • Failing to find a significant effect does not mean there is no effect ("we found there was no significant effect on..." is misleading because "no satistical significance" is "no information" [your study didn't tell anybody anything] not "no effect" -- to prove "no effect" you need a different statistical test)

And lots of others. It then suggests Bayesian reasoning as an alternative to traditional statistical tests.

Most post-PhD scientists are aware of the common mistakes, but being aware that we make mistakes doesn't necessarily stop us from making them. If you chose a random set of conference proceedings, it is almost certain you will find at least one paper (and I suspect usually a dozen or more) that have statistical mistakes in them.

I dated a short, summery girl once (1)

Chess Piece Face (247847) | more than 4 years ago | (#31518398)

She was like a little ball of sunshine.

As for statistics, does this really surprise anyone in a time when net polls are being reported as hard news?

Example: Standard Deviation (4, Interesting)

cytoman (792326) | more than 4 years ago | (#31518402)

My doctor was explaining to me that my blood sugar readings should not have a standard deviation of more than 1/3rd of the average blood sugar reading. Just to test if he knew what it meant, I asked him what a standard deviation was. Oh the fun when he tried to bullshit his way out of that one! He eventually told me that when I plot my data in Excel I can ask it to give me statistics on the column and it would mention what the standard deviation value was. But when I pressed on and asked him what a standard deviation is, he shooed me off and told me to go look it up. Never did he confess that he had no clue.

Re:Example: Standard Deviation (1)

LingNoi (1066278) | more than 4 years ago | (#31518494)

I don't see this being a problem. It's not his job to know, just like it is not his job to know how to write the excel spreadsheet to come out with the values he uses to help you.

That being said it wouldn't hurt if everyone had a better understanding of statistics.

Re:Example: Standard Deviation (4, Informative)

cytoman (792326) | more than 4 years ago | (#31518532)

Standard deviation is what you learn very early in school. And this was a endocrinologist - a specialist who no doubt took a lot of Biostatistics courses and such, and used a lot of statistics all through his education. And you are telling me that it's not his "job" to know? Wow! We are talking the most basic stuff that anyone with a degree in the sciences should know. It's almost like saying that an English major can be excused if he doesn't know that 2+2=4 because "it's not his job to know".

Re:Example: Standard Deviation (1, Interesting)

ottothecow (600101) | more than 4 years ago | (#31518638)

Nah, it is his job to know how to use the calculation.

I certainly don't remember how to do all those statistics calculations by hand but I use SAS and excel almost every day and they don't seem to have forgotten...give me a few more years and I might be at the point where I wouldn't be confident trying to explain what a standard deviation actually "is"

Re:Example: Standard Deviation (4, Insightful)

cytoman (792326) | more than 4 years ago | (#31518684)

You are missing the point - he did not know what a standard deviation means! That is unforgivable for anyone with a medical degree...hell, it's unforgivable for anyone who has passed a course in statistics in school.

Re:Example: Standard Deviation (0, Redundant)

Lunix Nutcase (1092239) | more than 4 years ago | (#31518744)

Yeah because you never forget anything, right?

Re:Example: Standard Deviation (1)

Colonel Korn (1258968) | more than 4 years ago | (#31518830)

Yeah because you never forget anything, right?

Well he should be able to at least say that it represents variability in a repeated measurement. He should at least know that.

Maybe he did know that but was stumbling because he was embarrassed for not knowing the formula.

Re:Example: Standard Deviation (2, Informative)

Ethanol-fueled (1125189) | more than 4 years ago | (#31518770)

s = sample standard deviation = sqrt((sum(x-xbar)^2)/(n-1)), where xbar is the mean
sigma = population standard deviation = sqrt((sum(x-mu)^2)/N), where mu is the mean
s is approximately equal to (highestValue-lowestValue)/4, range rule of thumb
Unusual values are outside +/- 2 standard deviations
Z = ((x-mu)/sigma) where Z is in terms of standard deviations.

Re:Example: Standard Deviation (2, Insightful)

PSUspud (7236) | more than 4 years ago | (#31518766)

As a statistics teacher (HS / Tech school level), this doesn't surprise me in the least. Statistics and statistics education has become a giant game of "plug the numbers in and damn the understanding". When a student has never calculated a standard deviation by hand, how can they be expected to know what the heck a root mean square deviation from the sample mean really is?

Going further, I would say that statistics is a tool for answering questions. Like any other tool, it works well for some jobs and not for others. So far, no problem. But the problem comes from students that are just not willing to understand the questions that statistics can answer. Case in point -- a p value of 0.05 does _not_ mean that the null hypothesis has a 95% chance of being wrong. That's what stats students want it to mean, because they are not willing to ask the questions that stats can answer.

Until students are willing to actually do the work, for the sake of actually learning, I don't see any hope.

Re:Example: Standard Deviation (0)

Anonymous Coward | more than 4 years ago | (#31518796)

Why would you care to test him? You sound like a very unpleasant person to deal with.

Re:Example: Standard Deviation (4, Interesting)

Opportunist (166417) | more than 4 years ago | (#31518848)

Doctors are notoriously bad with statistics. But the real kings of bad statistics are psychiatrists.

Notice how a LOT of studies in psychiatry are essentially statistics, statistics and a bit of statistics? It might be the reason why a lot of the courses you have to pass to become a shrink also consist of a lot of statistics, statistics... you get the idea.

NOBODY who decides that his course of studies would be psychiatry decided for that because he enjoy statistics that much, though. Actually, most psych students struggle badly with statistics. Psychiatry is one of the fields where the label doesn't match the contents. It looks like you're going to do a lot of messing with people's minds (aka "solving their psychology problems") but actually, judging from the courses, you become a refined statistician who had a bit of a counceling tutoring on the side.

That's not what people become shrinks for, though. They want to sit in their office, put people on their couch (or, more modern, in a comfy chair) and get 100 bucks an hour for listening to some idiot whine. And most do just that and will do fine.

It gets bizarre when they somehow end up in a spot where they have to rely on their statistics. Hey, you got a masters in that, and that entails a buttload of statistics, so you can do it... Nobody really cares that 9 out of 10 that somehow managed to get their diploma by either learning what they absolutely needed (and forgot it right after the test, certain that they'd never need it again, because ... ya know, listening to idiots and stuff, not sitting there plotting standard deviations...) or by cribbing altogether.

And then you get studies of the usefulness of psychotropic drugs and wonder whose black hole they pulled that out of...

Re:Example: Standard Deviation (3, Interesting)

JumpDrive (1437895) | more than 4 years ago | (#31518852)

I agree with your concerns. Being a chemical engineer and a physical scientist, I have often found medical doctors understanding of chemistry and other sciences lacking. I once had an argument about chemical kinetics involved in a prescription drug I was taking, he basically told me I didn't know what I was talking about and blew me off. After another run in with him over another issue I fired him. But that's just one of my personal issues with a doctor.

Back when I was in graduate school me and my colleagues in graduate science taught pre-med chemistry and physics, which was a really watered down version of chemistry and physics which were taught to engineers and science majors. To be honest I thought it was kind of scary. All these years I was taught that medical student were supposed to be the best and the brightest, but we spoon fed them "baby chemistry" and "baby physics".

Since that time I have had many discussions with professors about this and they and I have come to the same conclusion, "the best and the brightest do not go into medical school". Thirty or forty years ago this may have been true, but economics has taken a turn and it just isn't the case anymore.

And why would they? They can make more money on Wall Street, they don't have to hassle with bureaucracy of health insurance, they don't have to hassle with lawyers, so why would the best and brightest go into medicine.

And you want to know what kind of income a hot little girl with a business degree can get. Pharmaceutical sales can pay 6 figures for one good figure. So the next time you see that good looking girl pulling that bag through your doctors office realize she is probably making a lot of money. More money than the average general practitioner .

Maths anxiety (0)

Anonymous Coward | more than 4 years ago | (#31518416)

I've found that statistics knowledge amongst maths-anxious non-math-majors only seems to deepen the fear they have of mathematics in general. One was talking about a basic regression model. He had no appreciation for what anything was (i.e. it was all "plug in the numbers" for him), and when I tried to explain, he'd consider doing mathematics to be some kind of labour to be avoided!

Luckily this was bachelor's level.

Maths anxiety is a viscous cycle which needs to be broken at an early age at home.

Not only, but also... (0)

Anonymous Coward | more than 4 years ago | (#31518456)

A lot of the "science" done today isn't actually much more than agreeing that their idea is right and "well supported." It's the mathematics; the statistics and models that they use that are most often the first and most obvious signal flares. An awful lot of them should really just give their degrees back to their university and go off do something useful, like play in the traffic, or maybe catch a flight to Saudi and run up and down the main streets yelling insults to Islam, or even tootle down to Chelsea's ground and start singing Inter Milan chants. The claims of "science" and "it's a scientific fact" in the new millennium all too often tend to be completely against reality.

It's a tough situation (1)

robbyjo (315601) | more than 4 years ago | (#31518464)

Actually, it's a tough situation. There is no real life experimental data can 100% fit the assumptions of commonly used statistical models. Real life data is messy. There is some degree of simplification. In addition, resorting to whiz-bang fancy methods that "fit" the real data may not be easily interpretable. Ease of result interpretability is what medical scientists want. There are other issues as well, such as computing time, equations derivability, etc.

In addition, many many medical scientists use statistics as a tool to filter things (e.g. candidate genes, target enzymes, treatments, etc). In this case, 100% accuracy is not really important. Once the scientists narrow down the genes, they can test the validity directly in either test animals or real people.

No surprise here (0)

FrozenGeek (1219968) | more than 4 years ago | (#31518472)

These days, most people cannot deal with basic algebra. Case in point: my sister, who has a master's degree in the social sciences, reaches for a calculator to calculate the sales tax on purchases (and she does it because she cannot manage without it).

Why on earth would we expect Joe Average to be able to comprehend the meaning of statistics?

Re:No surprise here (1)

initdeep (1073290) | more than 4 years ago | (#31518510)

this is why people now consider master's degrees to be the equivalent of a high school diploma.

if you want real fun, take the average master's degree idiot and start having them manually add fractions without changing them to decimals. such as adding a bunch of measurements off a tape measure together.

hilarity ensues....................

Re:No surprise here (5, Funny)

Homburg (213427) | more than 4 years ago | (#31518538)

I think your example would be more persuasive if it involved algebra, though.

Re:No surprise here (2, Informative)

skine (1524819) | more than 4 years ago | (#31518560)

It's perfectly reasonable that someone use a calculator for sales tax (if an exact answer is desired).

Also, sales tax is multiplication - not algebra.

Re:No surprise here (1)

Lunix Nutcase (1092239) | more than 4 years ago | (#31518568)

And what are we supposed to make of your post where your supposed case for people not knowing algebra has nothing to do with algebra?

Re:No surprise here (5, Insightful)

coolsnowmen (695297) | more than 4 years ago | (#31518582)

You are a jerk.
  You are insulting your sister because she is bad at mental math? It is a skill; one not required for extensive knowledge of the social sciences. Additionally, maybe if sales tax is simple in your state like 10%, but where I live it is 4.5% which is not always easy to get exactly right in your head.

I had a roommate who was brilliant,funny, a singer and an artist, and yet, he couldn't calculate tip to save his life, but I don't certainly hold that against him.

Re:No surprise here (1)

shermo (1284310) | more than 4 years ago | (#31518722)

I had a friend who is doing a PhD in maths and he can't calculate basic arithmetic to save his life. It's a redundant skill for pretty much everyone.

Re:No surprise here (1, Informative)

Anonymous Coward | more than 4 years ago | (#31518676)

Arithmetic is not algebra. Arithmetic is "What's 10% of $24.45?" Algebra would be "On a given day i, John sells n_i apples to Peter at x_i dollars each, and this price includes sales tax which is a constant proportion 0p1. Let x_1= .. x_2= ... ... What is the tax on the apples sold on days 1 to 12 inclusive?"

The difference is 24.45 . 10/100 versus p\sum_{i=1}^{12} n_ix_i. Granted, there isn't much difference there really, but come on, there is a time and a place for everything, calculators included.

Re:No surprise here (0)

Anonymous Coward | more than 4 years ago | (#31518690)

I bet she knows what algebra is.

Re:No surprise here (1)

Antique Geekmeister (740220) | more than 4 years ago | (#31518762)

Given that sales tax varies based on type of purchase in some states, and is weird numbers like 6.5% in others, it can vary quite a lot. And oh, my dear lord, try dealing with "valua-added-tax" in Europe....

Two weeks of six sigma classes... (2, Funny)

ctmurray (1475885) | more than 4 years ago | (#31518476)

Our company six sigma training included two weeks of collecting and analyzing data with a stats package. I got enough experience to even train me how to use the program. I can still do a few things that come up regularly. Probably the best thing to come out of six sigma (for me at least).

Personal experience (5, Interesting)

nanoakron (234907) | more than 4 years ago | (#31518484)

As a doctor myself, I feel I should add my $0.02...

Throughout med school we had the odd scattered lecture on statistics, and later when reading papers I used to skim over most of the maths just to look for the P value at the end (one representation of how statistically significant a result is).

However, I then took a formal stats course and was amazed at how little I understood - Monte Carlo techniques, Markov models, and even something as trivial yet important as the difference between a parametric versus a non-parametric test.

And then it struck me - most of the research I had read had applied parametric statistical tests to their data - that it, the researchers made an assumption that the underlying distribution of results would fall on a normal curve. Yet this simple assumption may be all it takes to skew the data when they should have chosen a non-parametric test instead.

So yes, stats are vitally important, badly taught, and focus too much on the maths rather than the concepts. Remember that we're doctors, not mathematicians - the last set of sums I did were in high school. If I need to analyse data, I'll probably plug it into SPSS - although now with my eyes open.


Re:Personal experience (0)

Anonymous Coward | more than 4 years ago | (#31518550)

Of course nobody needs to learn everything about stats in order to use it, but the "focusses to much on the maths" is something that can't be avoided: in order to run a marathon, you need to learn how to walk, and then how to run. Walking and running, in principle, have nothing to do with marathons.

Re:Personal experience (1)

Cryacin (657549) | more than 4 years ago | (#31518622)

I figured that marathons are just running on a much higher level. Makes sense when you think that the sport was invented by some Greek guy running towards Sparta with an army of soldier with pointy spears chasing after him. That sort of thing would make anyone run long and hard.

Re:Personal experience (0)

Anonymous Coward | more than 4 years ago | (#31518600)

Well one thing to consider is the stigma in the honors/college prep programs in HS where statistics is looked at as the "Math for dumb kids" where the "brighter" students take calculus and the like.

Re:Personal experience (1)

spasm (79260) | more than 4 years ago | (#31518680)

And that, my friend, is why the NIH's constant push to produce more 'physician-researchers' continues to drive me nuts. Because they rarely insist K awards and other early-career training mechanisms require physicians intending to do research in areas where stats are important actually get any stats training..

Re:Personal experience (5, Insightful)

Frequency Domain (601421) | more than 4 years ago | (#31518718)

...And then it struck me - most of the research I had read had applied parametric statistical tests to their data - that it, the researchers made an assumption that the underlying distribution of results would fall on a normal curve. Yet this simple assumption may be all it takes to skew the data when they should have chosen a non-parametric test instead.

So yes, stats are vitally important, badly taught, and focus too much on the maths rather than the concepts. Remember that we're doctors, not mathematicians - the last set of sums I did were in high school. If I need to analyse data, I'll probably plug it into SPSS - although now with my eyes open.

That's a good insight. I'm a statistics professor, and some of the problems I see are a) people generally get exposed to a single course in statistics; b) they're usually mathematically unprepared for it; c) so much gets squeezed into that one opportunity that heads are exploding; d) because of (a) - (c), everybody wants you to "just give 'em the formula"; e) since statistics is so widely used, there's a plethora of courses that are being taught by people who themselves are victims/products of (a) - (d), and are very happy to "just give 'em the formula"; and so e) most people plug and chug data through a stats package with no idea of the applicability, limitations, and interpretation of the results. The sheer volume of bad analyses is enough to make you weep, and contributes to the widely held perception about "lies, damned lies, and statistics". And that completely ignores the intentional falsehoods propagated by people who are trying to support various advocacy viewpoints, and will happily mislead the public with biased samples, Simpson's paradox [wikipedia.org], invalid assumptions, etc.

Re:Personal experience (0)

Anonymous Coward | more than 4 years ago | (#31518802)

I'm more worried by the idea that a doctor is some kind of scientist. They aren't, any more than a car mechanic, plumber or cable guy is.

Pirates cause cool weather (1)

wisnoskij (1206448) | more than 4 years ago | (#31518486)

Funny Stat correlation: http://www.seanbonner.com/blog/archives/piratesarecool.jpg [seanbonner.com]

Re:Pirates cause cool weather (1)

symbolset (646467) | more than 4 years ago | (#31518518)

Current meme trends indicate an 87% chance this will become a global warming thread.

Re:Pirates cause cool weather (1)

sycodon (149926) | more than 4 years ago | (#31518700)

It's funny that you say that because is not global warming a statistical creature?

Re:Pirates cause cool weather (1)

Nefarious Wheel (628136) | more than 4 years ago | (#31518574)

Careful, sir or madam, with that graph you are treading dangerously close to a theological argument here. Global Warming is the Flying Spaghetti Monster's way of telling us we need more pirates. If you want to know exactly how pirates and global warming correlate, please send money, and we will lease you an AVOM (Awesome Volt-Ohm Meter) with a blank face with which you can scare yourself until your midichlorians take over your reflexes.

"Luke Skywalker's a Jedi of course;

And he's prone to have much intercourse;

So he calls up his Princess, to beg for some incest,

Grabs a blindfold and uses the Force.

Rats and Stats (1, Interesting)

Anonymous Coward | more than 4 years ago | (#31518508)

When I did my BA in psychology Statistics was the core of the degree. It was the one subject that you could not escape and had to take for the full year every year of the degree. I heard later that the Psychology department at that Uni was sometimes disparagingly described as teaching Rats and Stats psychology.

Statistical assumptions are often ignored (1, Insightful)

Anonymous Coward | more than 4 years ago | (#31518522)

Statistical methods are typically developed for fairly specific mathematical models. A practitioner may error greatly by using a statistical method outside of its intended purview. For example, many statistical tests assume that different groups of observations are independent or correlated in a specific way. If this isn't true then the resulting inferences can be very inaccurate.

Unfortunately the spread of "easy to use" statistical software is making this problem worse. Many scientists just enter their data and select an analysis from a drop-down menu - thinking that just because their data is in the right format that the results will accurate. It would be better if people had to think about what analysis to choose rather than just treating the choice of a test like the choice of a visual effect in photoshop.

IAAS (statistician), for what it's worth...

Fair and Balanced: Fox quotes the Bible as saying (2, Funny)

vandelais (164490) | more than 4 years ago | (#31518524)

that there are only 3 kinds of scientists: those that are good at math and those that aren't.

Current Data (1)

Dripdry (1062282) | more than 4 years ago | (#31518556)

Does that mean that we should send people who know what they're doing to sort through results and draw more meaningful conclusions? Or just rerun the tests?

This seems obvious, so please don't waste mod points here, people who know what they're actually talking about will probably chime in.

Study says (1)

oldhack (1037484) | more than 4 years ago | (#31518576)

They're all buncha crap, and I say this with 95% confidence interval, or sum such stat shit that I wish I can remember.

Countless? (1)

andr00oo (915001) | more than 4 years ago | (#31518584)

> countless conclusions in the scientific literature are erroneous

Number of Publications: Finite
Number of Conclusions: Finite
Time taken to count erroneous conclusions: Finite

Countless Conclusions? I don't think so!

A large but unspecified number of conclusions in the scientific literature are erroneous: Not so compelling

Excellent (2, Insightful)

zoso1132 (1303697) | more than 4 years ago | (#31518594)

One of the best articles I've seen on stats (and their misuse). I'm taking a data analysis course at the moment and I've spent at least a dozen hours simply computing confidence intervals, testing the null hypothesis, and determining significance. It really has changed how I view statistics because it keeps pounding in these very key but oft-ignored principles.

bad title (5, Interesting)

obliv!on (1160633) | more than 4 years ago | (#31518608)

It is not a shortcoming of statistics that other people, like various scientists who aren't statisticians, don't know how to use or properly interpret statistics. It is a shortcoming of their knowledge.

It is not a shortcoming of the Copenhagen interpretation of quantum mechanics or the Chicago school of economics if I don't understand or know how to correctly interpret their results. It is my shortcoming and fault for not knowing enough to connect the dots.

I do statistical research some of that is through interacting with researchers in the biosciences. Often when I go to talk to a researcher and ask them if they could use some statistical or mathematical or computational assistance with their research it has almost always been a fruitful starting point to long conversations and getting into the research. Now sometimes it was simply a matter of looking at their F-test results or ANOVA scores and telling them what it meant (like with a regression model relating proportions of certain characteristics between taxa), more useful interactions for me often mean working on new algorithms or estimators or working with fitting a model from their empirical data because there isn't a reliable standard model to work off of (like intergenic distance between genes in an operon) that kind of challenge makes less engaging work worth the hassle. Maybe I'm odd because I've worked hard to have a good background in both statistics and biology, but I shouldn't be.

Although here is an observation that perhaps supports some of the intent of the article from my own experience. I was speaking with a biology graduate student and it came up that they had a biostatistics course in the department. Of course as a statistician my mind goes towards survival function, failure rate, life tables, censored data, bioassy, epidemiology, microarrays, clincal trials, topics along those lines. It turned out their course focused z tests, t tests, f tests, confidence intervals, point predictions, least squares regression, multiple regression, ANOVA, and things along these lines just with simulated problems in a lab setting. That is not necessarily a bad thing, but much of the core math was under played or missing like model assumptions and alternate formulations or things like dummy variables. The worst part was that even though they were doing well with the class they had no confidence in actually using the statistics and didn't understand how to interpret the meaning of something like a confidence interval, they knew how to calculate one, but it wasn't clear what it actually meant to them.

The corollary to the notion in the summary I'd rant and claim is that scientists overall have less than desirable skills in mathematics, statistics, and computation than those who studied those disciplines principally and that's hurting science. However many in those three disciplines really know little beyond basic results in any of the sciences which hurts the applicability of these mathematical fields to the sciences and likely hurt our ability to develop certain types of discipline specific results that can be generalized from work in application problems.

In either case whether you're a typical scientist or a typical math/stat/comp person in order to become proficient enough in the other areas it requires going an awfully long out of the way compared to any counterpart who simply does not care and goes straight through as many before have. While in some areas of research on either side it is no problem to do as has been done and not further knowledge into those other areas. Increasingly results that have the highest levels of impact are coming more and more from truly interdisciplinary research. In order to further encourage that for those who are interested in such fields (aside from making more clear what areas in any of the fields fringe to such interdisciplinary work) we need more incentive to study more than one field and/or better ways of enabling fruitful cooperation between the camps.

Stats are completely useless!!!! (0)

Anonymous Coward | more than 4 years ago | (#31518620)

E.g.: Study shows a cancer group of size 3000 is cured by drug A 99% of the time.
1% it fails.

30 patients are dead. No correlation it seems at the time.

*2970* patients are saved.

20 years later, it's proven a dormant undetected/sequenced gene is responsible for the 1% failure of the drug, making it ineffective.

Statistics allowed the drug to be approved at the time that saved millions of lives.

I hate stats as much as 70% of the average Joe :), but anyone with an education knows its importance. (Esp. those dudes that are breathing right now because that drug saved their lives)

So the article in short, don't lie about your stats(or in general don't lie!) and you can benefit humanity.

PhD Candidate in Biostatistics Here (-1, Troll)

dorpus (636554) | more than 4 years ago | (#31518634)

I used to be a devout atheist. But the more I've learned about science, the more I've learned that it is a giant flimsy pile of assumptions. In reality, science works just like a religion, with all the same dogma, persecution of questions it doesn't like. We have a theory of evolution that is being overturned every week, whenever someone finds an old bone in the "wrong" place. We have a geology built on a faith in isotopes that are supposedly trapped in mud for millions of years, an untestable postulate. We have a public health system whose job is to tell people what to do, though its recommendations get reversed every few years. Everybody should use hand sanitizers all the time, oh wait it has no effect. Table salt should be outlawed, oh wait salt doesn't cause any harm. Everybody over age 40 should take heart drugs every day, whether they have problems or not -- oh wait, heart pills turn out to have no effect on the risk of heart disease after all. Everybody should get colonoscopies, oh wait it causes more problems than it solves. Breast cancer screening, same story. Everybody should have red ribbons on their car antennas to show support for AIDS victims -- even though diabetes kills far more people and its effects are just as lethal. So, diabetes victims deserve no sympathy because it is a disease of sloth -- though the same could be said of AIDS. About 14% of Americans over age 30 have diabetes, according to this:

http://www.pophealthmetrics.com/content/7/1/16 [pophealthmetrics.com]

That sounds a lot higher than the 0.7% prevalence of AIDS in America. Both are incurable but treatable diseases. Incidentally, there is fairly good evidence that some types of diabetes are caused by viruses such as Coxsackie B4 virus. But even if it is proven conclusively, I have a feeling it won't produce the same headlines as the discovery of the HIV virus.

Re:PhD Candidate in Biostatistics Here (2, Funny)

MindlessAutomata (1282944) | more than 4 years ago | (#31518652)

I don't have to be a statistician to know that the above post is 97% bullshit.

Re:PhD Candidate in Biostatistics Here (1)

dorpus (636554) | more than 4 years ago | (#31518724)

You think I'm full of it? Wait till you hear professors at seminars, making up whatever theories they like. I've witnessed professors from household-name schools acting like this.

Probability Theory: The Logic of Science (1)

DuncanFoley (867254) | more than 4 years ago | (#31518636)

The clearest discussion of the logic of probability reasoning I know of is E.T. Jaynes' Probability Theory: The Logic of Science. (Cambridge University Press). Many of Jaynes' excellent papers on statistics are downloadable from http://bayes.wustl.edu/etj/etj.html [wustl.edu].

stop making them vie for grant money (0)

Anonymous Coward | more than 4 years ago | (#31518660)

maybe we'd get some honest science if it wasn't a bidding war.

only in medicine (5, Interesting)

rook166 (1459561) | more than 4 years ago | (#31518692)

In reading a couple of these types of articles recently I've noticed that the articles always talk about this being a problem across all journals, but only seem to mention a couple of different disciplines - medicine usually chief among them. Has anyone heard/read anything naming a hard science (e.g. chemistry or physics) as full of bad stats? My hunch is that this happens most often in medicine because you have the combination of controlling for a lot of variables as well as inadequate mathematics training.

Bad outcomes due to statistics? (1)

scdeimos (632778) | more than 4 years ago | (#31518734)

From TFA:

“There is increasing concern,” declared epidemiologist John Ioannidis in a highly cited 2005 paper in PLoS Medicine, “that in modern research, false findings may be the majority or even the vast majority of published research claims.”

One has to wonder, though: how much of that is due to misuse of statistics and how much is because it's paid research expected to get certain results in favour of those paying for the research?

Not Scientists (0, Flamebait)

Secret Rabbit (914973) | more than 4 years ago | (#31518750)

Ok, so the referenced fields that have problem with stats are both not Sciences. Medicine has no theories that govern the human body. All they do is memorize a bunch of crap and then poke some squishy bits and memorize how it looks and feels when healthy/normal v.s. unhealthy/abnormal. It's really the Engineers, Physicists, Chemists and to a lesser extent (though they are gaining market-share) Biologists, that make the true breakthroughs in Medicine.

And the social "sciences" are just plain an embarrassment when it compares to real Science. Seriously...

People in the real Sciences would have been forced to take enough Mathematics and/or Statistics to be able to properly interpret Statistics. And just as importantly, be able to do proper experiment design (Medicine, I'm looking at you). Then there's the whole not being able to tell the difference between causation and correlation. I could go on.

Re:Not Scientists (1)

dorpus (636554) | more than 4 years ago | (#31518764)

On the other hand, plenty of very smart physicists, mathematicians, etc. have approached medicine spouting much the same rhetoric as you. They very quickly became embarassed when they tried to apply their fanciful theories to medicine. If you have a better idea on how to tell apart correlation from causation in a medical context, let them know.

best class ever (1)

Barleymashers (643146) | more than 4 years ago | (#31518794)

I had taken a stats class in undergrad... did not really pay attention as I thought it had no use. While getting my masters I was obligated to take an advance statistics class. Going in, for the life of me I thought it would be a waste - it was the best class I ever took. I was able to use it in my job almost every week if not more ( most of the other classes were theoretical at best and had no real world application ). Ten years later, I still rely on things I learned in that class. Statistics should be mandatory for all in college regardless of major because it can be used for so many things.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account