Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Disappointing Cancer Study Results Go Unreported

timothy posted more than 5 years ago | from the laetrile-is-delicious dept.

Medicine 77

An anonymous reader writes "Science News reports on a new study showing that most cancer drug trial results are never published, probably leaving patients vulnerable to cocktails that have already been shown to be dangerous or useless."

cancel ×

77 comments

Sorry! There are no comments related to the filter you selected.

Hello! (-1, Troll)

Anonymous Coward | more than 5 years ago | (#25180441)

Did someone say cocktails?

Nothing wrong with cocktails! (-1, Offtopic)

perspectival (906492) | more than 5 years ago | (#25180445)

Nothing wrong with being vulnerabile to cocktails!

First Post ... sadly (1, Insightful)

fygment (444210) | more than 5 years ago | (#25180453)

Where's the surprise in this? No news here.

Re:First Post ... sadly (4, Insightful)

moderatorrater (1095745) | more than 5 years ago | (#25181155)

First of all, the mod who gave you "redundant" apparently doesn't realize that this has been widely known for a long time.

Second, this is new because someone's done the actual study and shown the degree to which studies don't go reported. Even if only half of the unreported studies were because of poor results, that's enough to skew things very, very badly.

Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

Re:First Post ... sadly (1)

PC and Sony Fanboy (1248258) | more than 5 years ago | (#25181215)

How is your post news? Everyone knows (or should know) of the survivor bias in research - only results get published. Failures, or lack of results, aren't news worthy- I mean, when was the last time your life made it into the papers?

Re:First Post ... sadly (1)

Wonko the Sane (25252) | more than 5 years ago | (#25181845)

Moreover, he notes that some drug-safety trials did not publish their findings, or did not do so early enough, such that they could inform subsequent trials. This practice is not likely to continue, he points out, since new federal rules will fine investigators who fail to post outcome data for all trials getting money from Uncle Sam. Moreover, for cancer trials, NCI will require that researchers begin reporting outcome data â" treatment successes and failures â" throughout the course of the trial, not just at the end.

That's good news, isn't it?

Re:First Post ... sadly (3, Informative)

stranger_to_himself (1132241) | more than 5 years ago | (#25184929)

Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

That's a bit simplistic - often nobody is directly responsible for not publishing results, it comes about because major medical journals are not interested in pubishing negative findings. There are of course exceptions, when the negative findings come as a big surprise or there is a lot of anticipation about them, like the reports of the Alzheimer's disease vaccination not working earlier this year.

The medical journals themselves know this is a problem, but they aren't sure what to do about it. One suggestion has been the journals should make a 'commitment to publish' for a trial based on its design and importance, and then publish the findings whatever they are, but that would of course mean they'd be publishing a lot less interesting stuff.

Also scientists can't be bothered to write up negative findings, because there are proportionally so many more of them than positive ones.

There is a problem of pharma companies suppressing bad findings or writing misleading journal articles, but that's a wholly independant issue and should be dealt with by the schemes in develpment for enforcing results to be left in public databases after the trials are finishsed. The problem with that is that not many GPs will be searching through these databases

The real solution is probably for doctors not to rely on scientific journals for information (which they often don't really understand), but to wait for advice from bodies like the National Institute for Clinical Excellence [nice.org.uk] or the Cochrane Collaboration [cochrane.org] who do very thorough searches and synthesis.

Plus. (3, Interesting)

Wolfier (94144) | more than 5 years ago | (#25185327)

There's a wide assumption that the researchers themselves really want to publish all results.

Unfortunately, as in almost the entire field of "science" nowadays, it's not the case.

Researchers themselves have a tendency to hide failures - given that most experiments result in failure, they tend to focus on reporting the ones reporting success.

This use of time simply makes most sense - they don't have the time to report all the failures, and reports of failures not as valuable as reports of success only makes it worse - think about what kind of views your peers will have towards you if most of your publications are negative results.

Sadly, this thinking is parasitic and is very prevalent across all research fields.

Journals are very selective given the limited number of pages they have. If I were a journal, I'd pick reports of success first. It's the evil of centrally-controlled publication, and the mindset that, "if a research is of any good value, it must appear on some journal".

Granted, peer review is a good thing, but there must be a way to give researches credibility without getting published on some journal.

Compounded with big-pharma-sourced funding with very fine strings attached...we have a really screwed up system.

Re:First Post ... sadly (1)

raides (881987) | more than 5 years ago | (#25187815)

Um. You name me 1 doctor in this world that posts negative results, and there will be 10 doctors to shut him down completely. This is nothing new. Doctors are not known for posting negative results to any journal. In fact there are age old Doctor jokes you can google that make fun of doctors that do post negative results.

you're dangerously wrong (2, Interesting)

jipn4 (1367823) | more than 5 years ago | (#25191133)

Even if only half of the unreported studies were because of poor results, that's enough to skew things very, very badly.

The basic idea is that you should only use drugs or drug combinations for which there is evidence that they work and are not harmful. If there's nothing published, don't use it!

but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

There's nothing "fraudulent" about it. Studies often fail for many reasons completely unrelated to the drugs themselves.

Scientific experiments are usually one-sided: a positive result tells you something, a negative result tells you nothing.

Re:you're dangerously wrong (1)

vux984 (928602) | more than 4 years ago | (#25195615)

Scientific experiments are usually one-sided: a positive result tells you something, a negative result tells you nothing.

That is soooooo wrong it hurts.

frist plost? (-1, Redundant)

themusicgod1 (241799) | more than 5 years ago | (#25180455)

quiet day?

Re:frist plost? (0, Offtopic)

Tubal-Cain (1289912) | more than 5 years ago | (#25180583)

Funny. On a "quiet day", a FP attempt gets fourth.

Re:frist plost? (0)

Anonymous Coward | more than 5 years ago | (#25180605)

Funny. On a "quiet day", a FP attempt gets fourth.

Ooh, I just knew an insightful comment would sprout from the seeds of trollery and redundancy hitherto planted!

Negative results (3, Funny)

Anne_Nonymous (313852) | more than 5 years ago | (#25180483)

This sounds reasonable. From now on I'll be emailing my pals every time I don't get laid.

Re:Negative results (4, Funny)

eln (21727) | more than 5 years ago | (#25180661)

It would be a lot more efficient if you just had a cron job that sent out an email every night.

Re:Negative results (5, Informative)

philspear (1142299) | more than 5 years ago | (#25181417)

Whoever modded that down as overrated clearly didn't know enough to judge. You don't publish negative results because of the time and effort it takes to write it up.

The reasons vary for different fields. A lot of times, researchers don't bother doing proper controls until after the experiment works or doesn't. If the trial run works, you do the controls afterward to verify your result was real, if you don't get the result you expect you might try it again, doing some troubleshooting, but at some point you have to make a choice between a control that would be particularly onerous or expensive, or giving up on the experiment entirely. If you get a negative result that you're not interested in, you generally don't do the controls to prove to others it was a valid result because you don't care and have better things to do. But that's what you would have to do to publish it.

A negative but true result can also be even more difficult to prove than a positive result would be. If you are expecting one protein to interact with another one, and you get no result, it could be that they are and your test just isn't working. If you do the experiment a different way and still show no interaction that makes it a little more convincing, but doesn't prove that both systems are working. You can't say for sure they don't interact in real cells.

In clinical trials you could think of additional reasons why someone would not care to publish the negative results. The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working. They could spend 5 years refining it before it actually works, then more years before it gets to market, and they only have a few years before the patent runs out. If they don't patent it and aren't sure it's a complete dead-end, another company might take the results and make a working drug from it, effectively stealing the expensive work to get up to that point.

Not a lawyer or an expert on the pharmecutical industry obviously, but publishing means making it known, and they're only going to do that if they're sure they're done with it, if then.

It makes sense that they're not going to be published, and while it's less than ideal, I think it would be worse to force the pharmecuticals to publish negative results of trials. If you make the clinical trial phase that risky, companies would be more reluctant to develop new drugs that haven't already proven effective, and advances in cancer treatment would slow.

Re:Negative results (4, Informative)

j0nb0y (107699) | more than 5 years ago | (#25181457)

The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working.

Except clinical trials are not required to attain patent protection. Any drug company with good patent practitioners is going to have a patent application filed *before* clinical trials. You can always abandon it later if the clinical trials don't work out. It's much cheaper to file and then abandon an application on a drug than to miss the opportunity to get a patent on a successful drug.

Re:Negative results (1)

philspear (1142299) | more than 5 years ago | (#25181501)

Ah. Well then, nevermind. Maybe it's just the innate "Keep secret all that you possibly can," attitude.

Re:Negative results (4, Interesting)

jamesh (87723) | more than 5 years ago | (#25182073)

I have been told by someone who would know (psychiatrist) that drug trials (maybe just here in Australia) have to be announced before the trial begins otherwise the outcomes can't be published. The idea is that you have to announce a trial before you begin, and so if the outcomes are bad and you don't publish, the bad outcome can be inferred from the lack of publication, even if the specifics remain unknown.

Re:Negative results (0)

Anonymous Coward | more than 5 years ago | (#25191411)

You may be correct about the economic model of drug development. Quite convincing. The problem is that drugs and medicine isn't just a private (money making) problem to solve. They even go around under the banner of "Science."

Science requires that knowledge and results be shared publically. Without public vetting, then how is what they are doing anything more than just private marketing preparations... magic... smoke and mirrors. If we aren't collectively able to address the problem of health and only value (pay) for it when we do it secretly and guard the trade then we are throwing out the advantages of Science itself!

A one sentence (4, Funny)

DanZ23 (901353) | more than 5 years ago | (#25180487)

Summary? You're practically forcing me to read TFA

More complaining and second-guessing (2, Insightful)

Kohath (38547) | more than 5 years ago | (#25180527)

Cancer patients are already vulnerable to cancer.

Not sure what the point of this story is. Sometimes things don't work out the way everyone wishes they would. Apparently every decision to say something or not say something always has to be second-guessed by third parties who have no responsibility or accountability -- but they get to demand things anyway.

I'm sure a lot more of these failed trials would be published if there was a financial incentive. The complainers should start a foundation and start paying the people who have better things to do than to write papers and publish info that's of no use to them. They should do that instead of complaining.

Re:More complaining and second-guessing (1)

Free the Cowards (1280296) | more than 5 years ago | (#25180559)

Do you realize how hilarious it is for you to complain about a study for complaining about studies? If you think it's wrong for third parties to second-guess studies and demand things with no responsibility or accountability, maybe you should stop doing it yourself.

Re:More complaining and second-guessing (1)

Kohath (38547) | more than 5 years ago | (#25180847)

I didn't demand anything. I suggested that the people demanding things start paying for the work they are demanding others do. Or they could just start minding their own business and stop demanding something for nothing.

If they're going to complain, why shouldn't they should expect complaints in return?

Re:More complaining and second-guessing (1, Flamebait)

Free the Cowards (1280296) | more than 5 years ago | (#25180885)

There is no difference between a suggestion and a demand when the person making it has no power over the target.

It's not about expecting complaints. It's about you saying that these complaints are bad, while doing the exact same thing yourself. What makes their complaints bad but yours legitimate?

Re:More complaining and second-guessing (1)

somersault (912633) | more than 5 years ago | (#25181057)

There is no difference between a suggestion and a demand when the person making it has no power over the target.

Yes there is. There may be no difference in the results, but there is a difference in the method.

As for your second point (and your sig), quit trolling with weak pseudo-logic. He wasn't complaining that they were complaining: he was complaining at where their complaints were directed. You're allowed to complain about complainers when the original complaints are moronic!

Re:More complaining and second-guessing (1)

Free the Cowards (1280296) | more than 5 years ago | (#25181159)

He's complaint was about studies complaining about other studies. He was also complaining about a study. I don't see any difference between the two other than that he has a whole lot less funding and, I'm guessing here, expertise. Otherwise they look the same to me: both of them are people telling other people that their efforts would be better applied in a different way.

Re:More complaining and second-guessing (1)

konohitowa (220547) | more than 5 years ago | (#25181369)

He's complaint was about studies complaining about other studies. He was also complaining about a study.
--
Fuck y'all.

Now we see the fallacy of your argument. Go back and read what he said again. He wasn't complaining about the study. He was complaining about the story and the author's complaining based upon their study. Possibly might be also referring to people commenting here complaining, but then he never said that specifically.

Oh - and - a heartfelt "fuck you" back at ya.

Re:More complaining and second-guessing (5, Insightful)

BabyDave (575083) | more than 5 years ago | (#25180671)

The point of the story is that companies publish the successful trials on a drug, but don't publish failed trials on that same drug - i.e. they cherry-pick the results.

Re:More complaining and second-guessing (5, Insightful)

nedlohs (1335013) | more than 5 years ago | (#25180889)

It's not just cherry picking - which wouldn't matter so much in itself. Drugs are very hit and miss you expect lots of things to just not work.

The problem is they study drugs X, Y, and Z in combination and find that not only does it not help it makes the patients worse. They don't bother publishing since they didn't get anything useful out of it and no one is going to cite them...

A year later another group decides to study X, Y, and Z in combination. So a pointless study that harms patients is done because this second group never found out about the previous study in their literature search.

Re:More complaining and second-guessing (1, Interesting)

Anonymous Coward | more than 5 years ago | (#25182197)

From the article... there is a registry of all of these trials (from the US, at least). Any group that decides to run a trial is going to check this registry first. If they find that a trial was registered, but no paper was ever released they would probably: a) assume the trial was not a success or b) contact those involved with the trial and ask for more details.

Medical journals are as much for practicing MDs as they are for researchers. Flooding them with failed trials would only dilute the valuable material that MDs rely on to better treat their patients.

I agree it would be nice to have a global registry like this, or even a set of journals dedicated entirely to failed trials.

Re:More complaining and second-guessing (0)

Anonymous Coward | more than 5 years ago | (#25183819)

It's not just cherry picking - which wouldn't matter so much in itself. Drugs are very hit and miss you expect lots of things to just not work.

The problem is they study drugs X, Y, and Z in combination and find that not only does it not help it makes the patients worse. They don't bother publishing since they didn't get anything useful out of it and no one is going to cite them...

A year later another group decides to study X, Y, and Z in combination. So a pointless study that harms patients is done because this second group never found out about the previous study in their literature search.

The only reason a company would test drugs X, Y, and Z in combination is if that company owns one of those three drugs (that's why they're doing a clinical trial). The other two drugs are already on the market and it is already known that they work both alone and in combination. That has already been published. The result that X does not work well together with Y and Z is of no use to anyone else since NOBODY else can actually use drug X until the patent runs out in 10-20 years.

So who is losing out here? Any real company (not a couple academics) will do their patent search before spending years and millions researching a drug that somebody else has already tried and dropped.

By the way, the existence of the clinical trial is known to the public. The FDA gets all of the results of all studies. As a company, you can see that Big Pharma X started trials then abandoned them. Connecting the dots isn't too hard. (especially when half of their consultants are on your payroll as well... that's just how it works. Not as secret as you might think)

 

Re:More complaining and second-guessing (3, Interesting)

Davak (526912) | more than 5 years ago | (#25181439)

As somebody who was previously in the academic medical field, this is not just a problem with cancer drugs. "Positive Outcome Bias" or "publish bias" is a huge problem.

http://www.ama-assn.org/public/peer/7_15_98/jpv71042.htm [ama-assn.org]

A negative study should be just as important as a positive study. If done well... obviously.

Published negative studies dissuade doctors from using certain offlabel treatments. Published negative studies prevent other docs from wasting time and money to discover the same results.

More importantly, many clinical changes are based on meta-studies... which as basically studies which combine all the available data. If negative studies are not published, it throws off these metastudies... and thus bad care occurs.

Re:More complaining and second-guessing (1)

poopdeville (841677) | more than 5 years ago | (#25181741)

More importantly, many clinical changes are based on meta-studies... which as basically studies which combine all the available data. If negative studies are not published, it throws off these metastudies... and thus bad care occurs.

How does the positive publication bias "throw off" metastudies? Please, feel free to use statistical terms. I do not see how this is possible off-hand unless the metastudies use seriously flawed methodology.

Presumably, they aggregate "like-typed" (in the sense that they have "similar" independent variables) data from published studies. Unless the type system is fairly strict, you will end up with nonsense. But unless a negative result directly contradicts a "known" positive result (in which case it would probably be published), the kind of data you can get out of a negative result is obviously going to have a different type than from a positive result. I don't see that they can be mixed.

Re:More complaining and second-guessing (1)

lostraven (928812) | more than 5 years ago | (#25183399)

"Published negative studies dissuade doctors from using certain offlabel treatments. Published negative studies prevent other docs from wasting time and money to discover the same results."

This is my gut reaction to reading TFA. I understand that in a sense, this isn't new news. Some of the commenters have talked about the time and cost associated with organizing and publishing negative/unintended results. However, from a scientific point of view, this seems like the single most important reason to publish that data; to allow other scientists to both learn from other's mistakes and improve on said mistakes if possible.

Re:More complaining and second-guessing (1)

tburkhol (121842) | more than 5 years ago | (#25183465)

The point of the story is that companies publish the successful trials on a drug, but don't publish failed trials on that same drug - i.e. they cherry-pick the results.

This (almost certainly) has nothing to do with drug companies cherry-picking results. It's extremely difficult to get negative results through peer review, and as a result there are very few studies of any sort that report "X doesn't have an effect" or "Y is not involved in Z."

Part of this is because of the statistical modeling associated with these studies. It's relatively easy to be certain, and to prove, that the observations from two groups are different: this is the type I error [wikipedia.org] everyone loves. We've all agreed that we're willing to accept a 1-5% probability that we're wrong, so once you hit that statistical threshold, you satisfy reviewers.

To prove that two groups are identical is essentially impossible. The only thing you can formally test is whether the difference between two groups is smaller than some pre-defined margin, but there's no established formalism for choosing how big that margin is. Even worse, there's no established criterion for how frequently you're willing to make a type II error [wikipedia.org] . Some folks design their studies with 10% margin, but this number has a huge influence on the required sample size, and folks often use 20% as a desired power. Worse, many studies get started with no formal statistical design at all, and this more-or-less prevents them from making any meaningful statement of "sameness" between groups.

It's a shame, because it almost certainly means that a lot of scientists spend a lot of time reproducing experiments that other people already know aren't going to produce any meaningful differences. Even if there were great statistical models for sameness, it would still be tough to be intuitively confident that lack of differences reflect the actual world and not just crappy assays, and it's likely that we're just stuck with negative results being unpublishable.

Re:More complaining and second-guessing (2, Informative)

Anonymous Coward | more than 5 years ago | (#25180679)

Maybe try reading the article?

FTFA:

"Of particular concern," they argue, is the especially poor showing by industry-sponsored trials, since they tended to probe the value of patented drugs -- "many of which are in clinical use."

I would say that's newsworthy if not all that surprising. Just another case of big drug companies screwing over the general population to make a buck. They run studies to test their drugs and if the results are unfavorable they just bury the study while keeping the drug in circulation.

Re:More complaining and second-guessing (1)

beakerMeep (716990) | more than 5 years ago | (#25181493)

Sounds like you're complaining.

Re:More complaining and second-guessing (1)

Mr. Slippery (47854) | more than 5 years ago | (#25182319)

Cancer patients are already vulnerable to cancer.

So it's better for them to be vulnerable to cancer plus ineffective, expensive treatments, often with nasty side effects, that substitute for more effective ones?

I'm sure a lot more of these failed trials would be published if there was a financial incentive.

These failed trials would be published if the researchers were actually doing science. If they're not, let's call it what it is: product development. And let's remove the fiction that what they are doing is promoting the progress of science and thus deserves patent protection.

Cancer treatment is a product (0, Troll)

burnitdown (1076427) | more than 5 years ago | (#25180575)

The ideal product will lessen the symptoms, but not cure the disease, at least until a 10-year-course of expensive treatment is over.

This is how you fellow humans view you: as a potential walking dollar sign, in exchange for their goods and services.

I hear horse urine cures cancer. I have fresh horse urine for only $9 per ounce. Do you want to be cured, or not?

Re:Cancer treatment is a product (1, Funny)

Anonymous Coward | more than 5 years ago | (#25180745)

I hear horse urine cures cancer. I have fresh horse urine for only $9 per ounce.

Many of the older anti-drinking crowd in these parts, and others, commonly refer to beer as horse's piss so I guess I will just stick with beer. Though its price has risen absurdly in my lifetime, it isn't quite to $9/oz yet.

Re:Cancer treatment is a product (5, Funny)

Anonymous Coward | more than 5 years ago | (#25180765)

I have fresh horse urine for only $9 per ounce. Do you want to be cured, or not?

Nay.

Re:Cancer treatment is a product (2, Insightful)

zippthorne (748122) | more than 5 years ago | (#25180775)

No, they'd much rather have those 10 years worth of payments up front, so they can invest or re-invest it.

You seriously don't think people would pay more for a week-long cure than for a week's worth of "treatment-in-perpetuity?"

Re:Cancer treatment is a product (1)

westlake (615356) | more than 5 years ago | (#25182419)

You seriously don't think people would pay more for a week-long cure than for a week's worth of "treatment-in-perpetuity?"

.

The "cure for cancer" is just the beginning - at that point you can begin to think seriously about directing research towards the anti-aging drugs, the rejuvenation techniques, that have been stuff of epic myth, science fiction and fantasy since the beginnings of human history.

Re:Cancer treatment is a product (1)

wormBait (1358529) | more than 5 years ago | (#25180781)

If you can actually cure cancer, you can charge whatever you want for the treatment, so no one is going to throw out their cure because it isn't profitable. If they found a cure, they would still sell you add ons for years anyway to get money out of you. What usually happens is that they can't create a cure, so the treatments they can come up with only lessen symptoms. And since everything is on a gradient, you only test as far as insurance companies are interested, not to the limit of the drugs.

Re:Cancer treatment is a product (2, Interesting)

wiggles (30088) | more than 5 years ago | (#25180855)

Horse urine can't cure cancer, although it is used in hormone replacement therapy. I suppose some cancer patients driven to early menopause due to radiation and chemotherapy do use it, though...

http://en.wikipedia.org/wiki/Premarin [wikipedia.org]

Re:Cancer treatment is a product (0)

Anonymous Coward | more than 5 years ago | (#25180945)

"This is how you fellow humans view you: as a potential walking dollar sign, in exchange for their goods and services."

Too bad that particularly nasty trait (greed) is built right into our genes. I don't ever see any well-meaning social movement eradicating it from our species.

Is Greed built into our genes? (1)

burnitdown (1076427) | more than 5 years ago | (#25183485)

Too bad that particularly nasty trait (greed) is built right into our genes. I don't ever see any well-meaning social movement eradicating it from our species.

One reason I oppose the doctrine that we're all equal is that I've observed the more intelligent, confident and physically attractive people are less greedy.

There are people out there who, unlike Hollywood stars, look great without makeup, went to good schools and have good careers doing things that (unlike Hollywood) make life better for people.

Then there are the ugly fat guys in Porsches who seem to think that having their own business as a patent sniper makes them "equal" to someone who is smarter, more attractive, and has greater moral character...

Something worth thinking about, at least.

Financial incentive (3, Interesting)

Stickerboy (61554) | more than 5 years ago | (#25180703)

The obvious reason is that it takes time and money to publish study results, neither of which are recouped currently if the study showed negative results.

The obvious fix is to reward pharmaceutical companies financially for publishing all results. Form a subentity within the NIH with the power to purchase study data and results that can be published by the government or a peer-reviewed journal.

Re:Financial incentive (1)

Anonymous Brave Guy (457657) | more than 5 years ago | (#25181243)

The obvious fix is to reward pharmaceutical companies financially for publishing all results.

Or simply compel them by law to publish all study results for any drugs under consideration for active use at any later stage. Their excuse for the enormous advantages they gain through patent protection and the like, and the borderline ethics of a lot of their business practices, is that R&D in a regulated industry is expensive. The cost of publishing failed tests as well would be absolutely negligible if the R&D costs as much overall as they keep telling us.

Re:Financial incentive (1)

shilly (142940) | more than 4 years ago | (#25192337)

You can only publish if you can find a publisher. The most obvious solution is to expand the role of the (inter)national trial register to be a place to go to find out results as well.

Re:Financial incentive (0)

Anonymous Coward | more than 5 years ago | (#25182227)

Why give them even more money? The markup on certain drugs is already so high, we should penalize the drug co's for not publishing results

Re:Financial incentive (1)

AngelofDeath-02 (550129) | more than 5 years ago | (#25182907)

I know - anonymous coward and all..

But it's not cheap to develop effective drugs. The cost to produce them (effort vs gains) and the limited timespan they have to recoup said costs is a large reason they are so expensive.

What might work is a repository of failed experiments that isn't too stringent on the peer reviewing or reproduction. This way you get some history as to what was done (and didn't work) without the cost associated with PROVING that it did work.

If at any time in the future these results would be called into question - the person doing the research can verify the results or not, but they'll at least know it was done and can assess the potential gains.

Well, enough armchair speculation. I'm not qualified to given an opinion of consequence anyway.

Re:Financial incentive (1)

KlausBreuer (105581) | more than 4 years ago | (#25191895)

> reward pharmaceutical companies financially for publishing all results

What a brilliant idea. If that actually occurs, I'll immediately buy large amounts of stock in these companies. And then read their reports:

"Report on whale pee on cancer: unsuccessful"
"Report on heavy wishing on cancer: unsuccessful"
"Report on digital watches on cancer: unsuccessful"

The three reports above will already pull in a couple of ten thousand bucks...

Cargo Cult Science (4, Insightful)

McGregorMortis (536146) | more than 5 years ago | (#25180709)

This very subject was addressed, very eloquently as usual, by Richard Feynman in his famous Cargo Cult Science [lhup.edu] lecture.

Re:Cargo Cult Science (1)

blankinthefill (665181) | more than 5 years ago | (#25182673)

I love Feynman. Truly one of the greatest minds of the century, and he wasn't just smart about physics. His insights in any of his published works are worth reading: Not only does he have a great sense of humor, but he's interested in EVERYTHING, and he's able to write in ways that laymen can easily understand, while not overstating his own knowledge of the subject at hand.

Nothing new (4, Insightful)

edcheevy (1160545) | more than 5 years ago | (#25180755)

It's called the "file drawer problem" and impacts every field of science. If you don't find significant results, you don't get published, and you stick your "failed" study in the file drawer. As a result, "failed" studies on ANY topic usually get swept away. It's unfortunate, but there's nothing particular sinister about it (as the article seems to imply). There's just no incentive to publish the trials and studies that didn't work.

Re:Nothing new (4, Informative)

ceoyoyo (59147) | more than 5 years ago | (#25180921)

The pharmaceutical industry HAS been caught doing some sinister things with their studies. They do ten studies and publish the three that can be interpreted positively. The seven that can't never see the light of day. Unless of course they accidentally get published on the company web page. Then the sinister part: the drug is approved and sold to treat the condition. The clinical trials registry was formed to try to clean up that particular mess.

This particular story doesn't seem to describe anything sinister. The trials were all in the registry (that's where the authors got the data for the study). Sure, the journals tended not to publish the negative results, but they were all available in the registry.

As far as I can tell from the article they didn't even look at the actual worrisome situation where there are a few published positive studies and many more unpublished negative studies, for the same treatment.

Re:Nothing new (1)

edcheevy (1160545) | more than 5 years ago | (#25181547)

My impression from the article was that nobody checks this registry (aka file drawer). I didn't mean to give pharma a free pass -- the article suggests they have a greater rate of "file drawer" than non-industry research.

The article highlighted the problem perfectly -- only 20% of the studies in the registry are reaching publication. Since publication is where many people (doctors) learn about what's new, 80% of the data is never seen.

They are trying to do something about it, such as require results to be shared from any trial using government funding, but the problem persists if the results end up in a registry that nobody reads. Or the "Journal of Studies that found Nothing New", where each volume is 4 times as dense as any competing journal. Meta-analysis may be our savior but there needs to be new research generated so we can meta-analyze it -- researchers can't spend all their time looking at prior studies.

Now if some techie could put together software that does the meta-analysis for us (giving averages across, say, all studies of Treatment X for Lung Cancer on a query) it would be a pretty epic breakthrough. But the variables would have to be entered along with the studies, and peer reviewed for accuracy (study #10285 used inappropriate statistical analysis and was not as rigorous as the authors claimed; study #26581 had superb experimental control but only examined 20 year olds; etc etc). Actually, moderated & meta-moderated is probably a more appropriate term than peer reviewed given where we're posting! Hopefully scientists across all fields move towards that, where it's second nature to publish the results of your study in a quantifiable manner no matter the results, but I think it's a while off.

So get on it, somebody... my programming days are long over. :p Heck, get government funding from NSF and make a buck doing it!

Re:Nothing new (1)

ceoyoyo (59147) | more than 5 years ago | (#25185255)

If that's happening (which the study didn't seem to address in any way), then it's the physician's fault, not the researchers. All the data were published in the appropriate places. You can't ask the mainstream journals to publish everything, or they will become "The Journal of Studies That Found Nothing New." I would certainly hope that my physician would check for published negative studies before using some off-label experimental treatment on me.

Realistically, most physicians hear about treatments they're actually going to prescribe from drug companies or colleagues (or both), either through advertising or conferences. There certainly should be a law (if there isn't) that drug company literature has to include ALL data.

I disagree that meta-analyses should be automated. If you've ever done one, or read one that was properly done you'll know that meta-analysis are a complex research project in their own right. Study quality and methodology varies and most studies look at slightly different things. Combining them requires all sorts of subtle decisions. It would be very difficult to standardize everything so it was directly comparable, and almost certainly not practical.

Re:Nothing new (1)

edcheevy (1160545) | more than 5 years ago | (#25185585)

I thought it was pretty clear they don't consider the registry to be effectively disseminating results: "Data from fewer than one in five research trials are ever published. Findings from the vast majority of human trials become buried for reasons that may never come to light". If blame is to be placed, I agree it's always on the reader -- researchers don't bother publishing failed studies because nobody wants to publish them. They don't sell subscriptions. But that hardly means we can expect doctors to look at everything in the registry, or contact all of the researchers in that field and ask about any pilot studies they've done recently that didn't get published.

A meta-analysis is currently very hard because whoever is doing the meta-analysis has to assign a value for each of those things you mentioned and do it across every single study they examine. If we had a system where we did that as we went, rather than in a giant intimidating chunk (meta-analysis), we would be able to do a sort of meta-analysis on the fly. And maybe the registry in the article is trying to do that (I'm not familiar with it) but I don't think so. If everything that went into the registry was peer reviewed upon entry and values were assigned to those variables, there's no real reason it couldn't work. A strong meta-analysis shouldn't have too many subtle decisions involved. It should start out with clear delineations that multiple raters can agree on. There's no reason we couldn't set guidelines and apply the same logic of crowd wisdom to this that we applied to scanning images for Steve Fossett [iht.com] . There would always be some gray areas in either approach, but the presence of subtle decisions would seem to bolster the case for meta-as-we-go. Difficult yes, impossible no. :)

Re:Nothing new (3, Informative)

ceoyoyo (59147) | more than 5 years ago | (#25185869)

"Published" in a major journal. ALL of the studies were published in the registry. As I said, you can't ask the major journals to publish every failed study.

If you want to try and standardize trials, go for it. I wish you luck. Speaking as someone who actually does them, it won't happen. There are all sorts of reasons why trials are difficult to compare. Different centres, different technology, different standards of care. Ethics committees in different places demand different things. Not to mention you rarely have a situation where many trials are testing exactly the same thing. Most meta-analyses look at trials that are designed to look at slightly different things and then try to compensate for the differences in study design.

Re:Nothing new (1)

oDDmON oUT (231200) | more than 4 years ago | (#25192415)

"If that's happening (which the study didn't seem to address in any way), then it's the physician's fault"

Oh reeeeeeally?

"I would certainly hope that my physician would check for published negative studies before using some off-label experimental treatment on me."

Have you spent any time in a doctors office? I mean from the backend? Examine how the physicians time (let alone that of their staff) is allocated.

60% or more is tied up in administrative tasks that:
            C.T.A.s from lawsuits
            Are for compliance with a local/state/federal mandate or regulation
            Fulfill byzantine insurer or governmental stipulations so that they can paid for doing their job
            Ensure that all of the above t's are crossed and i's are dotted

The remaining 40% is allocated to patients and patient tasks, but is probably subject capitation, or other "cost effectiveness" measures.

Which leads us to the drug company reps, those wonderful marketing guys/gals who weasel their way in in the small slices of time in between all the rest to tell our doctors all about the latest and greatest wonder compound, leave samples of the same, touting how much better life will be for both them and their patients if only they prescribe it on a regular basis.

And you "hope" those health care professionals will make time to read negative study results over listening to a live pretty person who shows up on their door promising positive results.

/laughs

From TFA (1)

cvd6262 (180823) | more than 5 years ago | (#25181075)

The reason: Data from fewer than one in five research trials are ever published. Findings from the vast majority of human trials become buried for reasons that may never come to light, according to a new study in The Oncologist. It's published early and online September 24.

It's called publication bias and it exists in most fields. Publisher want to break significant results, so even if your research was extremely well designed, and the fact that you found negative results could be extremely important, they won't publish it.

Re:From TFA (1)

fuzzyfuzzyfungus (1223518) | more than 5 years ago | (#25181175)

There are actually a few journals that specialize in publishing negative results in various disciplines, founded in an attempt to solve that problem. Obviously it does nothing to deal with malicious burying; but having a place were researchers can get better than zero credit for depositing suitably rigorous records of unexciting results should at least help with the nonmalicious structural bias.

Amazing (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#25181173)

after all of this richard stallman is a still a big fat greasy turd. will someone please buy that guy a bar of soap and some shampoo? seriously.

Medical Profession and Institutions medieval (1)

syousef (465911) | more than 5 years ago | (#25181567)

I'm not at all surprised that findings don't get published. The only way to force them to be published would be to make it a condition of doing the study or a legal requirement. The pharm companies couldn't care less that patients die - they prefer the competition wastes time repeating the same tests rather than taking advantage of the work they've already done.

The entire way that the medical system works really needs a shake up. Other industries with less at stake have more stringent quality control processes. The medical profession has medical boards that amount to little more than exclusive clubs. The amount of shit I've heard about in the last few years would be beyond belief had I not experienced some of the incredible incompetence I've seen first hand that have done harm to me and mine. At this point I no longer consider it safe to take medical advice without reviewing it and researching it thoroughly using every corner of the internet at my disposal - from Google to Pubmed. Sure, if it's life and death and requires immediate attention, you have no alternative but to take what expert care a doctor can provide then and there without questioning it. However for anything where you have more than a few hours to make a decision trusting a doctor - particularly a specialist, and particularly a doctor you don't know well - is quite like playing Russian roulette.

I'm sure there will be a few in the medical profession who read this and think "Not another one who self diagnoses". To them a big "fuck you". When I can get doctors that can check the fucking contraindications on a medication before repeatedly upping the dosage (and almost killing someone dear to me) or doctors who aren't quick to try to scare me into doing unnecessary surgery assuring me I'd be crippled long ago if I didn't get it done, then I'll think about trusting my doctor to get it right. As it is I've even had a GP fuck up something as simple as removing ear wax.

This is a general problem with science (1)

tanke (1165397) | more than 5 years ago | (#25182009)

In the science world most unsuccessful experiments go unreported which means that millions of dollars are spent by different research groups repeating the same failed experiments/techniques. The main problem is that nobody wants to report failures because there is no incentive to do so.

Well duh (0)

Anonymous Coward | more than 5 years ago | (#25182133)

What doctor would still receive funding for trials if she/he has previously reported bad results? Same is true in academia. How many journals print articles on methods that didn't pan out? I haven't read any. Results are always cherry picked to show how someone succeeds rather than fails. You keep the failed methods to yourself and maybe share them with students, but never funding sources, that would be career suicide

This is "dark data" (1, Interesting)

Anonymous Coward | more than 5 years ago | (#25182237)

Some scientists are interested on getting all data published, even if it didn't result in big news. They call this the "dark data".

http://www.wired.com/science/discoveries/magazine/15-10/st_essay

I'm publishing the results of a failed cancer tria (1)

plasmacutter (901737) | more than 5 years ago | (#25182287)

My mother went through a 6 week series of trials of epsom salts against colo-rectal cancer.

The mixture was ineffective.

My, how very useful this information is for cancer patients!
Would my mother have received the quality of the care she eventually did get if her doctors had missed relevant articles in medical journals thanks to the massive signal to noise ratio?

If every failure were published, the cancer research community would suffer the same "eternal september" the usenet community did.

Re:I'm publishing the results of a failed cancer t (3, Insightful)

adamofgreyskull (640712) | more than 5 years ago | (#25183721)

My mother went through a 6 week series of trials of epsom salts against colo-rectal cancer.

The mixture was ineffective.

My, how very useful this information is for cancer patients!

Yes, because that is exactly the same thing. *sigh*

Would my mother have received the quality of the care she eventually did get if her doctors had missed relevant articles in medical journals thanks to the massive signal to noise ratio?

If every failure were published, the cancer research community would suffer the same "eternal september" the usenet community did.

What if the researchers developing new drugs and treatments had access to the failures of others so that they knew what *not* to try. Outside of your pathetically childish and facetious example about Epsom salts, this information could be invaluable. Would you have wanted your mother to die because scientists working for Pfizer didn't tell the community about a failed treatment that they had already tried which GlaxoSmithkline then spent 2 years replicating, at the expense of another possibly more fruitful avenue of research?

an argument for open source research (1)

BroadbandBradley (237267) | more than 5 years ago | (#25184065)

I wouldn't have been able to see so far had I not been standing on the shoulders of giants applies to stuff that doesn't work as well. Some things, Like medicine, are too important not to have companies and researchers work together on. This is just an example of how legislature is created by money. The Golden rule: those who have all the gold, make all the rules!
 

criminal (0)

Anonymous Coward | more than 5 years ago | (#25184169)

"Who conducted a trial seemed to influence the likelihood its data would see the light of day. Studies sponsored by industry (such as drug companies) had the lowest publication rate: 5.9 percent. By comparison, data from 59 percent of studies performed by clinical-trial networks were published."

This is bloody criminal. People participate in these studies to help support the good. They take the risk of testing dangerous medicines to help others, and results are hidden? Criminal is what it is.

only good results please (0)

Anonymous Coward | more than 5 years ago | (#25189717)

Disappointing ANY results in any scientific study often go unreported. most major journals like to accept only good results that lead to probable applications or breakthroughs in science

A lot of factors to consider (0)

Anonymous Coward | more than 4 years ago | (#25196379)

Not all trials will get published. Keep in mind that phase I trials aren't designed to test efficiency of the drug to treat a particular cancer. I believe phase I trials try to find the maximum tolerated dose. Sometimes you get surprises in phase I, but not usually. The info gathered is of more interest to the drug manufacturer than anyone else so not much to publish. Most times the trials are sponsored by large pharmacuticals and they won't publish any data until they go for FDA approval. It's their drug, they're paying the bills so it's they're data to do with as they please.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>