Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Scientists Themselves Play Large Role In Bad Reporting

samzenpus posted about 2 years ago | from the but-I-have-people-skills dept.

Medicine 114

Hugh Pickens writes "A lot of science reporting is sensationalized nonsense, but are journalists, as a whole, really that bad at their jobs? Christie Wilcox reports that a team of French scientists have examined the language used in press releases for medical studies and found it was the scientists and their press offices that were largely to blame. As expected, they found that the media's portrayal of results was often sensationalistic. More than half of the news items they examined contained spin. But, while the researchers found a lot of over-reporting, they concluded that most of it was 'probably related to the presence of ''spin'' in conclusions of the scientific article's abstract.' It turns out that 47% of the press releases contained spin. Even more importantly, of the studies they examined, 40% of the study abstracts or conclusions did, too. When the study itself didn't contain spin to begin with, only 17% of the news items were sensationalistic, and of those, 3/4 got their hype from the press release. 'In the journal articles themselves, they found that authors spun their own results a variety of ways,' writes Wilcox. 'Most didn't acknowledge that their results were not significant or chose to focus on smaller, significant findings instead of overall non-significant ones in their abstracts and conclusions, though some contained outright inappropriate interpretations of their data.'"

cancel ×

114 comments

Spin, spin spin spin (-1)

I'm Spinning Around (2729457) | about 2 years ago | (#41322145)

Look at me, I'm spinning around!

what about cheese (1)

For a Free Internet (1594621) | about 2 years ago | (#41322161)

I think dairy products are terrorism and should get the death penalty for them and we should use nukes against cheese, for FREEDOM!!

Well, of course my abstract contained spin! (5, Funny)

RevWaldo (1186281) | about 2 years ago | (#41322189)

The article was on quantum mechanics fer chrissakes!

.

Re:Well, of course my abstract contained spin! (1)

Livius (318358) | about 2 years ago | (#41322247)

Let's face it, even the scientists don't know what quantum spin is all about.

Re:Well, of course my abstract contained spin! (0)

Anonymous Coward | about 2 years ago | (#41322341)

Yes we do, thanks. It has no classical analogue, so it's hard to explain to anyone who hasn't studied quantum mechanics, but spin is very well understood at this point.

Just because YOU don't get it doesn't mean that we don't. How long have you spent trying?

Re:Well, of course my abstract contained spin! (1)

TapeCutter (624760) | about 2 years ago | (#41324257)

The ones who know maths do, but they can't describe it to the ones who don't in any other way.

Re:Well, of course my abstract contained spin! (1)

Bowling Moses (591924) | about 2 years ago | (#41325621)

"Let's face it, even the scientists don't know what quantum spin is all about."

MRI scans [wikipedia.org] couldn't exist without a thorough understanding of what quantum spin states are. Ditto for NMR spectroscopy [wikipedia.org] .

It's only Natural (3, Insightful)

happy_place (632005) | about 2 years ago | (#41322351)

There are a number of reasons scientists spin their work.

1. Science is quite boring. By nature it's supposed to be, objective, logical, and devoid of feelings. But Scientists themselves are not typically boring people, they're humans, and humans are emotional beings.

2. Scientists aren't communications experts and suck at making dry discipline accessible to the public. Never was this more obvious than when I was in college. How many brilliant researchers really sucked at teaching? Pretty much most of them.

3. Scientists want to think their work matters, and therefore are inclined to extrapolate applications of their science to the public. When those applications get reported as a sure thing, then an exaggeration is bound to happen.

4. And of course, Science that can be show to be of great public benefit gets funding. Cha-ching!

Re:It's only Natural (4, Insightful)

jhoegl (638955) | about 2 years ago | (#41322423)

3 and 4 are the main reasons, 1 is subjective and 2 is outright wrong. If 2 were correct they wouldnt know how to spin things in such a way as to hide the results the way they do.
3 and 4 are the most concerning, as that is what peer review is for and that is where there is failure due to the large volumes of data vs time.
So, it is abused by those that just want money to do stupid things.

Re:It's only Natural (-1, Troll)

scamper_22 (1073470) | about 2 years ago | (#41324453)

It's not just that they want money to do stupid things.

Many scientists come with large political biases or want to change the world in some way.

Having a conclusion/abstract that 'helps' that cause is often something they are looking for...

Scientists are just not the truth seekers of society today. They are empowered in giant institutions and their 'approval and allegiance' sought by politicians and pundits to push various agendas.

Not unlike countries and history where religious councils matter.

Re:It's only Natural (1)

Anonymous Coward | about 2 years ago | (#41324867)

I suspect that peer review actually encourages exaggeration of importance. After all, the reviewers are in the same field as the study author, and therefore inclined to believe that the niche is larger than it is. In my experience, reviewers have a much more positive response to a study that claims to have profound implications for [x] than to a study that reports findings and interprets them in the context of prior work. Reviewers are also under funding pressure, and the more important their niche is, the more likely funding agencies are to pay attention.

Re:It's only Natural (1)

Anonymous Coward | about 2 years ago | (#41325837)

While 4 is a problem, you have to consider the audience the article was written for. If the article is a scientific journal, which is what most scientist are writing articles for, then the spin is readily and easily recognized by other researchers as a kind of forecasting where this research might lead, rather than solid statements about the current work. In this case it may appear to be spin, but the intended audience knows what it is. On the other hand, if the article is one for the general public, then the scientist and any associated press offices have a responsibility to make it clear what is the current status of the work and what is merely optimistic forecasting.

Re:It's only Natural (1)

interkin3tic (1469267) | about 2 years ago | (#41325841)

And weighing against the will to cheat is the fact that most scientists are honest people who want to advance our knowledge. You generally don't become a scientist if you are just out to make money by any unethical means possible. If you're okay with lying in order to get fame and fortune, you are probably a lawyer, politician, salesperson, or executive. You might start off honest and then change, of course.

There's also the fact that few scientists are in a position to lie about their results and not have a colleague, collaborator, or someone else notice.

Re:It's only Natural (3, Insightful)

rtaylor (70602) | about 2 years ago | (#41322435)

5. It's possible that scientists which include spin and get good news coverage receive additional funding the next year. Those who don't may not, and eventually end up an assistant to someone who does spin.

No idea if the above is true but if our carrot/stick system is setup this way but if it is then spin is guaranteed.

Re:It's only Natural (1)

TheRaven64 (641858) | about 2 years ago | (#41324145)

They're talking about paper abstracts. These are (usually) the first thing that the reviewer reads and set their frame of mind for the rest of the paper. A good (meaning interesting, not necessarily accurate) abstract means a higher probability of the paper being accepted. It also means that it is more likely to be cited when people are thinking 'I need to cite a paper about this, but I'm not going to reread them all to work out which one makes the most sense here'. Both having papers accepted and having them cited help academics a lot.

Re:It's only Natural (1)

joe_frisch (1366229) | about 2 years ago | (#41324215)

There is a lot of good science that sounds boring to the public, and more importantly to the funding agencies, so the investigators try to make it sound more exciting.

An interesting physics question about how a wave function collapses when a measurement is made becomes "quantum teleportation". Using X-ray pulses to saturated a L-shell (M?) transition in Aluminum becomes "transparent aluminum". A new technique for measuring fast chemical reactions on a surface becomes a 'breakthrough for hydrogen power". The list goes on and on.

This is of course very common in industry. A new "breakthrough microprocessor that introduces a new paradigm in computing", is the previous generation processor done in a smaller process.

There are a huge number of people working in research, and their collective effort has made the world a completely different place. Since there are so many projects, the impact of any one project is generally so small that it seems uninteresting.
 

Re:It's only Natural (0)

Anonymous Coward | about 2 years ago | (#41322649)

Yeah your right for #4. We sometimes have to make far stretched connections to global/popular problematic to get funding. That's how it works Publish Or Die! Academics are under a lot of pressure to get those papers out there, and those most often that not have to be about what politicians care at the moment.

Re:It's only Natural (2)

crazyjj (2598719) | about 2 years ago | (#41322963)

4. And of course, Science that can be show to be of great public benefit gets funding. Cha-ching!

I would say number 4 should be at the top of the list--in 30pt. font and flashing bright red.

Science is supposed to be objective, above such matters as grant-whoring and self-promotion. But if such a creature actually exists, I've never met it myself.

Re:It's only Natural (0)

Anonymous Coward | about 2 years ago | (#41323833)

In my experience having worked in a couple physics fields, I don't think it should be in large, flashing text. I am not saying it isn't an issue, or should not be on the list, but I find it a bit disconnected from reality how much people try to emphasize stuff like #4 on the list over some of the other issues, including others not listed (e.g. trying to connect research to broader fields to get published in a broader topic journal). Too many people I bump into are spring-loaded to jump to the conclusion that some aspect of any science story is false or wrong, because they think the only thing a scientist does is try to get grant more grant money to no end.

Re:It's only Natural (1)

poity (465672) | about 2 years ago | (#41324019)

Take evolution for example. I have rarely read or heard of scientists describing evolution in the most mundane but factually correct way -- the genetic change within a species resulting from natural selection*, a process that is merely the dying off of lineages that could not cope with the environmental conditions, or could not compete with other lineages -- except in textbooks. I do, however, remember guest scientists on documentaries and nature magazines wax poetically about a species' epic struggle of survival in a sea of hostility.

It's like they thought people would be too dumb to understand, so they preemptively dumbed it down, and now we wonder why people are still dumb. It's because YOU -- the only hope they had of being educated -- gave up.

*Notice also how experts do like to make up grandiose names that illicit the image of a supreme intelligence, like "natural SELECTION" and "the INVISIBLE HAND". That's not helping, either.

Re:Well, of course my abstract contained spin! (1)

hweimer (709734) | about 2 years ago | (#41322371)

That's nothing, my most recent paper has spin right in the title [doi.org] !

Re:Well, of course my abstract contained spin! (1)

mcgrew (92797) | about 2 years ago | (#41323689)

I found that short abstract interesting, but reading it I think I know one reason why science reporting sucks -- most reporters can't read at that level. Hell, most reporters would have trouble with the average Wikipedia post about any facet of science, let alone a PhD level paper.

A solution might be ignoring abstracts (1)

Karmashock (2415832) | about 2 years ago | (#41322191)

Just make it standard for science reporters or editors for the science section to ignore the abstract entirely.

Re:A solution might be ignoring abstracts (1)

EvolutionInAction (2623513) | about 2 years ago | (#41322439)

But then they would have to read the paper itself! Good idea.

Re:A solution might be ignoring abstracts (0)

Anonymous Coward | about 2 years ago | (#41322577)

Journalists are supposed to know better than this. One of the tennates of journalism is to not be somebody's mouthpiece. Granted it's not always easy, but this is science, it's not like there's a shortage of scientists to ask for opinions on this stuff. What's more, if the journalists would understand what a preliminary study is, or peer review for that matter, it would cut down on this problem drastically. And taking statistics 101, would probably cut down on it further without having to really do much more work.

Journalists are supposed to be used to having people try to use them for their advantage or their view point, I'm not sure why science is magical in that respect.

Re:A solution might be ignoring abstracts (0)

Anonymous Coward | about 2 years ago | (#41322773)

It's not so much the article abstract as the press release which is generally written by the university's or company's media relations or marketing departments, not the original article authors.

Re:A solution might be ignoring abstracts (1)

oh_my_080980980 (773867) | about 2 years ago | (#41322853)

Read much? It's the press release not the abstract genius.

And the problem is the reporter not the scientist. I have read countless news article headings that had little to do with what the article reported, and then even less that what the actual research paper stated. News is about getting eye balls, either through subscriptions for papers or ratings for news shows.

Researchers do not need to gin up their research to gain funding. The agencies that provide funding are run by actual researchers that understand the research not your average moron.

Re:A solution might be ignoring abstracts (1)

Karmashock (2415832) | about 2 years ago | (#41323671)

First, fu for being a dick... no really... drink molten glass.

Second, researchers don't need to exaggerate? Then why do they... frequently?

This has been an ongoing and system wide issue in the halls of science.

Do journalists exaggerate as well? Oh god yes... more often then not frankly. But it's less acceptable for scientists to do it. And in any case, my only suggestion was that the abstract was ignored and have it be uncitable as a source. So the paper proper can be cited. But the abstract cannot be cited. Do that, and maybe the journalists will be forced to actually read the study in the first place which might lead to more nuanced reporting. We can only hope.

Re:A solution might be ignoring abstracts (1)

Anonymous Coward | about 2 years ago | (#41324089)

The thing I don't get, and maybe it is field specific, is every time I come across some story that seems to defy the laws of physics or includes some blatant claim that makes the work sound like an instant Nobel prize or makes it sound like the work originated what is actually a whole, established field or work, I look up the abstract and see none of those claims are there. This doesn't seem like a memory bias thing for me, as I can't think of any counter examples, although there is probably a bias in which stories I bother to dig out the paper for. I think just reading the abstract instead of repeating what other PR pieces say would go a long ways. Abstracts tend to be pretty short and to the point, there isn't much room for aggrandizing and connecting the research to other fields and implications. Maybe the introduction of the actual paper does that to some degree, although a lot of the time that is just there to make it relevant to the particular journal. Beyond details of the experimental process, the only time I found claims to run counter to the paper are when people on places like Slashdot start complaining about correlation ~= causation or missed factors, when the authors addressed such things explicitly in the discussion at the end.

what about the report in itself (0)

Anonymous Coward | about 2 years ago | (#41322193)

Maybe Christie Wilcox is also doing too much spinning on the subject
Who knows ...

My take on summary (-1)

Anonymous Coward | about 2 years ago | (#41322195)

French scientists spent much of their time reading press releases.

This just in...media reports exciting news. (5, Insightful)

Anonymous Coward | about 2 years ago | (#41322197)

Whereas the mundane gets nothing. For every person murdered, or in a car accident, there are thousands in the area who had a humdrum day. For every house that burns down, thousands don't.

People who hear about these bad things and think the world is going to heck, are forgetting that nobody cares to hear about nothing happening.

Attention whoring for funding (5, Insightful)

Rogerborg (306625) | about 2 years ago | (#41322215)

Fund science like you fund business, and it becomes an exercise in marketing and hot topic buzzwords.

OK, it might take more energy to make a solar panel than we'll ever get back from it, but look at the economies of scale that we're leveraging!

And self promoting .... (1)

Anonymous Coward | about 2 years ago | (#41322257)

Let's face it, to be successful in one's lifetime in any field requires some sort of self-promotion. I'm sure having a well known name makes it a LOT easier to get funding, tenure, book deals, etc ...

Re:Attention whoring for funding (3, Insightful)

Anonymous Coward | about 2 years ago | (#41322269)

As someone who is a Ph.D. student and research assistant, "whoring for funding" is pretty much SOP. It's pathetic and I hate it.

Re:Attention whoring for funding (3, Insightful)

oh_my_080980980 (773867) | about 2 years ago | (#41322895)

Chose another profession. As someone who was a graduate research assistant, we all knew grant writing was part of the job. You want to keep doing research then you need to apply for grants.

principle of least remorse (2)

epine (68316) | about 2 years ago | (#41323175)

Chose another profession. As someone who was a graduate research assistant, we all knew grant writing was part of the job. You want to keep doing research then you need to apply for grants.

What a horrible, defeatist attitude. I can't stand bugs in software (the vast majority exist because low standards are cheap). So I should chose a different profession?

As someone who was a graduate research assistant, we all knew grant writing was part of the job. You want to keep doing research then you need to apply for grants.

I've known since 1978 that "bugs were part of the job" and yet I persist.

Yours is an interesting perspective. The optimal solution to the marriage problem of jobs to talent is the assignment of least remorse: scientists who research on animals should have no feeling for animals, computer programmers should feel no embarrassment over bugs, politicians should enjoy lying, racers in the Tour should be human pincushions, etc.

To some extent, the world does work this way, but it's a strangely sociopathic step to actively endorse this.

Re:principle of least remorse (2)

jahudabudy (714731) | about 2 years ago | (#41324459)

The difference is that bugs are specifically instances where programming breaks-down, doesn't work as intended. Grant writing is a part of how research is designed to work. An academic researcher who doesn't like to write grants is more like a programmer that doesn't like to type - it's not the point of the job, but it IS a necessary task to do the job.

Re:principle of least remorse (1)

Urza9814 (883915) | about 2 years ago | (#41326041)

Replace 'bug' with 'poorly designed software' then.

Just because it's part of the design doesn't mean that design is a good one. You won't ever find a better way of doing things if you don't see anything wrong with the current one.

Re:principle of least remorse (1)

TapeCutter (624760) | about 2 years ago | (#41324513)

I've known since 1978 that "bugs were part of the job" and yet I persist.

I'd go further than that and say bugs ARE the job, when they stop being reported your product is dead, your business/corporate-department probably died earlier than your product. Note that by "bugs" I mean the software/documents don't do/say whatever the guy spending the money wants them to do/say. As I'm sure you know, a professional works on the basis that giving the customer/boss what they want does not necessarily mean giving them what they ask for, so right there you have your first major "bug" to iron out.

Re:principle of least remorse (1)

ColdWetDog (752185) | about 2 years ago | (#41325021)

Ah, this explains why Microsoft is still around ....

Re:Attention whoring for funding (1)

jerpyro (926071) | about 2 years ago | (#41323679)

I did choose another profession. Do I still burn for physics research and progress? Yes. Did I consider myself above the petty politics that are involved with getting funding? Yes. Did ANY of my classmates that originally set out to do research in Physics end up doing that? No. A couple of patent lawyers, a couple of quantitative analysts, a few went engineering, and some went in to IT, but with the politicking at the labs and the sensationalized self-promotion that you have to do, it feels like being a used car salesman to justify your job.

We wonder why America is starting to fall behind other countries in the Sciences, and it's not just that we can't get younger kids interested, it's that it SUCKS when you get there, so we have nobody out there promoting how awesome it is.

Re:Attention whoring for funding (1)

Anonymous Coward | about 2 years ago | (#41323957)

My advisor is Chinese and has a job in China in addition to his one here. He knows scientists in virtually every country where there is science. Just last week, he said that he believes the (research/science) system in the US is the best in the world and that's why people want to come here. I take his statement very much to heart: I was convinced it was only a matter of time before he left his job in the US entirely and returned to China. What he said makes it very clear he will not do that.

So, things here might be pretty bad, but they're better here than they are in other countries, at least according to my advisor.

Re:Attention whoring for funding (2)

TapeCutter (624760) | about 2 years ago | (#41324601)

I'm not an international researcher but I'd agree that the US is high on the list of "research friendly nations", the science the US actually does is quite a contrast to the popular culture it projects, not to mention infinitely more valuable to the rest of the world.

Re:Attention whoring for funding (0)

Anonymous Coward | about 2 years ago | (#41325981)

I'm a faculty member at a major university and have considered getting out because of this whoring issue.

I don't blame anyone for leaving academic science because of what it's become, but you have to also ask yourself: if you leave, does it solve the problem? It solves your problem, but not the problem itself, which is still a problem for society, etc.

I don't mean to guilt-trip anyone, but ensuring science as a discipline has integrity is important for society at large. "Choose another profession" might be wise advice to any given person, but it doesn't solve the problem, which has become immense.

What's The Alternative? (-1)

Anonymous Coward | about 2 years ago | (#41322947)

What's the alternative? I'll be frank, as an outsider with no idea at all about how it works or what the process is...

It appears that these research projects are simply begging for money(lots or money) to go off and "study" XYZ. But, all too often it seems like XYZ is utterly pointless crap that doesn't really need study and any questions on the subject can be answered by applying a modicum of common sense. Ten year studies to find that underprivileged urbanites are underprivileged don;t really deserve one million dollars, in my opinion. But, with a million dollars in funding, who needs to really work(produce) for the next few years?

So, that said, what's your Ph. D. in? What's your plan? The hate you have is why, because you have to be a whore or because you think that it should be easier to convince people to give you money for likely nothing in return?

Those of us outside academia have to work for our "funding". And in cases where we also whore ourselves out in order to draw outside funding we still have to pay the funding back or give up significant interest in the companies we are building. We don't like that either.

Re:What's The Alternative? (0)

Anonymous Coward | about 2 years ago | (#41323345)

It is worse than this, most of them actually have no idea what they are doing. 80% of what is published is actually inconclusive due to crappy study design.

Re:Attention whoring for funding (1, Insightful)

jellomizer (103300) | about 2 years ago | (#41322467)

Business when done properly will have a profitable result.
Science when done properly will have either a positive or negative result.

The scientific process for the Facebook Generation...
I have this crazy idea.
How to measure if my crazy idea works.
Lets run tests that measure my crazy idea.
Does the tests match my expected results?
If (Not even close) { Your idea was really crazy, try an other one }
If (close) { Your idea may have some backing but will need to be tweaked }
if (spot on) { These results may be a fluke, try again and by different people, preferably by people who think your idea is insane }

Science is about being open to everything, only to find ways to shoot the bad idea out.

Re:Attention whoring for funding (4, Informative)

cheesecake23 (1110663) | about 2 years ago | (#41325413)

OK, it might take more energy to make a solar panel than we'll ever get back from it, but ...

Will you JUST FUCKING STOP spreading this lie? The energy payback time for photovoltaic modules according to most studies is between 1-4 years [nrel.gov] , depending on the material and manufacturing process used. Their technical lifetime is 25 years or more.

(I know I'm late to the party and hardly anyone will read this, but this is for the three of you who will.)

HOMO (-1)

Anonymous Coward | about 2 years ago | (#41322233)

NIGGER ASuSOCIATION

Very Suspicious (0)

Anonymous Coward | about 2 years ago | (#41322241)

Who was it that reported this result? The reporters!

Re:Very Suspicious (1)

erikkemperman (252014) | about 2 years ago | (#41324167)

Heh, I was thinking the same.

... are journalists, as a whole, really that bad at their jobs? Christie Wilcox reports that ...

Re:Very Suspicious (1)

TapeCutter (624760) | about 2 years ago | (#41324735)

... are journalists, as a whole, really that bad at their jobs? Christie Wilcox reports that ...

I think we could probably patent the application of the uncertainty principle to journalism. Yeah I know skepticism has prior art, but if we don't use that word, your average examiner won't know about it....

In other news... (4, Insightful)

Comboman (895500) | about 2 years ago | (#41322253)

So basically, most reporters just regurgitate press releases rather than doing any of that actual journalism stuff. That's not unique to science/medical reporting. It happens in political reporting, business reporting, hell even sports reporting. The bad science reporting is just more obvious because it's easier to debunk.

Re:In other news... (1)

Anonymous Coward | about 2 years ago | (#41322921)

The bad science reporting is just more obvious because it's the local hobby.

All bad reporting is easy to debunk by anyone who takes a little time to get mildly familiar with the subjects. You notice bad science reporting because it is your hobby. I know that I do not notice bad sports reporting because I pay less attention to sports than I do to a youtube video of a lava lamp.

The system selects for CONmen and Shysters (4, Interesting)

Advocatus Diaboli (1627651) | about 2 years ago | (#41322273)

I remember writing a post about this phenomena about a year ago. The short version of the story is that over the last 30-40 years, universities and research institutes have increasingly recruited "scientist" with strong tendencies towards showmanship, fraud, lying and bullshitting. This change is largely due to changing nature of incentives as well as methods of evaluation and promotion in these institutions. Peer reviewed research and grants are probably the biggest culprit. Here is the link: http://dissention.wordpress.com/2011/02/06/why-all-publicised-breakthroughs-are-lies/ [wordpress.com]

Re:The system selects for CONmen and Shysters (0)

oh_my_080980980 (773867) | about 2 years ago | (#41322943)

Spoken by a complete imbecile. Fraud will not work in peer reviewed work, let alone grant applications - it's too easy to spot. Universities recruit scientists that have a strong publishing background, i.e., they publish lots of research. Any promotion is done by the University.

Re:The system selects for CONmen and Shysters (-1)

Anonymous Coward | about 2 years ago | (#41323409)

LOL. Are you in acadamia? The fraud is implemented by "subtly" manipulating data and using known-to-be incorrect statistical methods. Reviewers are doing the same thing so they won't detect it.

Re:The system selects for CONmen and Shysters (1)

gringer (252588) | about 2 years ago | (#41323559)

Fraud will not work in peer reviewed work, let alone grant applications - it's too easy to spot

Fraud, at least in the form of bending the truth, is common in both peer reviewed work and in grant applications (and particularly encouraged in the latter). The reality of funding is that scientists need to be increasingly devious in order to make the funding body believe that their work is more important and deserves to be funded over that other group of scientists who are doing similar work (and engaging in similar deception).

I have had trouble thinking of an appropriate solution to this. Most research that gets carried out is useful in some way and deserves some funding, but the outcomes are frequently negative for the particular situations that were initially tested. On the other hand, I don't think it's good idea to give $100,000 to every person who wants to look at how long earthworms will survive after being dropped in water.

Re:The system selects for CONmen and Shysters (1)

fermion (181285) | about 2 years ago | (#41325063)

Universities need a number of scientist who can build departments and bring in funding. If you went a major research University you can thank these researchers for the availability of professors who can expound on a subject in more than a superficial form. They bring in the funds that pay the professors, graduate students, equipment and even buildings. If they are an evil, they are a necessary and often benign evil. They are either the first or last author on a paper. These professors are high profile, but are no where near the majority. Other professors, researchers, and post docs do need to bring in funding and write papers, but they can be much more focused on the research and teaching. Despite what most believe, I find that professors are real schools do care about doing a good job teaching. It is just that they are not trained in it, and the students still have middle school mentality in which they foster an adversarial relationship.

In terms of papers, actual researchers understand there is a context. Papers are published as communication between scientists to exchange processes, use of equipment, and finding. I recall on paper which focused on a particular feature of a particular graph taken with well known equipment. The paper asserted that this feature indicated a particular characterization of a sample. I knew my samples showed the same feature, but did were definitely not characterized as such. Some might say the paper was bad, but it showed an interesting use of equipment, showed indicated that they feature which I often saw was not indicative of a feature, and therefore allowed us to explore the feature more deeply. The media, who is only interested in results not the advancement of science, would have certainly not have anything useful to say on this.

So it is the medias fault in that it tries to make science about reactionary ultimate findings instead of a slow process intent on acquiring knowledge. Of course given that a large number of people in the world believe that all knowledge is contained in their particular fiction, there is not much the media can do.

Re:The system selects for CONmen and Shysters (0)

Anonymous Coward | about 2 years ago | (#41325081)

So, how is it different from any other workplace then?

Oh, that's right! It does not! People gravitate towards someone with charisma. And it usually takes a lot of charisma to bullshit your way through mistakes you are bound to make along the way.

Secondly, scientists have *nothing* in this regard to lawyers or politicians.

How to fix current problems? How about publishing repeated experiments linked to the original, be that positive or negative results. What I hate about current research papers is there is no direct followup. For example,

    1. magic beans cure XYZ bladder cancer in 99% of fat mice
                        * positive linked publications - 1
                        * negative linked publications - 45
                        * indeterminate linked publications - 5

What you get currently is citation count, which requires lots of work by everyone looking at said research to see that it failed. And there is a positive bias to publications too - very little research about "it does not work" - so you don't even see the negative publications because they did not happen.

So yes, fix journal citations and you fix science. As for science journalists, you can't fix lazy.

Yes (0)

Anonymous Coward | about 2 years ago | (#41322275)

but are are journalists, as a whole, really that bad at their jobs?

If I didn't see egregious errors like this while reading online journal articles I'd say no, but as it stands...

Scapegoats (0)

Anonymous Coward | about 2 years ago | (#41322287)

This is nothing more than the so called journalist trying to pass the blame. The fact is, stories are sensationalized all the time and it does not matter what science it is, or what the subject is. Science, Education, Politics, Economic, Environment, etc. etc. So stop blaming the scientist.

Re:Scapegoats (1, Insightful)

khallow (566160) | about 2 years ago | (#41322437)

And those other fields also often have the same sort of collusion between the reporter and the subject. The noteworthy claim in the article is not that scientists are generating spin that journalists exaggerate, but instead that most exaggerations and errors by journalists originate in such spin.

How that claim became your above "blame the scientist", I'll leave as an exercise for the reader.

Re:Scapegoats (1)

jellomizer (103300) | about 2 years ago | (#41322609)

Well I would blame society on a whole.

But it is important to try to get groups to realize that they are part of the problem.

Whenever you think you are part of a group that is some how immune to being part of the problem, then you open yourself up to be a bigger part of the problem.

For example...
The Catholic Priest Scandal: Priests for generations, heck for thousands of years, have been considered by the public to better people then the rest of us. So claims against them will suffer punishment from the victim as they seem that much better of a person that it couldn't happen.

Pen State: These figures were so popular and idolized as hero's that they got corrupt because no one was willing to draw the line on them.

Politics: Well that is too easy. Most recent I know of is the New Jersey mayor. ...

We as a society want a real life Superman, who has massive power and incorruptible. But that is fiction. It takes a lot of work to keep yourself following your morals, especially as you gain more power in life. Usually the stumbling block is the phrase "I Deserve this" which is either a slip in your diet, or taking a bribe for a Hawaii vacation in turn of not putting an investigation to a companies practices.

Solar Cells, Anybody? (2, Insightful)

Jane Q. Public (1010737) | about 2 years ago | (#41322339)

If even 1/10 of the hype about "breakthroughs" in solar cell efficiency were actually to be combined and made real in the marketplace, we'd all be charging the utility companies now instead of the other way around.

Re:Solar Cells, Anybody? (2)

Shavano (2541114) | about 2 years ago | (#41322481)

and we'd be getting a 100 Watts per square centimeter.

Press Releases? (5, Informative)

Anonymous Coward | about 2 years ago | (#41322387)

To be fair, university press releases are not written by the scientists who did the research, and in my experience the scientist often doesn't even get the chance to proof and correct them. I myself had my 15 minutes of international fame several years ago (the phone literally didn't stop ringing, interview requests from around the world, etc), all on account of a shockingly inaccurate press release from the university about some interesting but not earth-shattering research that I did.

I wonder (1)

wbr1 (2538558) | about 2 years ago | (#41322421)

I wonder if the abstracts contain spin 0, 1, 1/2, 3/2 etc. ? If the don't contain spin, is it a new type of physics?

Sensationalized Version of the Headline (0)

Anonymous Coward | about 2 years ago | (#41322433)

Scientists say we can't trust them!

JELLLY BEANS SURE CANCER! (0)

puddingebola (2036796) | about 2 years ago | (#41322441)

Abstract: USA TODAY LOOK HERE! Extensive research conducted in my garage on 3 pet gerbils has yielded statistically questionable results indicating jelly beans may prevent cancer. One gerbil that was lethargic and probably had cancer was fed jelly beans over a period of 2 days. The second gerbil, which probably also had cancer, wasn't fed anything. The third gerbil was control. At the end of the 2 days, the lethargic gerbil was happily running in his wheel, completely cured, while the other 2 gerbils died of malnutrition.

Bad Reporting (0)

Anonymous Coward | about 2 years ago | (#41322551)

Bad reporting such as taking a subset (medical science), claiming in your abstract that it applies to a much larger case (all science), and the saying that things come from a press office but blaming the scientists alone in your headline?

Yeah, I can see how scientists are to blame for that...

Are are journals... (4, Funny)

bluefoxlucid (723572) | about 2 years ago | (#41322569)

The question is not "are are journals, as a whole, really that bad" ... the question is...

IS OUR CHILDREN LEARNING YET?!!

Re:Are are journals... (0)

Anonymous Coward | about 2 years ago | (#41322721)

Well played.

Re:Are are journals... (1)

Crudely_Indecent (739699) | about 2 years ago | (#41323031)

Don't you mean:
"Are are children learning yet?!!"

From the Study's Abstract (3, Interesting)

Joe Torres (939784) | about 2 years ago | (#41322639)

They define spin as: "“spin” (specific reporting strategies, intentional or unintentional, emphasizing the beneficial effect of the experimental treatment)" They also mention: "We considered “spin” as being a focus on statistically significant results ... an interpretation of statistically nonsignificant results for the primary outcomes as showing treatment equivalence or comparable effectiveness; or any inadequate claim of safety or emphasis of the beneficial effect of the treatment." (emphasis added) I understand the last two, but the first point doesn't make any sense at all. You can't really make conclusions (you can, but scientists will not believe it) about statistically insignificant results. "Spin" can be good in some cases (maybe not at all in clinical research): a research group that studies DNA repair might state, "Our findings on the function of the yeast homolog of SLHDT in dsDNA break recognition may represent a novel target for cancer therapeutics." In this case, the research group doesn't study cancer at all and have no business at all (from their results) mentioning it, but this might convince a cancer researcher to consider reading the paper and possibly looking into doing a quick/cheap experiment targeting SLHDT and testing this claim.

3 words (0)

Anonymous Coward | about 2 years ago | (#41322657)

Global Climate Change

Re:3 words (1)

oh_my_080980980 (773867) | about 2 years ago | (#41322967)

I take it you're not a fan of Gravity either....

Re: 3 words (0)

Anonymous Coward | about 2 years ago | (#41324069)

You are an absolute idiot.

Surprising xkcd link (2)

Hognoxious (631665) | about 2 years ago | (#41322793)

Science news cycle [phdcomics.com]

Re:Surprising xkcd link (4, Funny)

wonkey_monkey (2592601) | about 2 years ago | (#41323149)

Surprising xkcd link

Is that "surprising" in the sense of "not an"?

Re:Surprising xkcd link (0)

Anonymous Coward | about 2 years ago | (#41325605)

xkcd has so gained so much control over Slashdot users that they assume every web comic must be xkcd.

Are journalists bad at their jobs? (0)

Anonymous Coward | about 2 years ago | (#41322799)

Let me think about it for a moment. ...
YES!

Of course, if you redefine their job as what journalists do today (i.e. repeat "two sides" of everything uncritically), then no. However, since science doesn't give a shit about "balance" between views unless both are equally appliccable and useful, journalism is never going to manage to report on science. Which would make science journalism pointless.

Whew (1)

Anonymous Coward | about 2 years ago | (#41322839)

I guess scientists are supposed to do all of the work now, are they? They have to do the science, write the papers, market it for their funders, write the articles for the news corporations, AND be the fall-guys when something isn't 100% accurate?

Whew. I'm glad I didn't remain a researcher.

Parable (2, Interesting)

Anonymous Coward | about 2 years ago | (#41322925)

A congressman was touring his district when he came upon a bunch of people in a big field with bows and arrows. They were all firing arrows in all different directions.

"What are you doing?" asked the Congressman.

"We are shooting arrows," said the archers.

"But there is nothing to shoot at," said the Congressman. "Those arrows are provided at taxpayer expense! How dare you waste them in this way?"

"Well," said the archers, "as you can see, we are very skilled archers. We can shoot arrows so far that they go over the horizon and we can't see them any more. We think there are targets out there over the horizon that we can hit, even though we can't see them yet."

The Congressman said, "Very well. But how do you know where the targets are?"

One archer said, "We just have to fire in random directions, because we don't know where the targets are."

Other archers agreed with the first one.

But then one of the archers said, "I have a different strategy. I am pretty sure that there is a target roughly in this direction, so I am shooting towards it. In fact, I think I may have already gotten a bullseye or two."

"You don't know that," said the others. "You've never been over the horizon to see whether there is a target or not. You have no more idea than the rest of us"

"Stop arguing," said the Congressman. "All of you lot, give all your arrows to this gentleman here. He is clearly the only one who has a concrete plan for hitting a target. I can't have you wasting any more taxpayer money shooting arrows at nothing."

"Wait!" said another archer. "For all I know, I might have gotten a bullseye also! I don't know where the target is, but it is possible, you have to admit!"

"Hmm," said the Congressman. "Give this lady some of the arrows too."

"Ah!" said another archer. "You know, the same thing is true of me!"

"Yes," said another. "And me!"

Pretty soon all of the archers had explained to the Congressman that they, too, could possibly have hit a bullseye, and had all been allocated arrows.

"There," said the Congressman at last. "Now the public can have confidence that their money is allocated to worthwhile projects. Keep up the good work, but don't let me catch you wasting taxpayer money like you were before." And he walked off, while the archers resumed firing arrows in the same directions as before.

Re:Parable - response (0)

Anonymous Coward | about 2 years ago | (#41326685)

'Half the money I spend on [advertising/science/military/etc.] is wasted; the trouble is I don't know which half.'

Waaaait (0)

Anonymous Coward | about 2 years ago | (#41323389)

I thought the peer review process and the integrity of scientists, their ideals and dedication to the truth made science unimpeachable. You mean scientists can be BIASED? Noooooo! They're objective! Say it aint so, Joe!

Interestingly (0)

Anonymous Coward | about 2 years ago | (#41323955)

The abstract for this study itself was sensationalized.

Key phrase "AND THEIR PRESS OFFICES" (2)

Burb (620144) | about 2 years ago | (#41323965)

Hardly "Scientists themselves", is it?

you don't say (0)

Anonymous Coward | about 2 years ago | (#41324305)

Guess I'll go back to buying organic food again now.

Spin in Journals (0)

Anonymous Coward | about 2 years ago | (#41324511)

There is of course some spin in scientific reporting. This is to get published. Journals will only publish article there they find the most interesting. Scientists are required to put the most optimistic language, the most exciting looking data, so that scientists use their work to build off and cite the publications.

This is a basic problem in the way science currently operates. The funding for each scientist is usually based on how many articles they publish, compared to the competitors. So we have to spin every result we can get our hands on to publish and get cited in other publications.

The competitive nature of funding in science has forced us resort to spin rather than accurate language and completely ignore negative results.

I hate Odds Ratios (1)

canajin56 (660655) | about 2 years ago | (#41325239)

One of the worst "bad abstract tricks" is putting your findings as Odds Ratios. What's an Odds Ratio? You probably know that the "probability" of an event is "Event over Total". The probability of rolling a 6 on a standard die is 1/6. The "odds" of an event is "Event to Not Event". The odds of rolling a 6 are not 1:6, they are 1:5 for (or more often said, 5:1 against). So then the odds ratio (OR) of two groups is the ratio of ratios, or the ratio of the odds for one event compared to the odds of another. So a big source of confusion is thinking the odds and probability are the same thing. Clearly they aren't. And clearly the closer they get to even odds, the bigger the difference. The odds of tossing a coin and getting heads are 1:1, but that's a 1:2 probability.

An example of the odds ratio in action: You ask 1000 men if they smoke, and you get 300 who say "yes" (made up statistics). That's odds of 300 to 700, or 3:7. You ask 1000 women if they smoke, and 250 say "yes". That's odds of 250:750, or 1:3. The odds ratio is then (3:7) : (1:3) or 9:7, or 1.2857...:1 So in the abstract you will see that this study has found that males have an OR of 1.29 when compared with women. And they'll just sit back and let the journalists call that "almost 30% more likely!" When it's not. That's how much higher the odds are, and odds are not probability! And of course you can't forget about confidence intervals. It's actually even worse than that. An increasing number of medical papers will take the OR of 20:1 and go straight to "20 fold more likely to blank!" when the probability ratio is 3.5:1 not 20:1.

Part of the problem is not enough statistics courses for scientists. I had to take 2 as part of my degree, and they never covered odds ratios, or odds at all actually. Only probabilities, which are more useful to reason about usually. This is further compounded by people using odds and probability interchangeably. I see on things like scratch and wins and store give aways "Odds of winning 1 in 3", which is a probability.

Re:I hate Odds Ratios (1)

tibit (1762298) | about 2 years ago | (#41325451)

Wait a minute: how the heck OR 20:1 implies probability ratio 3.5:1? The probability of event A is 1/21 = 0.048, probability of B is 20/21 = 0.95. The ratio p(A)/p(B) = 0.051 or ~1/20. It's seem that the OR approaches probability ratio as the OR goes away from 1:1. It's seem to me that OR is farthest away from probability ratio when OR = 1:1. Or else I'm not getting what you mean by OR 20:1 implying 3.5:1 probability ratio. Probability ratio of *what*?

Abstracts vs. articles in medical science (1)

Harvey Manfrenjenson (1610637) | about 2 years ago | (#41325609)

If all you look at is the abstract and "conclusions", of course you're going to get an unbalanced view of what the study said. Think of all the other information that is contained in the body of the paper. There's a discussion of the methodological limitations of the study, there's a discussion of all the outcome measures which DIDN'T reach statistical significance, there's a discussion of adverse events, and there's usually also a discussion of where this study fits into our knowledge of the topic as a whole (e.g., "We found fish oil supplements to be effective for arthritis but 5 of 7 previous studies have shown the opposite results"). All of this is crucial information, but you couldn't stuff it all into the abstract even if you tried.

So yeah, I'm going to lay 90% of the blame for this on the journalists. My impression is that most of them don't read the body of the article. In some cases, I would bet that they don't even possess copies of the article (since articles, unlike abstracts, are usually kept behind a very expensive paywall). That would have been an interesting statistic for the researchers to look at.

Reporters need to do their job (1)

SoftwareArtist (1472499) | about 2 years ago | (#41326015)

This seems like a weak attempt to shift the blame for bad reporters. Their job is to get at the facts and report what is really true. That's what reporters do - at least if they're any good. So scientific press releases contain spin? Shocking! Just like press releases in absolutely every other field. Any reporter who just parrots a press release without understanding it and getting at the truth is a bad reporter.

Yes, science is complicated. Yes, it takes specialized knowledge to understand. Just like every other field. That's why there are science reporters who supposedly have that specialized knowledge.

I think they ARE ARE (1)

jameshuckabone (2648439) | about 2 years ago | (#41326225)

Check the first sentence assholes.

Re:I think they ARE ARE (1)

ajlitt (19055) | about 2 years ago | (#41326321)

You must be new here.

Oh wait, you are. Rage on.

No more evident is this..... (1)

Kr1ll1n (579971) | about 2 years ago | (#41327227)

than in scientific articles regarding abortion.

From the Guttmacher Institute study regarding mortality rates between abortion and childbirth;
http://www.ncbi.nlm.nih.gov/pubmed/22270271 [nih.gov]
"METHODS:
We estimated mortality rates associated with live births and legal induced abortions in the United States in 1998-2005. We used data from the Centers for Disease Control and Prevention's Pregnancy Mortality Surveillance System, birth certificates, and Guttmacher Institute surveys. In addition, we searched for population-based data comparing the morbidity of abortion and childbirth.

CONCLUSION:
Legal induced abortion is markedly safer than childbirth. The risk of death associated with childbirth is approximately 14 times higher than that with abortion. Similarly, the overall morbidity associated with childbirth exceeds that with abortion."

So here you have someone claiming scientific fact, and stating that the methods used were based on "estimated mortality rates". When questioned about this, The only response the doctors involved could provide was that statistical information in the US sucks in regards to maternal mortality issues. So in a nutshell, we estimated, and feel our estimations permit us to announce to society that X is safer than Y.

That's why when debating abortion, I lean on the Finland statistics. Finland's population is not nearly as diversified (less likelihood of genetic predisposition to external mortality factors), and has access to universal healthcare. With the universal healthcare, they also have much more reliable information in regards to maternal mortality rates, and they have no dog in this fight, since nearly unrestricted and fully legalized abortion has been available to the Finnish people since 1970.

http://www.ncbi.nlm.nih.gov/pubmed/14981384 [nih.gov]
http://www.ncbi.nlm.nih.gov/pubmed/9292639 [nih.gov]

However, to this day when discussing the issue with some people, they pull out the Raymond\Grimes authored, Guttmacher Institute-backed study as though it is infallible, and by the author's own admission, that is definitely not the case. The study authors should be attempting to correct this perception, but refuse to do so, as it would mean the study is fallible and flawed.

I have even had someone claim the study (which is paywalled) is 100% legitimate, but when I asked this same person about the Finland studies, their response was "I cannot comment on those, since they are paywalled."

Right now, people want to be right about something, science be damned, and it is causing our society to fail miserably in all areas. I gave abortion as an example, since it truly exists in the science and politics bubble that causes this problem.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...