×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

How Statistics Can Foul the Meaning of DNA Evidence

timothy posted more than 2 years ago | from the persecution-rests dept.

The Courts 215

azoblue writes with a piece in New Scientist that might make you rethink the concept of "statistical certainty." As the article puts it, "even when analysts agree that someone could be a match for a piece of DNA evidence, the statistical weight assigned to that match can vary enormously, even by orders of magnitude." Azoblue writes: "For instance, in one man's trial the DNA evidence statistic ranged from 1/95,000 to 1/13, depending on the different weighing methods used by the defense and the prosecution."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

215 comments

Damn Lies and Statistics! (2, Interesting)

lawnboy5-O (772026) | more than 2 years ago | (#33307088)

Isn't it the case that we are more often in the way of our own discovery and explana-tative power

Lies, Damn Lies, and Statistics! (2, Funny)

Local ID10T (790134) | more than 2 years ago | (#33307138)

You can prove anything with statistics.

Also 99.9% of all statistics are made up.

Re:Lies, Damn Lies, and Statistics! (5, Insightful)

Kitten Killer (766858) | more than 2 years ago | (#33307522)

You can prove anything with statistics.

No. You can prove anything with BAD statistics. Unfortunately, most statistics are bad.

-Scientist Statistician (enough to know that I don't know statistics)

Re:Lies, Damn Lies, and Statistics! (0)

Anonymous Coward | more than 2 years ago | (#33307556)

Also 99.9% of all statistics are made up.

Crap. Studies show that no more than 87% of statistics are made up, and of those 4% are roughly right anyway.

Statistics (0)

Anonymous Coward | more than 2 years ago | (#33307744)

I always heard is was 90% of statistics are made up 50% of the time.

Re:Damn Lies and Statistics! (2, Interesting)

Jurily (900488) | more than 2 years ago | (#33307190)

DNA evidence is the new fingerprint. News at 11.

Re:Damn Lies and Statistics! (5, Informative)

HangingChad (677530) | more than 2 years ago | (#33307858)

Sadly true, but there's so much about DNA analysis that you don't get on an episode of CSI. On TV DNA analysis only takes a few minutes and matches are proudly announced by flashing messages on the DNA machine.

In real life good DNA matching takes days, cost a lot of money and, as the article points out, matching can be in the eye of the beholder. DNA samples are incredibly easy to contaminate, whole labs can become contaminated over time if they don't have and follow strict contamination protocols. And there has been more than one reported case of harried techs gun-decking DNA analysis when police and prosecutors were certain they had the right guy.

Well done DNA analysis can be an amazing crime fighting tool but the science is not perfect and it's okay to be skeptical. There is no magic identification test that's completely fool proof. And DNA tests are only as good as the fool running the test.

Re:Damn Lies and Statistics! (0)

Anonymous Coward | more than 2 years ago | (#33308936)

I was almost caught smoking some cannabis, as it happens I tossed the pipe in a bush, and my accomplice threw the baggy in another. They hunted everywhere for some evidence, and eventually they found the pipe.
Needless to say I denied any knowledge of it, but then the cop said he was going to take it back to the station to DNA test it, at that point I laughed in his face.
Fucking ass-holes, I know It's their job to enforce the law, but I bet they wouldn't have put that much effort into investigating a real crime.

Re:Damn Lies and Statistics! (4, Insightful)

finarfinjge (612748) | more than 2 years ago | (#33307348)

The trouble that this paper and many others illustrate is the HUGE ignorance of proper statistical methods in the scientific community. Such things like a students T test are - statistically speaking - simple. Yet they are often beyond many in the science community. Thus, there is a tendency for misuse of technique, which in turn leads to divergent interpretations of what a data set means. The legal profession is even worse, as they don't care about the laws of mathematics. In a court, you are not required to answer to a professor of mathematics, hence you can assert anything. If your opponent doesn't have the necessary skill or knowledge to call BS on what you say, you can win an argument with a completely baseless assertion. Take an example. A man is fired for missing work on a Monday. The company's lawyer states "Fully 40% of this employee's absenteeism occurs on Mondays and Fridays. It is appalling that this weekend extending behaviour continues, and we must do something about it". The mathematically challenged lawyer for the poor sap can't see the issue with this and lets it stand.

JE (always wanted to use that example. May have the justification a bit!)

Re:Damn Lies and Statistics! (1, Insightful)

Anonymous Coward | more than 2 years ago | (#33307682)

I'm a phd student in physics, and I am very grateful that my advisor insists on doing statistics from first principles, i.e. understanding everything from the principle of maximum likelihood. There are lots of subtle statistical errors you can make if you don't completely understand what you're doing.

Re:Damn Lies and Statistics! (0)

Anonymous Coward | more than 2 years ago | (#33308166)

Of course there's still a zillion statistical errors to be made within the MLE framework, and maximum likelihood in and of itself isn't the only principle (nor necessarily the right one to use) in statistics.

Re:Damn Lies and Statistics! (1)

Beardydog (716221) | more than 2 years ago | (#33307892)

It's always more fun to write a random generator to compare real results to, and it always winds up taking more factors into account than I would otherwise.

Re:Damn Lies and Statistics! (1)

techno-vampire (666512) | more than 2 years ago | (#33308646)

The legal profession is even worse, as they don't care about the laws of mathematics. In a court, you are not required to answer to a professor of mathematics, hence you can assert anything. If your opponent doesn't have the necessary skill or knowledge to call BS on what you say, you can win an argument with a completely baseless assertion

Well, then, if you expect your opponent to pull something like that, bring in a statistician, qualify him as an expert witness and let him rip the assertion to shreds.

Hmm... Good (-1, Troll)

Anonymous Coward | more than 2 years ago | (#33307114)

This sounds like a good reason to stop releasing all of those convicted murderers and rapists who were freed on DNA evidence.

Re:Hmm... Good (2, Insightful)

Anonymous Coward | more than 2 years ago | (#33307258)

Your post sounds like a good reason for you to shut the hell up.

Re:Hmm... Good (0)

Anonymous Coward | more than 2 years ago | (#33307370)

Why? The convicts who are now being released weren't convicted on DNA evidence. Since DNA evidence is now obviously a sham, it shouldn't be used to free someone who was justly convicted with other evidence.

You either believe your justice system is fair or else you scrap the entire thing. Your alternative would mean that we would have to release every murderer and rapist.

Re:Hmm... Good (5, Insightful)

Lunix Nutcase (1092239) | more than 2 years ago | (#33307460)

it shouldn't be used to free someone who was justly convicted with other evidence.

And you know that the other evidence wasn't faulty, how? Police make mistakes, witnesses lie or remember things wrong, etc etc.

You either believe your justice system is fair or else you scrap the entire thing.

Or you ditch that false dichotomy and realize that within every system mistakes will be made. There is nothing in fixing past errors that means you throw out the whole system.

Your alternative would mean that we would have to release every murderer and rapist.

No, actually it wouldn't.

It's not over until the fat lady sings. (1)

westlake (615356) | more than 2 years ago | (#33308716)

And you know that the other evidence wasn't faulty, how? Police make mistakes, witnesses lie or remember things wrong, etc etc.

You can't be certain of anything.

But neither can you retry every case infinitely because there are some remaining doubts. There will always be doubts.

The appeals court is a court of law.

It's only job is to decide as a matter of law whether a conviction should stand.

You raise your objections to matters of fact or opinion in pre-trial proceedings. You raise them again at trial. But you must get your objections on record before your case goes to appeal.

It is a lot to ask an appellate judge to believe that matters which seemed inconsequential to you then should be given any weight now.

Re:It's not over until the fat lady sings. (2, Insightful)

TheEyes (1686556) | more than 3 years ago | (#33309298)

But neither can you retry every case infinitely because there are some remaining doubts. There will always be doubts.

You can and you should retry cases if there are doubts; you should acquit immediately if there are any reasonable doubts.

The rule of law in this country is founded on the idea that people are innocent until proven guilty, beyond a reasonable doubt. Not "pretty sure," not "it's too much trouble to give you a fair trial, so we'll just convict you anyway." Beyond a reasonable doubt. That standard of proof will inevitably mean that people who actually committed crimes will be let free, and some will indeed go on to commit more crimes, and that is unfortunate. This country, however, is supposed to be based on the love of freedom, and the notion that everyone deserves not to be railroaded by a kangaroo court bent on throwing as many people as they can into a life of permanent second-class citizenship (convicted criminals have few rights in this country, and remain persecuted even long after they've "paid their debt to society.") You can't have that if you are willing to sacrifice freedom for temporary, largely illusory, safety.

It's fine for saying "it's somebody else". (4, Informative)

Ungrounded Lightning (62228) | more than 2 years ago | (#33307574)

This sounds like a good reason to stop releasing all of those convicted murderers and rapists who were freed on DNA evidence.

Not at all.

There is no problem determining that the DNA is from somebody else than the accused. All it takes is a single marker that's different. That's easy.

The problem is going from some bunch of markers that match to saying "This IS the bum! (Well, except for a one-in-[some number] chance it really isn't.) That requires a lot of information about prevalence of genetic markers, whether there is a correlation between their distribution. That information isn't well researched and the different estimates are based on different wild guesses by different experts. Further, the whole independent-probability thing gets knocked into a cocked hat with FAR lower numbers if the police found the accused by searching a DNA database for matches. And what if he had an evil identical twin? Or somebody with access to PCR gene-amplification materials, a DNA sample, and an atomizer decided to frame him?

IMHO DNA evidence is decisive for the defense. But pending a lot more research it's still voodoo for the prosecution.

Re:It's fine for saying "it's somebody else". (4, Insightful)

sjames (1099) | more than 2 years ago | (#33307730)

Simply, DNA evidence is by nature exclusionary. The scientifically correct result of a DNA test is excluded or not-excluded.

Re:It's fine for saying "it's somebody else". (2, Interesting)

Qzukk (229616) | more than 2 years ago | (#33307818)

IMHO DNA evidence is decisive for the defense

DA's these days refuse to accept that. For instance, down here in Texas we had a guy convicted of raping a woman. The woman claimed two guys raped her. Two sets of male DNA were recovered. The technician lied^Wmistakenly testified on the stand that the guy matched one set. One MASSIVE scandal later, his DNA was retested and didn't match either set of DNA.

That should be it, right? Well, the DA spent quite a lot of time fighting the release, insisting that his that this guy was one of the rapists and wore a condom and the woman couldn't count to three. Of course, I'm sure the fact that we end up paying people who get imprisoned because the government fucks up had no bearing at all on the government's desire to convince everyone they didn't fuck up.

Re:It's fine for saying "it's somebody else". (0)

Anonymous Coward | more than 2 years ago | (#33307962)

"If I owned both Hell and Texas, I live in Hell and rent out Texas" - Mark Twain

Re:It's fine for saying "it's somebody else". (2, Insightful)

winwar (114053) | more than 2 years ago | (#33308162)

"Of course, I'm sure the fact that we end up paying people who get imprisoned because the government fucks up had no bearing at all on the government's desire to convince everyone they didn't fuck up."

Actually, probably not. The DA may just not want to admit they made a mistake. It's uncomfortable to process those facts so the DA doesn't. They probably even believe that the person is guilty. Cognitive dissonance and the like is pretty powerful.

Re:It's fine for saying "it's somebody else". (2, Insightful)

corbettw (214229) | more than 2 years ago | (#33308356)

DAs are rewarded for getting guilty verdicts and sending people to prison, not for finding the guilty party and punishing them. There's a very subtle difference there, and it means that a DA with so-so evidence against a defendant who's easily portrayed as scum (with a PD for a lawyer) versus rock-solid evidence against an upstanding citizen (who can afford their own attorney) will prosecute the former over the latter. It's an easy win, who cares if the guy is really guilty?

Re:It's fine for saying "it's somebody else". (1)

mhajicek (1582795) | more than 2 years ago | (#33308590)

Remind me to put someone else's skin under the fingernails of my next murder victim.

Re:It's fine for saying "it's somebody else". (0)

Anonymous Coward | more than 2 years ago | (#33308636)

IMHO DNA evidence is decisive for the defense. But pending a lot more research it's still voodoo for the prosecution.

I'm reminded of the Duke lacrosse team "rape" case. She had DNA from at least four men, none of whom were accused of raping her. Did that prove she wasn't raped? Not at all. The lack of any other evidence that she was raped was enough to cast doubt on her story though. However, if some of that DNA showed a high probability of being from one of the team members it would've been a different story.

1/13 (5, Funny)

nacturation (646836) | more than 2 years ago | (#33307160)

"Members of the jury, there's only a 1 in 13 chance that the defendant is actually the killer based on the DNA evidence. If the defendant were sitting in the jury with you, then there's an equal chance that it was any one of you. And since we can eliminate all 12 of you, that leaves only the defendant left over. So you must find the defendant guilty of all charges since he's the only one left out of 13 people. The prosecution rests."

Re:1/13 (-1, Flamebait)

blair1q (305137) | more than 2 years ago | (#33307526)

Isn't that how we determined there must be WMD in Iraq?

Re:1/13 (3, Informative)

gandhi_2 (1108023) | more than 2 years ago | (#33307598)

No, we did that with our copies of sales receipts.

You used them already? Hmm...well this is embarrassing.

Re:1/13 (0, Offtopic)

blair1q (305137) | more than 2 years ago | (#33308748)

It's nice to see there are those who still think we were right to lie about WMD in Iraq, and have to express their disgust for the truth by hiding behind mod points.

Re:1/13 (0)

Anonymous Coward | more than 2 years ago | (#33307672)

Ladies and gentlemen of the jury, please take a moment to look at the jurors around you. Pretty shifty, right? And why are they looking at you like that? You know, where would be the best place for a murderer to hide? In the jury of his own trial would be pretty sneaky, right? And where's the last place he'd want to be? On trial. The only person you can trust in this room is the defendant. At least he's under guard. If you think that shifty slimy murdering scumbag next to you in the jury is looking at you weird then you must acquit my client.

Re:1/13 (1)

anagama (611277) | more than 2 years ago | (#33307738)

What about the alternate juror or jurors? Usually there is at least one alternate who sits through the evidence, and often more than one. So while your logic is unassailable in a 12 person jury with no alternates, you could have real problems if there are thirteen jurors (12+1 alternate).

Re:1/13 (0)

Anonymous Coward | more than 2 years ago | (#33307942)

I think that the prosecuter would say:
"Members of the jury, there's only a 1 in 13 chance that the defendant is the NOT killer based on the DNA evidence. ...

Re:1/13 - 1/18 (1)

hydromike2 (1457879) | more than 2 years ago | (#33308142)

In reality it needs to be at no more than 1/18 to be sure, 1 defendant, 12 jury, 2 lawyers(minimum if not defending yourself), 1 judge, 1 bailiff, 1 of those people who records everything said, and usually some spectators. At 1/13 there must be at least a co-conspiritor in the room!

Re:1/13 (0)

Anonymous Coward | more than 3 years ago | (#33309144)

Where on earth am I going to find a good lawyer whose a world class mathematician? Youve got the old right / side left side of the brain word / numbers issue and then Charlie epps works for the FBI

Numbers don't lie. (2, Insightful)

Beelzebud (1361137) | more than 2 years ago | (#33307254)

But liars love to use numbers!

Re:Numbers don't lie. (1)

iammani (1392285) | more than 2 years ago | (#33307342)

So anybody who uses a number must be a liar?

Re:Numbers don't lie. (0)

Anonymous Coward | more than 3 years ago | (#33309268)

So anybody who uses a number must be a liar?

Fuck you 3 times.

Re:Numbers don't lie. (1)

DriedClexler (814907) | more than 2 years ago | (#33307464)

No, no, no, that's not how it goes. It's "Numbers don't lie, but liars sure number [in the billions]!"

Here's a tip: DON'T GET ARRESTED !! (0, Flamebait)

Anonymous Coward | more than 2 years ago | (#33307272)

If you are arrested, statistically YOU FUCKING DID THE CRIME !! Don't do the crime if you can't do the time: DON'T DO IT !!

And for all you fucking lip readers out there:

D.O.N.'.T. D.O. I.T. !.!.

Re:Here's a tip: DON'T GET ARRESTED !! (1)

The_mad_linguist (1019680) | more than 2 years ago | (#33307300)

I see you have an excellent grasp of traditional literary technique. By addressing a non-present audience (in this case, the lip-readers), you have created a subtle interplay between apostrophizing and apostrophes.

Here’s a tip: Go fuck yourself. (3, Insightful)

clone53421 (1310749) | more than 2 years ago | (#33307380)

Why don’t you just suggest that anyone who’s arrested is “statistically” guilty and we should just skip the trial...

Re:Here’s a tip: Go fuck yourself. (2, Informative)

sjames (1099) | more than 2 years ago | (#33307766)

That procedure is called charge inflation and plea bargaining. It's done all the time.

Re:Here’s a tip: Go fuck yourself. (0)

Anonymous Coward | more than 2 years ago | (#33308792)

...and people wonder why law enforcement gets no respect.

MADD mothers do it all the time (3, Insightful)

mangu (126918) | more than 2 years ago | (#33308542)

Why dont you just suggest that anyone whos arrested is statistically guilty and we should just skip the trial...

That's being done routinely all over the world today.

People who drink are statistically more likely to commit traffic accidents, so they are convicted without the need to actually do any harm to anyone.

Re:Here's a tip: DON'T GET ARRESTED !! (2, Funny)

david_thornley (598059) | more than 2 years ago | (#33307608)

I would imagine the lip readers you're addressing are too interested in the sex to pay attention to your letters.

Re:Here's a tip: DON'T GET ARRESTED !! (1, Insightful)

anagama (611277) | more than 2 years ago | (#33307832)

Idiots like you are a perfect example of what happens to people after years of living under administrations (past and current) which have been and are, hell-bent on destroying civil liberties and due process. The remaining people who fail to fall in line will simply be arrested, and that will take care of any questions.

Re:Here's a tip: DON'T GET ARRESTED !! (0)

Anonymous Coward | more than 2 years ago | (#33307876)

Statistically speaking, Anonymous Cowards are rectum sucking, spooge eaters.

I'm the exception.

Apparently you're dead in the middle of the distribution curve.

I'll get you a napkin for your face. It looks like you could use it.

Re:Here's a tip: DON'T GET ARRESTED !! (0)

Anonymous Coward | more than 2 years ago | (#33307954)

If you are arrested, statistically YOU FUCKING DID THE CRIME

LOL fuck you. Explain the people arrested for "Resisting Arrest" but no other charges. They didn't do jack but got arrested anyway.

Re:Here's a tip: DON'T GET ARRESTED !! (0)

Anonymous Coward | more than 2 years ago | (#33308322)

Sounds like a trouble maker. Officers of the law do not go around whilly-nhilly arresting blokes because they will resist, they are arrested because they are no good. I've given many a drunkard this choice: you can go sleep at home tonight if you can get up and walk away, or you can be placed under arrest. Those stupid enough to get arrested, need to be, and it always goes like this

CASE CLOSED
GUILTY
NEXT

Re:Here's a tip: DON'T GET ARRESTED !! (2, Insightful)

Anonymous Coward | more than 3 years ago | (#33309116)

that would be true if cops took their jobs seriously in objective manners.. they statistically do not. the kind of person that gravitates to law enforcement is one with deep seated insecurities and thus the desire to make others conform to his expectations. it's no wonder that fallacies like appeal to authority and appeal to popularity are among cops' favorite justifications.

Whaa? (5, Insightful)

esocid (946821) | more than 2 years ago | (#33307282)

In the Smith case, the sample containing another person's DNA showed alleles at seven out of a possible 15 loci, but at four of these loci, the alleles matched those of both the victim and the defendant. "The 1 in 95,000 figure in effect treated these alleles as full-weight evidence that the DNA came from the victim, ignoring the alternative possibility that the allele we saw could have been from the defendant," says Balding. If the opposite position is taken, and these alleles are ignored, you come up with a figure closer to 1 in 13. "It's a question of which loci you consider," he says.

Since when in the hell do you count common matches as proof that it comes from one person? Some of these labs are doing something very wrong, and I hate to think of both the false positives, and negatives, that came from their "expert" opinions.

Re:Whaa? (3, Insightful)

sjames (1099) | more than 2 years ago | (#33307878)

Since when you're a prosecutor that doesn't really care if the defendant is guilty or not as long as he gets locked up and you get credit.

The way the "justice system" currently works, if crime magically stopped right now, prosecutions and convictions would continue unabated.

Juries (2, Interesting)

gibson123 (1740752) | more than 2 years ago | (#33307304)

And add to the fact if you have served on a jury before, many times this information is highly technical and is very easily miss-represented by the lawyers to jury members from all walks of life.

Re:Juries (1)

jtownatpunk.net (245670) | more than 2 years ago | (#33308276)

Add to that the fact that most people are stupid and it gets really ugly. I'd never be able to get a jury of my peers because my peers would be smart enough to get out of jury duty. Unless they were bored and just wanted something to do for a day or three.

Fellatio Witness (1)

Myopic (18616) | more than 2 years ago | (#33307462)

It seems to me that the simple defense to such a charge would be to go get the person who recently sucked the defendant's dick, test that person's DNA, and compare it to the other DNA samples. Presumably, however, even a bad lawyer would have thought of this, so I must not really understand the case.

Three cheers, for sure, for an opportunity to topically discuss fellatio on Slashdot.

Re:Fellatio Witness (1)

CyberBill (526285) | more than 2 years ago | (#33307808)

I was thinking the exact same thing!

Its not like they randomly selected this guy off the street, swabbed his johnson, and said "This might be the rapist!"

They must have already had pretty substantial evidence to be able to get him, so really the low probability doesn't matter, because 1/13 means that 12/13 chance it would have cleared him by finding someone else's spit on his stick. The DNA evidence still makes the case against him stronger.

GUILTY!

Re:Fellatio Witness (1)

winwar (114053) | more than 2 years ago | (#33308040)

"They must have already had pretty substantial evidence to be able to get him, so really the low probability doesn't matter, because 1/13 means that 12/13 chance it would have cleared him by finding someone else's spit on his stick."

It didn't matter in this case because of the witness (and of course witnesses never make mistakes....) But it matters in other cases where the only lab evidence may be DNA. In that case, the difference is reasonable doubt.

I'd say there is a serious problem with DNA evidence. The "experts" don't know, can't decide and/or don't disclose how accurate it is. As a result most people think it's infallable, including the lawyers and judges. That's bad.

DNA is politically charged (4, Informative)

hessian (467078) | more than 2 years ago | (#33307488)

Genetics means "out of your control" and touches on some raw nerve issues, so there's a lot of throwing around of "statistical" information and unrealistic mental models.

For example of statistical confusion:

New research shows that at least 10 percent of genes in the human population can vary in the number of copies of DNA sequences they contain--a finding that alters current thinking that the DNA of any two humans is 99.9 percent similar in content and identity.

http://www.hhmi.org/news/scherer20061123.html [hhmi.org]

And broken mental models:

http://en.wikipedia.org/wiki/Lewontin's_Fallacy [wikipedia.org]

Until our knowledge improves, you're going to see more "politicization" of DNA-related science.

But each time you measure it you reduce the (1)

blair1q (305137) | more than 2 years ago | (#33307514)

uncertainty.

Each time the answer comes up the same, even in a test with much less accuracy, you improve the chances it's the right answer.

Throw some statistical analysis at it and come up with a way to combine the tests you have into one probability, which will be higher than all the probabilities you got from the measurements.

Unless the tests disagree, and then you're talking a homework problem.

Re:But each time you measure it you reduce the (0)

Anonymous Coward | more than 2 years ago | (#33307754)

Oh, there you go, you just fixed all the problems with DNA in the courtroom.

Or maybe you didn't.

Re:But each time you measure it you reduce the (0)

Anonymous Coward | more than 2 years ago | (#33307804)

uncertainty.

Each time the answer comes up the same, even in a test with much less accuracy, you improve the chances it's the right answer.

Throw some statistical analysis at it and come up with a way to combine the tests you have into one probability, which will be higher than all the probabilities you got from the measurements.

Unless the tests disagree, and then you're talking a homework problem.

That is generally true unless your test contains an unknown bias. Then it is possible that you get the same wrong measurement every time.

Re:But each time you measure it you reduce the (1)

sjames (1099) | more than 2 years ago | (#33307936)

We see how well that worked out on Wall street!

The problem with low accuracy tests is that they may not be RANDOMLY inaccurate. If some common factor causes each of the tests to read wrong, then the combined result of all of the tests remains no more accurate then the least of them.

Re:But each time you measure it you reduce the (1)

blair1q (305137) | more than 2 years ago | (#33308068)

There isn't much in common between different DNA tests. Temperature effects, maybe. And the fact that the DNA is the same. Which is the thing you're trying to show is the same. Which is why you chose a different test instead of running the same test twice.

I'm not sure how Wall Street fits in. The Stock Market is a random walk, not a sequence of nucleotides.

Re:But each time you measure it you reduce the (2, Informative)

sjames (1099) | more than 2 years ago | (#33308654)

Temperature effectsm, the same properties of the nucleotides (even if from distinct DNA), the same crime scene contamination effects, the same statistical laws (if it turns out not to be as unique as we think, all tests necessarily fail) and at least some of the chemical reactions are in common.

In one documented case, a surprise common factor was a quality control person who didn't know the non-sterile swabs still couldn't be touched before packaging. (The CSI:NY episode borrowed that from real life).

Wall street fits in because they too believed that a series of high risk propositions (this test is accurate or this sub-prime loan won't default) could be somehow bundled together to make a low risk proposition (AAA bonds in the Wall street case, beyond reasonable doubt in the DNA case). In both cases, the unexamined dependencies are the downfall.

Re:But each time you measure it you reduce the (1)

honkycat (249849) | more than 2 years ago | (#33308092)

True, but this only works if you disclose the results of every test that you used to try to demonstrate that you have the correct identity. After all, if you ran 13 independent tests that each had a 1 in 13 false-positive ratio, you're virtually guaranteed that at least one of them will come back positive. If you then only show the jury the results of that test, even the 1 in 13 ratio is false since you selected that test *after* you knew the results.

Combining clean science with an adversarial prosecution model is difficult---the prosecution is mostly out to get a guilty verdict, not get a correct result. The defense *must* have full access to every piece of investigation that the prosecution undertakes, not just those that they consider important enough to take to trial. Those "inconclusive" and failed tests are crucial if you want to perform a proper statistical check on the results of a test. The defense needs to be able to ensure that the whole story is told, and it doesn't seem that there's enough regulation in place to let this happen. Frequently they're not allowed to challenge the statistics presented by the prosecution in DNA or fingerprinting cases, and that breaks the system really badly.

Re:But each time you measure it you reduce the (1)

sexconker (1179573) | more than 2 years ago | (#33308534)

uncertainty.

Each time the answer comes up the same, even in a test with much less accuracy, you improve the chances it's the right answer.

Throw some statistical analysis at it and come up with a way to combine the tests you have into one probability, which will be higher than all the probabilities you got from the measurements.

Unless the tests disagree, and then you're talking a homework problem.

So basically, you know absolutely nothing about DNA analysis.

Re:But each time you measure it you reduce the (1)

mhajicek (1582795) | more than 2 years ago | (#33308694)

Example: We know the killer has red hair. Testing the defendant for red hair turns up positive. Since there are many others with red hair, we should test the defendant many times to be REALLY SURE that he has red hair. Lock'em up.

People don't understand statistics (4, Interesting)

91degrees (207121) | more than 2 years ago | (#33307538)

I speak from personal experience. I use them al the time and still don't really understand them. Not how they apply in criminal investigations anyway.

Let's say you have evidence that matches 1 in a thousand people. You search through your database of all 1000 suspects and you get a single match. Did he do it? Logically you'd expect this to mean you can be 99.9% sure. You then search through the database of a million random people. You get 1000 matches. Does this mean there's only a 0.1% chance that your original suspect was guilty? Well, maybe there's some other compelling evidence that makes it most likely that one of those 1000 people were the culprits. But you have 10000 outliers. They're each a tenth as likely to have committed the crime. You get 10 matches. So, once again we're at the 50% probability of guilt, or something in that ballpark.

I'm sure this is a somewhat different example than that given in the article but that's not the point. The point is that is there a 99.9% probability, a 0.1% probability, a 50% probability or some other probability of guilt? Or am I just trying to confuse you by throwing numbers at you?

Re:People don't understand statistics (2, Funny)

Monkeedude1212 (1560403) | more than 2 years ago | (#33307582)

Or am I just trying to confuse you by throwing numbers at you?

Bingo! You now know how Stats work in the court room!

Re:People don't understand statistics (4, Insightful)

terrymr (316118) | more than 2 years ago | (#33307974)

DNA testing probabilities go something like this ....
we found say 5 markers that match the defendant and the sample. (I picked a small number to make the example shorter)
each of those has the following probabilities of occuring in a random person :

1) 1 in 1000
2) 1 in 10
3) 1 in 10000
4) 1 in 7
5) 1 in 100

so we multiply all those together and get a probability of mismatch of : 1 in 7,000,000,000

I even told a guy at the state crimelab that was stupid - not that he cared.

Re:People don't understand statistics (3, Insightful)

mhajicek (1582795) | more than 2 years ago | (#33308736)

That of course assumes that there can be no correlation between markers.

Re:People don't understand statistics (4, Insightful)

HungryHobo (1314109) | more than 2 years ago | (#33308246)

If the only thing that pointed you at him was that search of the database then it tells you almost nothing about how likely he is to be guilty on it's own.

If you find a suspect be searching through a database of a million people with a test that has a 1 in a million chance of making a false positive and no other evidence exists then the chances of that match should not be used in any way to establish guilt in court.
But then lawyers don't care about using stats correctly.

If however you find someone, they have a knife with the victims blood on it and they have a motive and you compare their DNA to the DNA found at the scene then that same test with a 1 in a million chance of a false positive is a perfectly valid piece of data to submit in court.

Re:People don't understand statistics (1)

shadowofwind (1209890) | more than 2 years ago | (#33308740)

One problem is that DNA variation isn't uniformly distributed across all population samples. Suppose a defendant has a 1 in a million match, relative to the 'general' population. Usually, the population that had access to the victim and motive is much more closely related to the defendant than the general population. Suppose for example that both defendant and victim are members of a small ethnic minority. Do prosecutors and defense attorneys take this into account correctly?

In other arenas, the same issue applies to identity verification by fingerprint, automatic face recognition, or other means.

Everyone knows 9/5 of statistics are made up (3, Insightful)

burtosis (1124179) | more than 2 years ago | (#33307570)

This should only suprise people who think court cases are about facts and justice. It is well known that facts just get in the way of what's true and real.

Several orders of magnitude? Not quite (2, Interesting)

davidwr (791652) | more than 2 years ago | (#33307666)

When expert A says he's certain of a match to "1 in a billion" he's really saying he's certain to 0.999999999. When expert A says he's certain to 1 in a million that's certain to 0.999999.

Compare this to the "not so far apart" difference between expert A saying "he's 1 in 10" and expert B saying "he's 1 in 5." The difference between 0.9 and 0.8 certainty is a lot greater than the difference in certainty in the first example.

By the way, if I'm on a jury, I'm interested in "who else could've done it" not raw numbers. If two people leave the crime scene and blood is a "certain to 1 in 5" match to the defendant, that is, there's a 20% chance of a mistake, and the only other person who was at the crime scene has been ruled out, the only way I'll acquit is if the defendant either makes a very very strong case he didn't do it or provides some explanation for the evidence that doesn't require either of the initial suspects to be guilty.

In more practical terms, if you can raise the odds of certainty high enough that it's implausible that two people within 100 miles of the crime scene at the time of the crime are a match, and you make a very strong claim that the DNA sample is a result of the criminal being there at the time of the crime, the defense is going to have to work very hard to get me to acquit.

Re:Several orders of magnitude? Not quite (1)

BobMcD (601576) | more than 2 years ago | (#33307904)

By the way, if I'm on a jury, I'm interested in "who else could've done it" not raw numbers. If two people leave the crime scene and blood is a "certain to 1 in 5" match to the defendant, that is, there's a 20% chance of a mistake, and the only other person who was at the crime scene has been ruled out, the only way I'll acquit is if the defendant either makes a very very strong case he didn't do it or provides some explanation for the evidence that doesn't require either of the initial suspects to be guilty.

This. From TFA it seems that not only had this defendant been suspected to the point of being lawfully arrested, but he had also received oral sex from someone with genetics similar to the victim. If this were the ONLY evidence in the case, it might not meet the 'reasonable' standard. But we're told it wasn't. I'd like to see the evidence before making up my mind completely, but the point seems pretty moot. Even the low-certainty figures point that he could have done it. All he'd have to do to obfuscate the evidence is have the person who fellated him testify on his behalf. THEN we might toss the DNA, but not before.

The problem is average, mean, and variance... (1)

mr.otakhi (1882396) | more than 2 years ago | (#33307746)

The study of statistics is the failed attempt to study sample paths in a stochastic dynamic system. Systems that is are so complex or chaotic that each sample path manifest very rich or seemingly random behavior. Unable to figure out why, statisticians resort to the study of its average, means, and variances by combining may sample paths and study aggregated behavior. However, this usually won't solve any problem, because the real mechanics behind of an observed event is usually hidden in the causal relationship within each sample path. That's why medical science is progressing so slowly and unpredictably.

Re:The problem is average, mean, and variance... (2, Funny)

mhajicek (1582795) | more than 2 years ago | (#33308820)

That's why medical science is progressing so slowly and unpredictably.

On average anyway.

DNA evidence being suspect regardless (0)

Anonymous Coward | more than 2 years ago | (#33307764)

Many cases are based on something like a few hairs, or a single cigarette butt.
Have you never had some random persons hair end up on you from a store, or the bus, or work, or what have you.
How about get a cig butt stuck to the sole of your shoe.. Or have you ever seen a bum grabbing old cig butts out of a ash tray and walk off?

Lots of ways for “DNA Evidence” of yours to end up some place you never were.

Now, if you somehow manage to get semen some place you never were

Re:DNA evidence being suspect regardless (1)

mhajicek (1582795) | more than 2 years ago | (#33308854)

Now, if you somehow manage to get semen some place you never were

It means you should have washed your keyboard before disposing of it. Or more realistically, you should have incinerated your used condoms rather than throwing them in the trash. Great place to get false evidence.

And if the people are relatives? (4, Interesting)

udin (30514) | more than 2 years ago | (#33308130)

I served on a jury in which DNA evidence was presented, along with the expert witness' estimation of probability that two random people would have the same number of matching points of comparison (DNA is only matched at a relatively small number of points in the strand).

In this case, however, there were many people present at the discovery of the object from which the DNA was taken for analysis. As it happens, several of these people were relatives (brother, mother) of the person the prosecution were trying to persuade us was the person that possessed (in legal terms) the object.

The question that I kept hoping the defense attorney would ask was "what are the probabilities of an erroneous match if the people are relatives, not just two random people off the street"? Unfortunately, he didn't.

As it happened, there were so many other peculiarities in this case as well as some pretty bizarre testimony from prosecution witnesses that we voted to acquit without making much of the DNA evidence.

Re:And if the people are relatives? (1)

noidentity (188756) | more than 2 years ago | (#33308766)

I think you may have been able to have questions asked, also if you for any reason feel the guy shouldn't be held guilty, then you can ensure the guy isn't found such. You have that power.

Re:And if the people are relatives? (1)

martin-boundary (547041) | more than 2 years ago | (#33308868)

I served on a jury in which DNA evidence was presented, along with the expert witness' estimation of probability that two random people would have the same number of matching points of comparison

Me too, and I have to say that I was highly unimpressed because the context was whether a particular blood sample was either a father's or his son's. In the end, it wasn't particularly relevant to the verdict though as the son had admitted to the crime anyway.

Innocence not guilt (2, Insightful)

fermion (181285) | more than 2 years ago | (#33308200)

Lack of match of DNA found at a crime scene, like a fingerprint, provides reasonable doubt, so that suspect tends to go free. Note that it does not certainty of innocence, merely reasonable doubt. Since, in the US, we are required to prove beyond a reasonable doubt, it is often sufficient to kill a case.

DNA, like a fingerprint, should not be enough to convict. Many articles have been written on the faulty statistics that are used by prosecutors to posit faulty odds like 1 in a million, when in fact the odds are more like there are many possible people who could have done this, and we have randomly chosen one. The job is then to prove that this is not just a random choice from a database, but, based on other evidence, this is person who actually committed the crime.

This is going to become more of an issue as we get more DNA in databases and solve crimes by matching DNA to the database. In this case, the match will be a random choice between several people, and it will be a mistake to convict based primarily on DNA evidence.

Just a basic stats question (1)

Evets (629327) | more than 2 years ago | (#33308286)

I'm being purely hypothetical, but let's work out a word problem.

So let's assume that there's some DNA in question, and that dna according to all 10 other calculations (assumed accurate) has a unique match number of 1 in 100,000.

Nobody ever asked if the defendant has a twin. (Again, assuming that an identical twin would have matching DNA which I don't actually know for certain).

Let's assume for simple math that in the real world twins occur 1/100 times.

Is the statistical uniqueness now 1/1000, 1/2, 1/50,000, or 1/100,000 or some other number.

Re:Just a basic stats question (1)

Kitten Killer (766858) | more than 2 years ago | (#33308828)

Yes, an identical twin would have identical DNA.

(Generally speaking. I'm sure someone can come up with something impossibly rare about somatic mutation in the early embryo stage or something, but this is generally correct.)

This happened to me ... (5, Informative)

goodmanj (234846) | more than 2 years ago | (#33308384)

... in a case I was on the jury for. (Sorry for the bait-and-switch title, couldn't resist.)

This was a case of armed home invasion. The victim was a big bruiser of a man, a multiple convicted drug addict. The defendant was a scrawny young Cape Verdean guy. (Cape Verdean drug gangs are common in the area: this is important later.) The victim testified that, after buying drugs from the defendant, he got a series of enraged voicemails demanding the return of the defendant's cell phone. A few hours later, the defendant allegedly shows up at the victim's house with a gun and barges in yelling. A struggle ensued, a shot was fired into the floor, and the guy with the gun fled.

Evidence against the defendant included eyewitness testimony from the defendant, matching ammunition found at the defendant's house, and crucially a do-rag found at the scene of the scuffle. DNA tests matched the do-rag to a mixture of at least 3 people, including the defendant. The DNA mixing was probably due to really awful police work: a paper bag borrowed from the defendant's cupboard is not a proper evidence collection container.

As in TFA, mixed DNA dramatically affected the "probability of exclusion" statistics: the state's expert testified there was a 1 in 50 chance that a random man on the street would match the DNA on the do-rag. The odds that a random *black* man on the street would match were much higher, like 1 in 20; the defense pointed out that the odds that a random *Cape Verdean* would match would be much higher.

We've grown used DNA evidence saying things like, "not one other person on the planet could match this DNA", but in this case, the odds were good that the DNA evidence would match at least one other person sitting in the *courtroom*. The defense also took the unusual tactic of introducing the defendant's sister, who testified that her *other* brother looks very much like the defendant, and she said it was *his* voice on the enraged voicemails. What are the odds that the DNA matches the *brother* instead? Damned good.

Between the fact that the eye witness seemed shifty and unreliable and was probably on crack at the time of the incident, and the fact that all the physical evidence could just as well implicate the brother as the defendant, we couldn't rule out the possibility that the cops got the wrong guy, so we found him not guilty. If I had to take a bet, I'd say he did it, but I wouldn't bet his life on it.

Anyway. Moral of the story is: on cop shows and in the public awareness, DNA evidence is rock solid and incontrovertible. But in the real world, the statistics of DNA mixtures make things a whole lot less cut-and-dried.

Re:This happened to me ... (4, Interesting)

mhajicek (1582795) | more than 2 years ago | (#33309004)

...we couldn't rule out the possibility that the cops got the wrong guy, so we found him not guilty. If I had to take a bet, I'd say he did it, but I wouldn't bet his life on it.

I'm glad you think the way that you do. Too many people would call him guilty if they figured there was a 55% chance of it being him. Hmm, someday someone will have to draw a line in the sand as to what odds constitute reasonable doubt. If a trial could be conducted in a completely unbiased, Bayesian way and a probability of guilt were established, what number would be needed to convict?

Easy to interpret when it DOESN'T match (1)

SoftwareArtist (1472499) | more than 2 years ago | (#33308442)

DNA evidence is a far more useful tool for defense than prosecution. Showing that a DNA sample really came from a certain person is difficult, and can never be 100% accurate. But showing it didn't come from someone is easy. If it obviously doesn't match, that's that. There's no question at all in how to interpret it.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...