Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Cheaters Exposed Analyzing Statistical Anomalies

CmdrTaco posted more than 3 years ago | from the actually-they-do-prosper dept.

Education 437

Hugh Pickens writes "Proctors and teachers can't watch everyone while they take tests — not when some students can text with their phones in their pockets, so with tests increasingly important in education — used to determine graduation, graduate school admission, and — the latest — merit pay and tenure for teachers, Trip Gabriel writes that schools are turning to 'data forensics' to catch cheaters, searching for data anomalies where the chances of random agreement are astronomical. In addition to looking for copying, statisticians hunt for illogical patterns, like test-takers who did better on harder questions than easy ones, a sign of advance knowledge of part of a test or look for unusually large score gains from a previous test by a student or class. Since Caveon Test Security, whose clients have included the College Board, the Law School Admission Council, and more than a dozen states and big city school districts, began working for the state of Mississippi in 2006, cheating has declined about 70 percent, says James Mason, director of the State Department of Education's Office of Student Assessment. 'People know that if you cheat there is an extremely high chance you're going to get caught,' says Mason."

cancel ×

437 comments

Sorry! There are no comments related to the filter you selected.

Headline misleading (3, Insightful)

fridaynightsmoke (1589903) | more than 3 years ago | (#34699758)

The headline should be "Cheaters Exposed By Analyzing Statistical Anomalies"? I thought the cheaters themselves were doing the analyzing, to get ahead of the cheat detection.

Re:Headline misleading (1)

Ynot_82 (1023749) | more than 3 years ago | (#34699818)

Depends,
If they did poorly on the previous "Analyzing Statistical Anomalies" test...

Re:Headline misleading (1)

Thanshin (1188877) | more than 3 years ago | (#34699968)

The headline should be "Cheaters Exposed By Analyzing Statistical Anomalies"? I thought the cheaters themselves were doing the analyzing, to get ahead of the cheat detection.

I thought a new branch of Statictical Anomaly Analysis had managed to replicate the effect of airport porn-o-matics.

Re:Headline misleading (0)

poetmatt (793785) | more than 3 years ago | (#34700140)

its a question of who is more effective and how.

if people need to cheat to beat a test the question is, what is causing this? Is it the students, the teacher?

to simply go after cheaters is putting a band-aid over the real problem.

Re:Headline misleading (-1)

Anonymous Coward | more than 3 years ago | (#34700360)

Mod parent off-topic. His post has nothing to do with the one he's replying to.

This doesn't prove anything (4, Insightful)

Anonymous Coward | more than 3 years ago | (#34699798)

If I fall into the anomaly category without cheating, I'll be screwed. What can I demonstrate in my defense? Not much. I find it hard to believe they can prove that you cheated without actually video-taping you cheating or something along those lines.

Anomalies are what they are, data anomalies, nothing more and nothing less.

Re:This doesn't prove anything (1)

ceejayoz (567949) | more than 3 years ago | (#34699894)

It'd be possible to do in a sensible manner. If half the class falls in the "statistical abnormality" category, and they have the same or similar abnormality, there are some pretty good conclusions you can draw. The same for someone who consistently shows the same abnormality across multiple tests.

The response might not be to fail someone immediately, either. It might be to watch them more closely on future tests, or to swap out the suspected compromised test at the last minute without anyone but the professor knowing it'll happen.

Re:This doesn't prove anything (4, Insightful)

Shadow of Eternity (795165) | more than 3 years ago | (#34700372)

In a sane and rational world, yes. In a school? The instant the business flags someone the school is going to treat that as the voice of god himself coming down from the heavens to demand blood and expulsion.

That's not even getting into the blindingly obvious conflict of interest here. Just like turnitin these guys are a business that relies on there always being cheaters to catch, it's in their interest to produce as many "catches" as possible without losing credibility. Hell at least with turnitin you're dealing with something you can prove, with these guys the student literally has no defense.

It's a witch hunt, pure and simple. If they're not failing they must be cheating somehow, if they are failing obviously they're innocent. No matter what the student is fucked.

Re:This doesn't prove anything (4, Informative)

Desler (1608317) | more than 3 years ago | (#34700032)

Yes, the anomalies in and of themselves do not prove anything hence why the article says:

When the anomalies are highly unlikely -- their random occurrence, for example, is greater than one in one million -- Caveon flags the tests for further investigation by school administrators.

Re:This doesn't prove anything (0)

Anonymous Coward | more than 3 years ago | (#34700148)

And we all know school administrators will only use that to decide whom to look more closer at and not use the flag as reason to punish someone. (Sorry if you need a new keyboard now).

A chance of about one in a million means that if you have one million pupils usually one will be flaged without any reason.

captcha: multiply

Re:This doesn't prove anything (3, Funny)

kevinNCSU (1531307) | more than 3 years ago | (#34700278)

A chance of about one in a million means that if you have one million pupils usually one will be flaged without any reason.

Luckily no single school has either over a million pupils or a deep understanding of statistics so we're safe ;)

Re:This doesn't prove anything (0)

Anonymous Coward | more than 3 years ago | (#34700316)

And what does that further investigation mean? Would you actually be surprised to find out that more often than not it actually means punishment?

I was 16 when I went to college and when I took my first midterm for chemistry it took me 10 minutes. Since an hour and a half had been allotted, and not having any other experience with midterms, I thought the written part I just finished was only part of the test. After the teacher noticed I was still sitting there after turning my test in she told me I could leave. The next fastest person took close to the whole time, and some actually needed more than the 90 minutes. At that point I was scared and thought it seemed too easy, perhaps I should have spent more time? Even the essay questions seemed easy. I was certainly an anomaly in terms of time, and having the second highest grade in the class could have looked like cheating. Fortunately the teacher was also my guidance counselor (or whatever they were called back then, being almost 20 years ago I don't recall exactly) talked to me and understood that I had an extremely good recall of anything I had ever read or heard (certainly better than today; aging stinks) and knew from talking to me that I was very interested in the class. Had she been a little less attentive, things could have easily gone differently. I've had other teachers assume I must be cheating because I didn't do daily homework but usually had the highest grade in class, frequently 100%. But when I corrected them, and proved that I was right, while perhaps not endearing myself to many of them, they backed off.

Re:This doesn't prove anything (4, Insightful)

0123456 (636235) | more than 3 years ago | (#34700084)

If I fall into the anomaly category without cheating, I'll be screwed. What can I demonstrate in my defense? Not much.

But this is good. Kids should learn that in the real world they'll be arbitrarily punished for doing well merely to further the career of the person they're working for.

Re:This doesn't prove anything (1)

mysidia (191772) | more than 3 years ago | (#34700100)

The "anomaly" doesn't guarantee cheating; but it tells them where to look. Kind of like a spam filter tells you what mail is probably spam, by sending it to a Spam folder.

They could assign everyone a few "extra" personalized tests every term and taylor the personalized test based on the statistical information. With suspected cheaters receiving some questions designed to expose them.

If it is rare enough that a non-cheater is identified as a cheater, for example

Once an anomaly has been found, they can list anomalies as people to scrutinize more closely on the next test. Especially if they show up as anomalous on more than one test.

They can start comparing answers to people that were near the anomalous tester for similarities, after detecting the statistical anomaly.

Re:This doesn't prove anything (1)

abelenky17 (548645) | more than 3 years ago | (#34700142)

What in the world could possibly cause "unusually large score gains from a previous test"? Studying!
  I know that after having failed a test, I've often buckled down, reviewed all the material from the beginning and aced the subsequent test.
Yet according to Caveon Test Security, I'd be a cheater.

I also take issue with "searching for data anomalies where the chances of random agreement are astronomical". On one recent test, a fellow student and I missed nearly exactly the same questions, because we were taught by the same TA.
We were both excellent students, with the same gaps in our knowledge, and the same mis-information on a few topics. We approached problems the same way, which led us to the same errors, exactly as we were taught.

The point is: Caveon's analysis is worthless. It apparently provides NO means to differentiate cheating from completely reasonable non-cheating explanations. All it can do is point out anomalies.

But if these tests are "used to determine graduation, graduate school admission and, the latest, merit pay and tenure for teachers", it is completely improper (and perhaps illegal?) to deny someone graduation, admission, pay, or tenure without PROOF, on the basis of an anomaly.
Really, these guys are peddling Snake Oil. Perhaps the TSA is buying?

Re:This doesn't prove anything (3, Informative)

Surt (22457) | more than 3 years ago | (#34700218)

Yet according to Caveon Test Security, I'd be a cheater.

According to the article, according to Caveon Test Security, you might be a cheater.

So you'd be investigated. And you'd pass investigation, because you didn't cheat.

Re:This doesn't prove anything (1)

JonySuede (1908576) | more than 3 years ago | (#34700440)

So you'd be investigated. And you'd pass investigation, because you didn't cheat.

No, that is false. There are innocent that get convicted in a criminal court, that have more rigorous procedure than an academic tribunal. So you would then have a chance to be wrongly convicted

Re:This doesn't prove anything (1)

Monkeedude1212 (1560403) | more than 3 years ago | (#34700302)

Once, before an Elementary School exam, a friend of mine said

"If you aren't cheating, you aren't trying"

He then went on to get 98% on that exam. I have always wondered to this day how much of that grade he earned.

Re:This doesn't prove anything (1)

Spazmania (174582) | more than 3 years ago | (#34700334)

Worse than that, actually. The methodology assumes that one wrong multiple-guess answer is as likely as any other wrong multiple-guess answer. But that isn't the case. Wrong answers are often chosen specifically because a particular and expected form of mistake will yield that answer. This skews the results to a point where the real chance of two people in a class getting exactly the same wrong answers could be as little as 1 in 100... and when you give test after test, it doesn't take long for 1 in 100 odds to produce a "winner" by pure random luck.

Re:This doesn't prove anything (2)

Chas (5144) | more than 3 years ago | (#34700348)

If I fall into the anomaly category without cheating, I'll be screwed. What can I demonstrate in my defense? Not much. I find it hard to believe they can prove that you cheated without actually video-taping you cheating or something along those lines.

Anomalies are what they are, data anomalies, nothing more and nothing less.

This to the Nth power.

It also fails to take into account a student's study patterns and general aptitudes.

As such "harder" and "easier" questions are a matter of perspective.

Again, lies, damn lies...and you know the rest.

Re:This doesn't prove anything (1)

HotBits (1390689) | more than 3 years ago | (#34700378)

So if I really study the hard problems and get them all right, but miss a couple easy ones, I'm a cheater? Sounds like a lot of false positives are possible if you dig too deep.

Re:This doesn't prove anything (1)

RapmasterT (787426) | more than 3 years ago | (#34700442)

Anomalies are what they are, data anomalies, nothing more and nothing less.

Not true, anomalies are 100% certain proof of whatever it is you've decided ahead of time that you're looking for. Just ask: 9/11 Truthers, Anti-Obama Birthers, Ghost hunters, Anti-evolutionists, anti-vaccinationists, etc etc etc.

People doing statistical analysis who don't understand that a standard distribution GUARANTEES a percentage of anomalous results, frankly don't have any business having a job in their field. The fact that they'll label people as cheaters when they should know that x% will be labeled that way falsely is grounds for a pretty awesome lawsuit.

And then what? (1)

Anonymous Coward | more than 3 years ago | (#34699820)

Improbable is not impossible. Are they going to boot people for a good test if they think it was the result of cheating or will they still need to catch them in the act (or find irrefutable evidence)?

Re:And then what? (1)

vtcodger (957785) | more than 3 years ago | (#34700036)

***cheating has declined about 70 percent***

Right. How in the world did they determine that? Most likely, they just made up a number.

All in all, I'd say the odds favor these guys being a bunch of card-carrying charlatans who couldn't catch a cheater if he had a "C" branded on his forehead. I'll bet that their road show has some nifty Power Point slides though.

Re:And then what? (1)

SamuraiHoedown (1769404) | more than 3 years ago | (#34700116)

93% of all statistics are made up on the spot, 77% of all people know this.

Re:And then what? (1)

jimmetry (1801872) | more than 3 years ago | (#34700438)

They did say since the program started in 2006... so that implies if they caught 100 people in its first year they caught 30 in its fourth. However, I'm sure the algorithms have changed over time to reduce false positives and now the students know what they're up against so the cheating methods will change as well. The statistic probably isn't made up, but it's completely unreliable.

Re:And then what? (1)

Desler (1608317) | more than 3 years ago | (#34700064)

When the anomalies are highly unlikely -- their random occurrence, for example, is greater than one in one million -- Caveon flags the tests for further investigation by school administrators.

I know, I know, you didn't bother to RTFA.

Re:And then what? (0)

Anonymous Coward | more than 3 years ago | (#34700118)

How the hell is that going to help? They just analyzed the test and came to the conclusion that it probably was the result of cheating, and that is not enough. What are they going to do? Waterboard the suspects?

Re:And then what? (2)

The MAZZTer (911996) | more than 3 years ago | (#34700092)

When the anomalies are highly unlikely -- their random occurrence, for example, is greater than one in one million -- Caveon flags the tests for further investigation by school administrators.

It does sound like they are acting responsibly and merely saying such marked tests need further scrutiny by the school, and only marking tests with a really low probability to be just lucky... but who knows what the individual schools do.

education system fail (0)

Anonymous Coward | more than 3 years ago | (#34699854)

multiple choice tests at any level are completely divorced from the concept of real world knowledge application

all tests should be scrapped and replaced with double blind judged projects

but that might make the poor teachers union members work a little bit instead of using the same multiple choice test for 20 years and relying on a computer to spit out the grades

Sooo... (5, Interesting)

redemtionboy (890616) | more than 3 years ago | (#34699858)

...the lesson here is to cheat just barely enough to get by, or to consistently cheat all the way through. A better lock just makes a better lock picker. Not that I'm saying we shouldn't discourage cheating, but the issue is why students want to cheat rather than gain the knowledge. I understand that testing is one of the best ways to gauge knowledge, but we're too focused on testing and no where near focused enough on education. Tests are proven to decrease a students interest in subjects. The solution is to move to more objective based learning where students complete projects or applications showing their knowledge of the material. Obviously testing is easier, but if our goal is education, we need to change.

Re:Sooo... (2)

Shados (741919) | more than 3 years ago | (#34700000)

the issue is that the gain from high grades, regardless of how they're obtained, is far too high. So many companies won't hire anyone under 3.0 GPA, colleges won't accept you if you're not good enough in highschool, etc.

For people that are borderline, cheating on one test in one class could be the difference between their dream job and, in certain fields in certain areas of certain countries, no job. I fortunately don't live in such a place, but I know people who do. Make college acceptance process smarter and more personalized instead of the streamlined garbage it is now, and educate employers. Then students will be able to cheat all they want, it won't matter.

Re:Sooo... (1)

Antisyzygy (1495469) | more than 3 years ago | (#34700098)

Grad schools wont accept you without a 3.0 undergrad and many Doctoral programs wont accept you without a 3.25.

Re:Sooo... (2)

Surt (22457) | more than 3 years ago | (#34700268)

That's usually only true if you're a raw grad with no contacts. If you REALLY want to go to grad school, but didn't have the grades, go get a job near/at a university, and volunteer part time for the lab you want to go to grad school in. You'll be accepted the next year, assuming you've proven yourself interesting.

Re:Sooo... (1)

Antisyzygy (1495469) | more than 3 years ago | (#34700420)

Makes sense. I "BARELY" made it in 3 years ago with a 3.06 . Then I got my head out of my ass and started studying harder.

Re:Sooo... (1)

redemtionboy (890616) | more than 3 years ago | (#34700184)

I'm fine with jobs going off of GPA as an initial barrier, why? Because sometimes for every job posting they'll get upwards of 1000 applications. You have to have some sort of factor to buffer out candidates or else it becomes extremely cost ineffective. Until we reach a much higher level of AI that can sufficiently grasp things other than GPA and keywords, we're stuck using such a system. So if you can't change GPA as the barrier, why don't you change what GPA stands for. Right now all it stands for is how well you test. We need to change the system so it better reflects knowledge, understanding, and capabilities of a student. You can cheat on a test, but you can't cheat on application of that knowledge.

Re:Sooo... (1)

kevinNCSU (1531307) | more than 3 years ago | (#34700358)

A better lock just makes a better lock picker.

I have a tall fence I'd like to sell to you which will make you a better jumper.

Another application (2)

countertrolling (1585477) | more than 3 years ago | (#34699866)

I would like to apply this idea to see when a politician is lying. But then I realized it would just overload. So then I figure we should try to see if it can detect when they are telling the truth. That way you work with a much smaller data set. Damn near zero. So it looks to be a total failure.. Nevermaind

Oh that's easy... (1)

Greyfox (87712) | more than 3 years ago | (#34700352)

boolean politicianIsLying() {
return true;
}

Wouldn't this require specially designed tests? (5, Interesting)

khasim (1285) | more than 3 years ago | (#34699876)

Gotta love this line:

David Foster, the chief executive of Caveon, said the company had not published its methods because it was too busy serving clients. But the company's chief statistician is available to explain Caveon's algorithms to any client who is curious.

Interesting. So their people have time to explain the methods to non-peers ... but not enough time to write them up for peer review.

Re:Wouldn't this require specially designed tests? (-1)

Anonymous Coward | more than 3 years ago | (#34700054)

Gotta love this line:

David Foster, the chief executive of Caveon, said the company had not published its methods because it was too busy serving clients. But the company's chief statistician is available to explain Caveon's algorithms to any client who is curious.

Interesting. So their people have time to explain the methods to non-peers ... but not enough time to write them up for peer review.

Reading comprehension FAIL. Caveon is too busy doing work to freely publish their methods, as you say for peer review. However, if you pay them (thus becoming a client), they have someone available to explain it to you.

Re:Wouldn't this require specially designed tests? (4, Informative)

StuartHankins (1020819) | more than 3 years ago | (#34700266)

If it doesn't stand up to peer review, or they don't bother to submit it to peer review, then this is likely snake oil. We (the proposed paying client) are not necessarily peers qualified to review their algorithms, but the peer community is. That is how peer review works.

Are you trying to be disingenuous, or did you really not understand the GP's point? Reading comprehension indeed.

I have an idea to stop using cells for cheating (0)

SirGeek (120712) | more than 3 years ago | (#34699884)

Have a little case with enough slots for desks. Each student puts their cellphone into a box corresponding to their desk.

No Cellphones makes it MUCH simpler to not have them used for cheating.

And the rule is - If you don't turn your cellphone in - YOU FAIL. Done.

Re:I have an idea to stop using cells for cheating (2)

redemtionboy (890616) | more than 3 years ago | (#34699908)

so, what happens if the student doesn't have a cellphone, or has two, or is borrowing one from a friend?

Re:I have an idea to stop using cells for cheating (1)

0123456 (636235) | more than 3 years ago | (#34700022)

so, what happens if the student doesn't have a cellphone, or has two, or is borrowing one from a friend?

Clearly all students should have to take the test naked to ensure they don't have a hidden cellphone.

Re:I have an idea to stop using cells for cheating (2)

kybred (795293) | more than 3 years ago | (#34700128)

so, what happens if the student doesn't have a cellphone, or has two, or is borrowing one from a friend?

Clearly all students should have to take the test naked to ensure they don't have a hidden cellphone.

Can I volunteer to be a proctor at some all female university?

Re:I have an idea to stop using cells for cheating (1)

omnichad (1198475) | more than 3 years ago | (#34699920)

Great - so for $30, I can go to the store and buy a prepay cell phone. I can cheat for life for only $30! And that's for a brand new fake phone. I could get a used one for next to nothing.

Re:I have an idea to stop using cells for cheating (1)

Krazy Kanuck (1612777) | more than 3 years ago | (#34700016)

Easier to just block the signals in class rooms, or provide special testing rooms with said technology. Probably a lot cheaper than an analytical service.

http://www.thesignaljammer.com/products/Prison-Cell-Phone-Jammer-62W.html [thesignaljammer.com]
"Our most popular prison cell phone jammer, and also a favorite for learning institutions."

Re:I have an idea to stop using cells for cheating (0)

Anonymous Coward | more than 3 years ago | (#34700216)

The owner of the Subway restaurant in my town bought a jammer to block phones in his restaurant, not realizing that radio waves travel (kind of the point, really). He took out the police department, the hospital, and every cellphone in town for the day and a half it took to track him down.

Re:I have an idea to stop using cells for cheating (1)

clone52431 (1805862) | more than 3 years ago | (#34700074)

What if someone doesn’t have a cellphone?

What if someone has two?

Re:I have an idea to stop using cells for cheating (1)

oodaloop (1229816) | more than 3 years ago | (#34700104)

Right, here's my cell phone. And I can still text on my iPod Touch using nothing more than bluetooth messaging. Or another cell phone. Or passing notes. But other than that, it's genius.

Re:I have an idea to stop using cells for cheating (1)

Wooky_linuxer (685371) | more than 3 years ago | (#34700188)

At least down here in Brasil people are already obliged to leave their cellphones during the tests, just common sense really. Also, any eletronic device like PMPs, earpieces, even wristwatches.

The real problem with statistical approaches is that they don't prove one has cheated. So unless it is only used as a way to start an investigation to gather proofs - such as, contacting the cellphone operator and seeing if there was any calls to/from the cellphone during the test, AND if they can also prove the cellphone was in the test area - there is no legal way someone should be punished for being a point outside the line.

Re:I have an idea to stop using cells for cheating (1)

The MAZZTer (911996) | more than 3 years ago | (#34700190)

So if I DON'T bring my cellphone to class, I fail? And the cheater next to me brings in two cellphones and only turns in one and uses the other one to cheat doesn't?

Not to mention a cellphone is a valuable thing which you don't want to just set down and walk away from. Even a classroom full of people would be hard-pressed to notice if a student doesn't bring in a phone but picks up "his" phone after the test, which happens to be an expensive model, and walks out of the classroom with it. The more people in the class, the easier this would be to pull off.

Math students are screwed! (2)

ArhcAngel (247594) | more than 3 years ago | (#34699916)

Teacher: I'm sorry I am going to have to fail most of you for cheating.
Students: But we didn't cheat!
Teacher: Then how do you explain how so many of you came to the same conclusion that 2+2=4?

Re:Math students are screwed! (1)

TheL0ser (1955440) | more than 3 years ago | (#34700162)

But the Party tells me that 2+2=5. So failing them all for getting the wrong answer makes sense.

Re:Math students are screwed! (2)

Gunnut1124 (961311) | more than 3 years ago | (#34700234)

THERE. ARE. FOUR. LIGHTS!

Oh really? (1)

selven (1556643) | more than 3 years ago | (#34699926)

Since Caveon Test Security, whose clients have included the College Board, the Law School Admission Council and more than a dozen states and big city school districts, began working for the state of Mississippi in 2006, cheating has declined about 70 percent

Or cheaters have become 70% less detectable.

Re:Oh really? (1)

khallow (566160) | more than 3 years ago | (#34700340)

Or cheaters have become 70% less detectable.

Given how dumb a large portion of the cheater population is, I'd say that there's probably a big decline in actual acts of cheating. It might not be 70% since as you point out, the smarter cheaters might have adapted to the detection mechanisms, but I see three groups of cheaters. The first is too ignorant or dumb to pass the test. They're not going to be able to master methods that require them to foil statistical checks. The second is lazy. They cheat because it's less work than studying. Throwing in additional checks that take considerable work to crack changes the effort balance. My take is that most of them aren't going to bother with sophisticated cheating when it becomes more work than studying. Then there's the third group which cheats because that is a challenge. They will no doubt figure out this.

Clarification (1)

Stenchwarrior (1335051) | more than 3 years ago | (#34699930)

"cheating has declined about 70 percent"

You mean, people caught cheating has declined 70 percent.

False positives? (2)

Abstrackt (609015) | more than 3 years ago | (#34699964)

From TFA:

"Your goal is not to catch a bunch of people and hang them," Dr. Fremer said. "Your goal is to have fair and valid testing."

I hope administration agrees. When I was in university I wrote a group paper with one guy whose wife was a professional editor, she helped us out by reviewing it and making suggestions, we had to fight not to get expelled because our paper was "too well written" to be our own work.

Re:False positives? (1, Interesting)

Frosty Piss (770223) | more than 3 years ago | (#34700080)

When I was in university I wrote a group paper with one guy whose wife was a professional editor, she helped us out by reviewing it and making suggestions, we had to fight not to get expelled because our paper was "too well written" to be our own work.

You had it proof read and edited by a professional. You did in fact "cheat", the work was not completely yours. This is essentially the same as buying your term paper on-line. You can rationalize it all you want, but the bottom line is your professor expected the work to be yours, not that of a "professional editor". I assume you went on to get an MBA?

Re:False positives? (1)

clone52431 (1805862) | more than 3 years ago | (#34700130)

When I was in university I wrote a group paper with one guy whose wife was a professional editor, she helped us out by reviewing it and making suggestions

Unless you specifically asked the prof and got permission to do that, you should generally assume that even if a paper is given as a “group” paper, it should still only involve the collaboration of people who are actually in that class, and in that group.

Re:False positives? (1)

geekoid (135745) | more than 3 years ago | (#34700374)

You did cheat. you had a professional editor review it.

Re:False positives? (0)

Anonymous Coward | more than 3 years ago | (#34700388)

Our paper was "too well written" to be our own work.

My father is a professional writer and editor and I have fallen into a similar trap. Thank goodness I kept his red pen markups and that Word keeps track of editing time and I saved multiple versions at different stages of being written and I had backups on the schools server made every day, the direct quote search on all drafts ended up negative and the style matched my previous work at the school. However, each of those alone was not enough. Finally, my friend, who was one of the student members of the ethics review board said, "This is ridiculous. Do you think he copied professional work, then changed it to a crappier version, multiple times; while he faked the rest of this? It would take less effort to write the paper!"

He quit the ethics review board. He got sick of the politics and stupid outcomes. They were ready to kick me out because they were convinced that I cheated and was covering it up. But other confirmed cheaters had to only write 10 page papers about why cheating was bad, and not even have to redo the original work. All because, the amount of donations to the school by the students family and friends was also a significant factor to what happened. My favorite being the person who showed up high to class and provided a 20 minute distraction (and whose father donated hundred of thousands of dollars to the school) wasn't even required to required to apologize to the instructor.

False Positives (4, Interesting)

Dartz-IRL (1640117) | more than 3 years ago | (#34699970)

And yet, there will also be false positives.

I did poorly on one test. Noticing this, I studied hard and greatly improved my grade in the next test. Would this flag up a warning that I'm a cheater?

Or for that matter, doing better on the 'harder' questions. Perhaps I decided to concentrate on doing those questions because they offered higher marks than the easier questions, or because I had a natural aptitude for some elements. I may have elected to study those materials harder.

Professors can't rely solely on 'statistical anomalies'. Illogical patterns may well have an explanation that has nothing at all to do with cheating or advanced knowledge of the test. Of course, we all know just how lazy a minority of our lecturers are.... and how likely they'd be to take the word of this agency as gospel.

Re:False Positives (2)

Desler (1608317) | more than 3 years ago | (#34700082)

That is why they don't automatically assume you are a cheater:

When the anomalies are highly unlikely -- their random occurrence, for example, is greater than one in one million -- Caveon flags the tests for further investigation by school administrators.

You get flagged and they do a further investigation.

Re:False Positives (4, Insightful)

Daniel Dvorkin (106857) | more than 3 years ago | (#34700422)

That is why they don't automatically assume you are a cheater ... You get flagged and they do a further investigation.

Being accused of cheating in the academic world is kind of like being accused of a sex crime in the world at large -- the burden of proof is essentially on you to prove your innocence no matter what the law says, it's very difficult to prove you didn't do it, there are people who will go to insane lengths to get you convicted, and even if you're cleared the damage to your reputation is done.

Re:False Positives (1)

flyingfsck (986395) | more than 3 years ago | (#34700166)

Exactly, I'm afraid this will detect students who 'spot' study. The ability to figure out what is important and only study that, and ignore the fluff, will be penalized.

Re:False Positives (1)

SteelKidney (1964470) | more than 3 years ago | (#34700284)

My ACT standardized test would have been flagged as the result of cheating. In the math part, my score got progressively higher as the topics got harder. The reason was pretty simple- it had been a couple of years since my algebra classes, and I was a bit rusty on some concepts. however, I was in the middle of studying for my calculus final, and was pretty damn sure of myself on those questions.

Re:False Positives (1)

JoshuaZ (1134087) | more than 3 years ago | (#34700368)

I did poorly on one test. Noticing this, I studied hard and greatly improved my grade in the next test. Would this flag up a warning that I'm a cheater?

That's a flag that is already used by teachers. That isn't the sort of more sophisticated statistical technique. I'm a calculus TA now, and I had a student last semester who got around a 75 on the first midterm and got a 100 on the next midterm, and a score in the high 90s on the last midterm. I'm pretty sure she just studied really hard (and it helps that she's fairly bright). But it is hard to tell in general what is happening. When one has 100 students in a class the probability is high that some will exhibit weird patterns by sheer chance. Unfortunately, with 100 students, it is also extremely likely that some of them will be cheating.

The thing that concerns me most about the stated techniques is looking at students who do well on the "harder questions" than on the easy ones. This happens all the time for innocent reasons, such as students budgeting more time to studying the harder sections of the material. Also, easier material is often covered early in a course and then not discussed in much detail later. On the final my students had this semester one thing they had to do was graph a line tangent to a specific curve. A lot of them did very poorly on that. I don't think cheating had anything to do with that. It much more likely had to do with the fact that students hadn't done it many times since the first midterm.

It's science, it CAN'T be wrong! (1)

eatblueshell (1702842) | more than 3 years ago | (#34699976)

Don't refute mathematical truth! Though I would imagine what it would do would identify potential cheaters, which would then be monitored later. Because, as stated, it would be unreasonable to kick someone out based on a probability of dishonesty. That said, I never saw the benefit of cheating, considering what good does an education do, if you never learned anything? Other than how to cheat of course.

What do tests have to do with education? (0)

Anonymous Coward | more than 3 years ago | (#34700008)

Nothing.

They are for discrimination. i.e. to choose one person rather than another. That has nothing to do with educating people.

 

Confirmation Bias? (1)

jambarama (784670) | more than 3 years ago | (#34700066)

Since Caveon Test Security . . . began working for the state of Mississippi in 2006, cheating has declined about 70 percent.

How can anyone possibly know this? If they're detecting 70% less cheating, how do they know it is because students are cheating less rather than cheating in less detectable ways? The company methods aren't published or peer reviewed, and the graphs on the website are post-hoc graphs from excel (rather than whatever they use to data crunch - R, SAS, etc). As pointed out in the article, the company says nothing of type 1 (false positive) or type 2 (false negative) error rates. If students study together and have similar answers, particularly if the test is open book, how likely are they to be flagged as cheaters?

I'd love a silver bullet to stop cheating, but I'd like it to be something we can all agree is a good check. More fundamentally, multiple choice tests are a lousy way to test anything but recall - they're just not capable of testing real learning: comprehension, application, analysis, synthesis, or evaluation. Maybe we ought to move towards other types of tests, less vulnerable to hidden cheating.

Re:Confirmation Bias? (1)

geekoid (135745) | more than 3 years ago | (#34700406)

If cheating drops that fast, it's a fair assumption. Cheaters getting craftier will happen, but not that fast.

The real problem is such a short time span to collect any data to actually come to this conclusion.

cheating is as old as tests (1)

metageek (466836) | more than 3 years ago | (#34700068)

Cheating has always been around, I doubt that cell phones are making it more frequent. Grades based on testing are themselves very unreliable too. Should they associate p values to the test scores?

Hmm, I am atypical (0)

Anonymous Coward | more than 3 years ago | (#34700090)

I usually perform much better on hard questions than on simple ones. In one extreme case in multiple choice test I had all easy questions wrong, half of the medium difficulty questions right and all the hard ones with no error. Does it mean I cheated? Does my brain cheat itself?

And so (1)

Dunbal (464142) | more than 3 years ago | (#34700094)

The "dumbing-down" of America continues. Because it's much easier to turn the wrath of the "system" against anyone who stands out rather than actually following the steps involved to prove a person's guilt. Publicly flogging an innocent person is just as effective a deterrent as flogging a guilty one. It does, however, speak volumes about how those entrusted with authority view their powers.

While statistics may be absolutely certain about what the odds are over the long term of getting any particular number on the roulette wheel, it absolutely cannot predict the next spin. Using "statistical analysis" to "catch someone" is absolute, utter bullshit and any faculty using this should be run through a statistics course and then fired. It proves nothing. Get up off of your fat arses and do your damned job. You can tell in under 5 minutes which students have studied and which haven't, just by talking to them, and this information is far more valuable than any statistical snake oil.

Re:And so (1)

geekoid (135745) | more than 3 years ago | (#34700432)

I wish people like you would think...or at least read the article.

Statically anomalies are red flagged for further investigation.

Just like if the same number came up 3 times in a row on a roulette wheel the pit boss, and the eye, will start paying the wheel little more attention.

Pretty Impressive (2)

Fnord666 (889225) | more than 3 years ago | (#34700122)

With more than 100,000 students tested, proctors could not watch everyone -- not when some teenagers can text with their phones in their pockets.

And how exactly did they read those text messages if their phone was in their pocket?

Re:Pretty Impressive (1)

gotpoetry (1185519) | more than 3 years ago | (#34700410)

They text the questions to someone else while taking the test. That person who will take the test later then knows what the questions are.

Re:Pretty Impressive (1)

KeithJM (1024071) | more than 3 years ago | (#34700434)

And how exactly did they read those text messages if their phone was in their pocket?

If a student in the first class of the day texts the questions and/or answers to a student in a later class, they could read the messages at their leisure.

Makes no sense (0)

Anonymous Coward | more than 3 years ago | (#34700126)

As an engineering student I had many false positives in many classes. There were times when I had to make an A on the final to get a C in a class and I did. This method of catching cheaters will only add to the stress a student is under. I was accused of cheating because I made a perfect score on a final and I had to go through a whole review process to prove my innocence. The process was definitely guilty unless proven otherwise. While a cheater might make perfect scores on all his test because he was cheating from the start.

Deterrence theory vs retribution theory (1)

GPLDAN (732269) | more than 3 years ago | (#34700154)

Played out about as purely as it could be. Here again, as in criminal law, we see that deterrence is always a better choice than retribution. It's why the death penalty doesn't persuade anybody not to commit a capital crime. It's retributive. It says nothing about the probability of being caught to begin with, so it does not change the murder rate.

How do you cheat with your phone in your pocket? (1)

dFaust (546790) | more than 3 years ago | (#34700156)

Maybe I'm a bit slow on this, but I'm still trying to figure out how one cheats with their phone in their pocket. I get how on some phones you can send a text with your phone in your pocket, but how exactly do you receive an answer with the phone in your pocket? Does the person aiding you send you back 1 text for answer A, 2 for B, etc. and you count the number of times your phone vibrates? Of course in a smaller room, I'd imagine some people would be able to hear the vibrating and ask you to give up your phone.

Testing important to education? no. (1)

drb226 (1938360) | more than 3 years ago | (#34700198)

so with tests increasingly important in our current (broken) educational system — used to determine graduation, graduate school admission and, the latest, merit pay and tenure for teachers, Trip Gabriel writes

Tests are not important to education. They contribute little to the actual process of learning. They are simply the (very rudimentary) measuring stick to see how "tall" you are in the learnosphere. Sadly, measuring sticks only measure one dimension. Almost all fields of study have many, many more dimensions that bubble sheet/essay testing cannot measure.

let's hope they are right (1)

roman_mir (125474) | more than 3 years ago | (#34700210)

Let's hope they are correct in their assessment and it's cheating that declined 70% (by the way, why not more than 70%?) as opposed to something else happening - like people cheating 70% MORE during all other times, not just during exams to throw off the statistics during the exams.

Don't confess (0)

Anonymous Coward | more than 3 years ago | (#34700224)

I often have students who do worse than they should based on random chance. ie. Each question has five choices, a student who chooses randomly should get four questions right on a twenty question test. Scores of 2/20 are common.

Given that students often do worse than random chance, there must also be an equal number of students who do better than random chance. In fact, there is a finite chance that a student will get a perfect mark based on random chance.

The bottom line is that, if they accuse you of cheating based only on statistics, they have no real proof. They have to hope that they can get you to confess.

My solution is to give students with suspicious results a chance to take another test in a room by themselves with a proctor or video surveillance.

proactive masking? (1)

circletimessquare (444983) | more than 3 years ago | (#34700228)

you can proactively mask your cheats with statistically valid test answers, right and wrong. thus, the cheats won't be caught by the test analysis software, it will be thrown off the scent

however, if you can actually master this methodology, and the test you are cheating on is a test in a college level statistical analysis class, perhaps you deserve the A nonetheless

Re:proactive masking? (1)

clone52431 (1805862) | more than 3 years ago | (#34700240)

however, if you can actually master this methodology, and the test you are cheating on is a test in a college level statistical analysis class, perhaps you deserve the A nonetheless

If you can actually pull that off in a college-level statistical analysis class, couldn’t you have probably passed it without cheating in the first place?

Re:proactive masking? (1)

circletimessquare (444983) | more than 3 years ago | (#34700318)

it's called humor

Re:proactive masking? (1)

clone52431 (1805862) | more than 3 years ago | (#34700424)

Well, I didn’t think it was terribly funny, but I’d grant you that it was slightly thought-provoking.

It Better Be Good (2)

b4upoo (166390) | more than 3 years ago | (#34700254)

Some emotional disorders can cause lapses in concentration in which complex questions are solved and easy questions answered incorrectly. I would think that any accusation or action in regard to cheating best have one heck of a strong proof or the lawyers will have a feeding frenzy.

These systems need reasonable human oversight (0)

Anonymous Coward | more than 3 years ago | (#34700258)

In my time at Georgia Tech I was accused of cheating on a physics test by some similar analysis. I hadn't done anything wrong, but i had to defend myself to the head of the department and a panel that was investigating the matter (How can I prove that I didn't cheat?!?!?) After the head of the department grilled me on the subject matter on the spot he felt that I probably didn't cheat, but they still put a note in my file that I was suspected of cheating and told me that if they suspected me of cheating again it would be used to show a pattern and I would be expelled.

I think what must have happened is they didn't take in to account that I have a fever of 101 for the first test and was recovering from a surgery right before the second test so my scores were sub-par. I worked my butt of to be ready for the final and got an A-.

I am sure these systems are statistically a great tool, but in specific cases the human element that is using it has to over ride the mathematical analysis.

Uh no? (1)

OverlordQ (264228) | more than 3 years ago | (#34700288)

like test-takers who did better on harder questions than easy ones, a sign of advance knowledge of part of a test or look for unusually large score gains from a previous test by a student or class

Sounds awesome, lets punish those who started studying since they bombed the first test.

Cheating should be rewarded, not penalized... (1)

mswhippingboy (754599) | more than 3 years ago | (#34700296)

if the purpose of education is to prepare for life in the "real world".
Cheating is a part of everyday life and if you are going to compete in a world of cheats, you need to refine your cheating skills as early as possible.

How else are you supposed to be a competent financial analysis, stock broker, lawyer, etc.. Success in many fields is all based on being the best cheater.

In fact, there should be a requisite course taught in schools titled "How to cheat and get away with it".

Re:Cheating should be rewarded, not penalized... (-1, Troll)

MichaelKristopeit334 (1966808) | more than 3 years ago | (#34700400)

odd that you cower behind a chosen pseudonym while preaching the state of your morals.

you're an ignorant hypocrite.

cower some more, feeb.

you're completely pathetic.

Side effect demographic research (1)

vlm (69642) | more than 3 years ago | (#34700338)

The most interesting part of this expensive and heavily studied technology will be the results (or lack of) with regards to demographics such as race, sex, parents income, political beliefs, whatever.

Re:Side effect demographic research (1)

vlm (69642) | more than 3 years ago | (#34700436)

The most interesting part of this expensive and heavily studied technology will be the results (or lack of) with regards to demographics such as race, sex, parents income, political beliefs, whatever.

Err, I should follow up, I'm not so much interested in "such and such group cheats more" but more interested in cognitive research.

For example its widely believed that men inherently have better spatial analysis skills than women.

So given "imagine a cluster of 64 computers wired in a six dimensional hypercube. They are on a straight linear shelf and must be placed one foot apart for cooling purposes. So theres a line of 64 PCs, which is 64 feet long. Each node connects to six neighbors. What is the longest ethernet cable required?" which is big time a trick question. Maybe a better one would be optimally arranged short average cable length or something. Or how many of each length of cable required. Whatever, anyway, theoretically if there were actually enough female CS students to not be a rounding error, this software might pick up the groups as two separate groups of cheaters. Not because they are cheaters, but because of inherently different cognitive approaches to the problem.

If you ain't cheating, you ain't trying (1)

aethelwyrd (1410845) | more than 3 years ago | (#34700392)

If you get caught, you ain't trying hard enough

Tests? what tests? (0)

Anonymous Coward | more than 3 years ago | (#34700426)

And yet, no mention of how little importance "timed data recollection, now with life altering consequences!" really is in the real world.
Its looking more and more like one of those sadistic game shows, you know?

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>