Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Indiana First With Computerized Grading

michael posted more than 10 years ago | from the replaced-by-a-small-shell-script dept.

Education 524

Mz6 writes "Computerized grading has been talked about previously, however, the New York Times reports that Indiana has become the first state to grade high school English essays by computer. The computerized grading process, called 'e-rater', uses a 6-point rating scale and uses artificial intelligence to 'mimic the grading process of human readers'. The system was tested over a 2-year pilot program and produced results virtually identical to those of trained readers. The big question is, will other states begin to emulate Indiana by tossing human grading?"

Sorry! There are no comments related to the filter you selected.

I smell lawsuits, how about you? (5, Insightful)

Allen Zadr (767458) | more than 10 years ago | (#9205566)

... and produced results virtually identical to those of trained readers.

Funny, because the way I read that is, "Produced lawsuits where the cost is virtually identical to about 20 times the short-term savings."

I see this coming from both sides. The obvious, the grading was wrong, and I lost a scholarship. To other people suing after dropping out of collage level english classes (the test said I was better than I was).

Re:I smell lawsuits, how about you? (0)

Anonymous Coward | more than 10 years ago | (#9205609)

That's nothing which wouldn't also apply to human grading though.

Re:I smell lawsuits, how about you? (1)

Allen Zadr (767458) | more than 10 years ago | (#9205836)

I disagree. The computer spits out a number, and you get to live with the results. I imagine a paper that's both on topic, and well organized... but uses different verbage in the initial description and the conclusion. Perhaps using words that were introduced throughout the body.

A computer will probably not be able to recognize this. A Human would.

Re:I smell lawsuits, how about you? (5, Funny)

m0rph3us0 (549631) | more than 10 years ago | (#9205627)

I think you've taken one to many collage level english classes my friend.

Re:I smell lawsuits, how about you? (5, Funny)

m0rph3us0 (549631) | more than 10 years ago | (#9205673)

s/to/too/. I feel stupid now.

At least the parent proves something... (4, Insightful)

Ieshan (409693) | more than 10 years ago | (#9205769)

At least the parent post proves one simple truth: human english teachers can be replaced by simple shell scripts.

Re:I smell lawsuits, how about you? (0, Redundant)

dtfinch (661405) | more than 10 years ago | (#9205736)

You to.

Re:I smell lawsuits, how about you? (1)

DrEldarion (114072) | more than 10 years ago | (#9205632)

I'm sure they wouldn't make it so there was no dispute process. They'd have to be complete morons to make it so the grade the computer gave you was final. (not that that's unheard of from high-school personnel...)

I'm sure the computer can't tell sarcasm, or tell when a paper is a parody, or give extra points to a particularly good paper from a stupid student or take away points for a particularly bad paper from a brilliant student.

Re:I smell lawsuits, how about you? (2, Insightful)

magefile (776388) | more than 10 years ago | (#9205725)

extra points to a particularly good paper from a stupid student or take away points for a particularly bad paper from a brilliant student.

A "stupid" student doesn't deserve more points just for improving than a non-"stupid" student, and vice versa.

Re:I smell lawsuits, how about you? (3, Insightful)

DrEldarion (114072) | more than 10 years ago | (#9205853)

Why is that? If you had two students, one that you knew was brilliant and one that you knew wasn't the sharpest knife in the drawer, would you give them the same grade for the same paper? If the smarter person put 20 minutes of effort into the paper while the dumber person worked their ass off for a week on it, why shouldn't the grades be different?

This is why essays SHOULD be subjectively graded instead of objectively graded. You need to take into account the writing abilities of the student and determine if it's a good or a bad paper based on what they're capable of in addition to technical aspects of the paper.

Isn't this why there are remedial and accelerated english classes? To take into account the different levels of intelligence in students? If you took an 'A' paper from a remedial class, it's quite likely that it would be a 'D' in the accelerated class.

Re:I smell lawsuits, how about you? (1)

micromoog (206608) | more than 10 years ago | (#9205864)

Then what's the point of education, if not improving one's own abilities?

Re:I smell lawsuits, how about you? (1)

Troed (102527) | more than 10 years ago | (#9205820)

Exactly. How would the computer know that I tried to make a mockery out of 1337 sPeAkErS in the middle of a s3nt3nc3?

"It looks like you're writing a letter ... "

Re:I smell lawsuits, how about you? (4, Interesting)

Shalda (560388) | more than 10 years ago | (#9205729)

I disagree. I see lawsuits as no more likely. Furthermore, any process where you're subjectively evaluting something there has to be quality controls and an appeals process. My wife once held a part time job grading essay questions on a high school exit exam. Every few hours of grading exams, she would have to take and pass a "calibration battery" of 10 exams. Quality control is fundamental to the process.

What I see as being problematic is kids learning to beat the system. Typically these systems are predicated on gramatical analysis (use of punctuation and sentence compeleteness) and evidence of citing the text the question is based off. I'd bet its a real easy system to beat.

Re:I smell lawsuits, how about you? (1)

Glonoinha (587375) | more than 10 years ago | (#9205868)

-I'd bet its a real easy system to beat.

Ya think? Yea, it's called stealing a copy of the program, running your essay through it to see what it doesn't like, and continue to tweak it until the program gives the essay an A. Submit essay for grading, rinse, repeat.

Re:I smell lawsuits, how about you? (1)

ari_j (90255) | more than 10 years ago | (#9205733)

I don't think that bad grading is what's hurting your ability to get a writing-based scholarship, bud. Your comment is barely comprehensible, has almost no examples of correct English grammar, and has some spelling issues and capitalization deficiencies, as well.

Re:I smell lawsuits, how about you? (1)

Anonymous Coward | more than 10 years ago | (#9205841)

I read this article on DrudgeReport.com HOURS ago. Way to be on top slashdot.

As for lawsuits... why? They've done tests and found that the computer grading gave pretty much the same grades/choices that real life professionals gave on the same tests. In addition, a computer doesn't have the bias that teachers have. Ever been a kid that a teacher just loathed and despised - for no discernable reason whatsoever and had to make over a HUGE curve just to get a "fair" result? Or the opposite - a kid that the teacher favors and so you get a grandious positive curve in your grade thanks to that preferential treatment?

third post!!! (-1, Troll)

Anonymous Coward | more than 10 years ago | (#9205574)

I mis read the title as "india first in e-grading"

OSS? (3, Interesting)

Karamchand (607798) | more than 10 years ago | (#9205578)

Is this program available under an open source licence? It sounds really interesting!

Re:OSS? (3, Insightful)

twistedcubic (577194) | more than 10 years ago | (#9205591)

Cool idea. Imagine high school students re-writing their essays until the grader software gives them an A+.

Re:OSS? (0)

Anonymous Coward | more than 10 years ago | (#9205857)

Cool idea. Imagine high school students re-writing their essays until the grader software gives them an A+.

Huh? You only need the binary to do that, whether it's open or closed source doesn't come into it.

Re:OSS? (0)

Anonymous Coward | more than 10 years ago | (#9205640)

how the fuck is the parent redundant?!

Re:OSS? (4, Funny)

Glonoinha (587375) | more than 10 years ago | (#9205828)

Slashdot First With Computerized Moderation.

Now with Computerized Moderation the famous Slashdot message site can pre-emptively down-mod 'Redundant' posts long before they are actually 'Redundant.' The computerized modding process, called 'e-modder', uses a 6-point rating scale and uses artificial intelligence to 'mimic the modding process of human readers - including doing stupid shit like modding the first instance of a concept as Redundant'.

I would have loved this is a kid (5, Interesting)

cheezus (95036) | more than 10 years ago | (#9205580)

it would have been my goal to make the most wrong essay I could that would still generate a good grade from the system.

Re:I would have loved this is a kid (3, Interesting)

jeffy124 (453342) | more than 10 years ago | (#9205620)

an opinion paper on why dyhydrogen monoxide must be banned would perhaps do the trick?

Re:I would have loved this is a kid (0, Redundant)

jbrader (697703) | more than 10 years ago | (#9205816)

Well of course it should be banned. Hundreds of people drown in huge pools of it every year.

First post? (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#9205584)

Hope so!!

Google Bombing (1, Interesting)

m0rph3us0 (549631) | more than 10 years ago | (#9205587)

I can't wait til someone figures out how to google bomb the grading system.
I wonder if it will be as simple as repeating a high ranking sentence?

Re:Google Bombing (2, Funny)

superpulpsicle (533373) | more than 10 years ago | (#9205679)

Computerized grading is great.
Computerized grading is superb.
Computerized grading is excellent.
Computerized grading is outstanding.
Computerized grading is god.
Computerized grading is great.
Computerized grading is superb.
Computerized grading is excellent.
Computerized grading is outstanding.
Computerized grading is god.

Essay Result = A+

I already want a copy of this. (5, Insightful)

ShitPissFuckCuntTits (781511) | more than 10 years ago | (#9205590)

I bet I could write the other side of the equation: a program to create nonsensical gibberish that always gets A's. What would a teacher do if you handed in something like that? Apply a double standard to the student?

In fact, (3, Interesting)

ShitPissFuckCuntTits (781511) | more than 10 years ago | (#9205648)

As a student the first thing I would hand in is twenty paragraphs from refreshing this [rubberducky.org] . In political science class, of course.

Re:I already want a copy of this. (1)

glenrm (640773) | more than 10 years ago | (#9205798)

If I was the teacher I would give the a student the student what the deserve an A.

Re:I already want a copy of this. (4, Funny)

Kozar_The_Malignant (738483) | more than 10 years ago | (#9205814)

> I bet I could write the other side of the equation: a program to create nonsensical gibberish that always gets A's.

I'll bet half the people here thought this as soon as they read the headline. The normal /. response is to spend 10 hours at the computer coding up something to do a task you can do in 1 hour with a pencil. :-) Of course the high school geek who does this will make piles of money selling the output to the football players, buy a nice ride, and get a prom date with a cheerleader. Isn't that the way it works?

Stupid (4, Interesting)

phreak0003 (726129) | more than 10 years ago | (#9205593)

I live in Indiana, and I have taken these. They are not graded fairly, and they determine 10% of the final grade. A computer can obviously not grade essays fairly, so it shouldn't be done. I got a 5/6, which, according to the computer, was extremely well. However, this was an 83%, which brought down my grade significantly. This computerized grading isn't fair at all.

Re:Stupid (2, Interesting)

RoderickMcDougall (661783) | more than 10 years ago | (#9205661)

I agree. Did you receive any comments from the computer or was it just an unjustified percentage? I would be most interested to know how they have managed to justify this questionable practice. Almost as someone is rushing to boast about being at the forefront of technology without the goods to back it up. Oh the children! Won't someone please think about the children!

Re:Stupid (2, Insightful)

garcia (6573) | more than 10 years ago | (#9205670)

I have taken plenty of essay exams where I felt I wasn't graded fairly by a human teacher/professor.

Some essays were graded out of a couple points. A paper out of 6 points carries less weight overall. If this is the only exam (ie AP tests) a 5/6 is looked at as a high score.

I don't see your point.

Re:Stupid (3, Funny)

Anonymous Coward | more than 10 years ago | (#9205697)

Maybe you got docked for overuse of commas.

Re:Stupid (0)

Anonymous Coward | more than 10 years ago | (#9205735)

Oh give the kid a break. His score of 5/6 was extemely well.

Re:Stupid (2, Insightful)

Roland Piquepaille (780675) | more than 10 years ago | (#9205775)

Not only computers may not give fair grades, but there's a deeper problem with grading using computers: to me, students working to get good grades from a computer conjures up images of sheeps going in the wool-extraction machine. While this may be fine for sheeps, how do you think the students feel about it?

When I was in school, I was glad to know whatever essay I was writing was being read by my teacher, whom I had real human student/teacher relationships with, and whom I could discuss whatever was or wasn't right in the essay after class. The schooling system already lacks humanity, why de-humanize it even more?

Re:Stupid (2, Insightful)

garcia (6573) | more than 10 years ago | (#9205827)

When I was in school, I was glad to know whatever essay I was writing was being read by my teacher, whom I had real human student/teacher relationships with, and whom I could discuss whatever was or wasn't right in the essay after class. The schooling system already lacks humanity, why de-humanize it even more?

You had a different school experience than I did apparently. I felt that the human's reading my papers were distant, uninteresting, and less than worthy of grading someone else's work.

Generally comments were kept to a bare minimum on a good paper. "Good job!" or "Excellent research!" is about as lame as getting a 5/6 on a standardized essay exam from a computer grader.

On "bad papers" the comments were usually less than helpful. Don't just mark up the paper with "comma splice" or "vague". The teacher should have taught the class what a "comma splice" was or should have been following their own words of advice and kept themselves from being "vague" in their comments.

Re:Stupid (0)

Anonymous Coward | more than 10 years ago | (#9205824)

was extremely well

Perhaps that is where the 5/6 comes from.

I guess this rules out... (2)

Loco3KGT (141999) | more than 10 years ago | (#9205599)

schmoozing with the teacher to get higher grades.

In unrelated news, Delicious Red Apples have suffered a terrible sales slump.

Re:I guess this rules out... (1)

underworld (135618) | more than 10 years ago | (#9205734)

I guess now you have to give the teacher a Macintosh instead of a Red Delicious.... ;-)

That's ok (2, Funny)

paranode (671698) | more than 10 years ago | (#9205801)

schmoozing with the teacher to get higher grades

This works better for the Slashdot crowd. They are much better at romancing computers than people to get what they want.

comment moderation (4, Funny)

Jotaigna (749859) | more than 10 years ago | (#9205604)

for the time being, i would trust more that program to moderate my comments.



c'mon people i was only joking dont mod me down, not noooo!!

What about tricking the software? (5, Interesting)

gtaluvit (218726) | more than 10 years ago | (#9205605)

SPAM filters are tricked all the time depending on the text of an email. Google was f'd up not too long ago because of trackback linking in blogs screwing up their algorithms. Isn't this a similar situation? If a student can figure out a way to beat the grader, we'll have students learning to write to beat software, not form a well written essay.

Re:What about tricking the software? (0)

Anonymous Coward | more than 10 years ago | (#9205639)

Since when is SPAM an abbreviation and therefor written ALL UPPERCASE?

Re:What about tricking the software? (1)

Marxist Hacker 42 (638312) | more than 10 years ago | (#9205753)

If a student can figure out a way to beat the grader, we'll have students learning to write to beat software, not form a well written essay.

Which, if Harris Miller of the ITAA is actually right, is exactly what we need kids to learn so that we can compete with India.

So much for those essays (5, Funny)

Zancarius (414244) | more than 10 years ago | (#9205607)

Perish the thought should students start writing about the dangers of artificial intelligence. They may very well fail!

Neat (-1, Offtopic)

stealthmidget (761031) | more than 10 years ago | (#9205611)

Very cool...if I could get to the article without registering, I'd consider reading it and determining if this is anything more than the Word spell/grammar check function. But I can't, so I won't.

Re:Neat (1)

James Lewis (641198) | more than 10 years ago | (#9205731)

Well, as most people around here already know, if you past the link into google, and then click on the link google gives, NYTimes will let you in without registering.

Gaming the system (5, Insightful)

ePhil_One (634771) | more than 10 years ago | (#9205612)

While it gives identical results for now, I can easily see the coming books: How to Write an A essay! Form essays to get you into Harvard.

The GMAT books are already giving formula essays to get you past any writers block that might happpen on the exam day...

Ridiculous (1)

RoderickMcDougall (661783) | more than 10 years ago | (#9205614)

If Dr Sbaitso can't cure what ails me how the hell is a computer supposed to miraculously rise to the level of intelligence required to grade creative literacy. Perhaps its all a ruse and Dr Sbaitso really is behind the grading program. Hint, look for: How does it make you feel?

Indiana? I thought it said India (0)

Anonymous Coward | more than 10 years ago | (#9205615)

And I was going to make a joke about India outsourcing teaching jobs to computers... alas...

Re:Indiana? I thought it said India (0)

Anonymous Coward | more than 10 years ago | (#9205779)

Yeah well, racist Americans are a dime a dozen. No need to stand up and call yourself a jackass in every /. article.

Not the First (4, Interesting)

dcocos (128532) | more than 10 years ago | (#9205623)

My alma matter [wisc.edu] graded most of my computer programs with shell scripts and I graduated in 1997. So I don't think India is the first to do that.

Re:Not the First (2, Insightful)

gregarican (694358) | more than 10 years ago | (#9205669)

Grading freely-written essays and structured computer programming code is two separate things *to a degree.* And India isn't the first to do the latter. Indiana is, however, according to the article.

Re:Not the First (1)

gregarican (694358) | more than 10 years ago | (#9205747)

Actually I meant the former, not the latter. Everyone makes mistakes I guess :-)

Re:Not the First (0)

Anonymous Coward | more than 10 years ago | (#9205799)

The difference here is that these are essays, not programs. The grading of essays is (or at least I hope) a lot more involved than making sure that a program has the correct output, etc.

It makes me wonder just how accurate this program can be. Is there really a formula for good essays or writing styles? What happens if its a well written essay, but all the facts are wrong? I really wish the article had gone into greater detail.

Re:Not the First (2, Funny)

Sporkinum (655143) | more than 10 years ago | (#9205808)

So are all IT jobs being outsourced to Indiana?

Re:Not the First (0)

Anonymous Coward | more than 10 years ago | (#9205821)

My alma matter graded most of my computer programs with shell scripts and I graduated in 1997.

Yes, but the professor can test the correctness of your program with known inputs that should give known outputs. This is easy to automate.

English essays have far more complexity.

Yeah... (1)

Paulrothrock (685079) | more than 10 years ago | (#9205624)

Sure, this won't be hacked... Some script kiddie is gonna get the algorithm and get straight A's for his whole life on gibberish and devote more time to a new virus.

Oh, my. (1)

eddy (18759) | more than 10 years ago | (#9205626)

But does it do any semantic analysis? I assume a real person at least reads the material once, or this is going to pass some very interesting material.

First mistake? (1)

macshune (628296) | more than 10 years ago | (#9205635)

My Great Grandfather's Most Famous Quote
By H. Jones III

"We named the cat Indiana!"


Re:First mistake? (0)

Anonymous Coward | more than 10 years ago | (#9205840)

Henry: What did you find, Junior? Indy: Junior?! Dad... Sallah: Please... what does it always mean, this... this "Junior?" Henry: That's his name. Henry Jones, Junior. Indy: I like Indiana. Henry: We named the dog Indiana. Brody: May we go home now, please? Sallah: (To Indy) The dog!? (Laughs) You are named after the dog... Indy: (Embarrassed) I've got a lot of fond memories of that dog.

Those Indianans are ruining us! (5, Funny)

L. VeGas (580015) | more than 10 years ago | (#9205638)

Lets just outsource all our test grading to Indiana too.

It's About Time (3, Interesting)

USAPatriot (730422) | more than 10 years ago | (#9205644)

Many geeks like me did not like English class for the simple reason that grading was entirely subjective to the teacher's tastes.

If he or she didn't like what you wrote, or took a point of view opposite to theirs, you would get a lower grade. Frequently, the "special" students would get the benefit of the doubt, and easy grading just for exceeding their own limitations. An 'A' paper in one English class could be a B- in another, etc, etc.

With this computer grading, these students now know that they will be treated equally, and not bitch about potential human biases. Then, everyone will have a fair shot.

Re:It's About Time (0)

Anonymous Coward | more than 10 years ago | (#9205829)

...subjective to the teacher's tastes...

That's gibberish. Please learn to communicate.

Then, everyone will have a fair shot.

Wrong again. Read the article: "The system ...produced results virtually identical to those of trained readers."

In other words, the system is doing exactly what the teachers always did. What's that, you ask? Okay, I'll tell you: There are several components to a "grade" awarded by the system. Much of it comes from a database of personal information about the students. Female? +5%. African-American? +10%. Actual African from Africa? No extra points for that one, sorry. White male? -10%. Bible-believing Christian? -20%.

Under the provisions of the USA Patriot Act, students who have "displayed an abnormal/pathological interest in technology" (as defined by DSM IV) or who practice Islam are penalized and the content of the essay is scanned for "key words"; the results of the latter analysis are forwarded automatically to the US Dept. of Homeland Security for further scrutiny, without the student's knowledge or consent.

Finally, the content of all exams will be scanned for key "consumer words", and appropriate advertisements will be printed in the margins.

Even more efficient? (1)

yndrd (529288) | more than 10 years ago | (#9205654)

Let's just use a random generator to give out grades. We'll get the same bell curve distribution, won't we?

hmm (0)

Anonymous Coward | more than 10 years ago | (#9205660)

Does that mean you could have a program that would tell you what grade you'd get on your essay before you turned it it? And then fiddle with it until it's an A? Would that be cheating?

Trained readers... (3, Insightful)

ari_j (90255) | more than 10 years ago | (#9205677)

The system was tested over a 2-year pilot program and produced results virtually identical to those of trained readers

I think this says more about the training that the "trained readers" are receiving than it does about the software.

This would be wonderful, if it works (1)

magefile (776388) | more than 10 years ago | (#9205685)

As it is now, my current teacher doesn't like my essays, but isn't clear about what I can do to improve - a program might help with that. And it would be more consistent.

But somehow, I doubt it's that good. Plus, kids like me are just waiting for it to be released so we can figure out how to scam it. :) Too bad I don't live in Indiana.

Perfect scores every time (5, Funny)

Rorschach1 (174480) | more than 10 years ago | (#9205691)

Ok, since you know the grading software is going to make it into the hands of the students, here's my scheme for perfect essays:

Step 1: Feed some encyclopedia articles, Wiki pages, and other random material on your subject into a Markoff chain generator.

Step 2: Use a genetic algorithm to generate variations of the text. Fitness is determined by the grade calculated.

Step 3: Repeat step 2 until desired grade is achieved. (And, of course, Profit!)

The result is totally worthless, but at first glance would probably appear legitimate even to a human reader.

Sort of like Slashdot posts.

We have computers to help grade assignments... (1)

Anubis333 (103791) | more than 10 years ago | (#9205694)

Now if we could only get computers to help teach our children!

But.. (0, Offtopic)

abscondment (672321) | more than 10 years ago | (#9205700)

Some idiot will figure out how to trick the program and we'll have gobs of script kiddies getting A's in English, when in reality they write so poorly that they should be beaten.

Haxored!

Content? (1)

sokk (691010) | more than 10 years ago | (#9205704)

Well, what about the content of the essays? Anyways, any way to get a hold on the software :)? Would've been nice with a "clippy" that said what grade the current document is.

AI (5, Funny)

Bugmaster (227959) | more than 10 years ago | (#9205707)

That's actually a pretty novel way to approach the problem of creating Strong AI. Making smarter machines is hard, so what you do is dumb down the humans until even a coffee maker (or a grammar parser or whatever) would beat them in the Turing test. Damn, this is so sad.

e-Rater result? (2, Funny)

ackthpt (218170) | more than 10 years ago | (#9205709)

Input: All your base are belong to us!

Output: A+

identical results to those of trained readers... (3, Interesting)

GGardner (97375) | more than 10 years ago | (#9205716)

Maybe these says more about the readers than the computer program?

And the era of virtual teacher begins! (1)

gnuman99 (746007) | more than 10 years ago | (#9205719)

We dont ned a techer to marc mi worc anyway. The komputer said I writee good!

Seriously, how can a program replace a human when the program cannot comprehend structure like language? Computers cannot and should not replace English teachers or math teachers (well, beyond grade shool at least!). How can a computer program mark an English paper? How can a computer program check that a mathematical proof is correct? How can a computer program say that a particular train of thought is interesting, or pointless.

But above all, does the equal result of a computer software mean that the software is so good or does it mean that the markers grade papers like computers?!? (ie. without any insign)

This is both good and bad (3, Insightful)

n1ywb (555767) | more than 10 years ago | (#9205732)

Good: The computer probably won't grade you down for writing an anti-Bush essay, and it probably won't get fired for it. Good: Computers won't play favorites, and you can't kiss up to a computer. Bad: The computer really can't grade you up for expressing original ideas. Bad: It's probably possible to fool the computer somehow.

copyright and plagiarism (0)

Anonymous Coward | more than 10 years ago | (#9205742)

Possibly off-topic, but, IIRC, there was a computerized method of detecting plagiarism by students submitting college level essays/theses (I forget the URL) by comparing them to a library of previously submitted work. This seems to me to be a simple thing to link to such a system. When will I have to start copyrighting my assignments, giving my teachers a limited license to grade my work but not store in an information retrieval system?

I can't wait (4, Funny)

proverbialcow (177020) | more than 10 years ago | (#9205748)

...until some wiseass figures out a way spoof the grader, probably by sliding under the radar of whatever probabilistic models they've got that pass for spell- and grammar-checkers.

For example:

Flimblarm nif goondatakun, jut sekfar bel shon duc. Seempkin dar goolnac flar tefnek voz toulian; elmpar gef sogquel.

Grade: B+ Your use of double-negatives continues to haunt you, but I'm glad you've gotten over hanging participles.

Re:I can't wait (0)

Anonymous Coward | more than 10 years ago | (#9205823)

I would often get a grade A based on my handwriting and illustrations. My text was peppered with intermittent obscenities. So I's say this sort of system is probably pretty accurate.

Copying (1)

rf0 (159958) | more than 10 years ago | (#9205767)

What about if one person copies another will it pick that up and flag it?

Re:Copying (0)

Anonymous Coward | more than 10 years ago | (#9205804)

Probably not... but lets patent the idea of doing both grading and plagiarism checking at the same time!

In Other News (5, Interesting)

ThisIsFred (705426) | more than 10 years ago | (#9205774)

Indiana parents are the first to buy (en masse) licenses for Essay Constructor Pro v2.0. The software produces essays that are indistinguishable from those written by real students, using the latest screen-scrape-from-Internet 'n' plagiarism-from-non-credible-sources techniques.

Indiana Director of State Board of Ed comments: "Isn't it wonderful how technology is improving education?"

I have an algorithm they can use... (0)

Anonymous Coward | more than 10 years ago | (#9205780)

... and save a bunch of money.

Randomization!

Statistically speaking, 75-80% of the people won't question their grades. For the rest of them, let a lengthy, expensive, and convoluted appeals process sort 'em out. Either that, or threaten to grade them again on the chance that their score is lowered.

I wanna try, I wanna try! (1)

lpangelrob2 (721920) | more than 10 years ago | (#9205782)

So if I did this, what would happen?

The word emission generally means sending something out. Because of this argument, Hauser is a city located in Kootenai County, Idaho. After Idaho, the Liberals formed the government in Alberta for the first 15 years of the province's existence.

Yeah, those were random snippets from the Wikipedia. Who knows? Maybe this technology [macdevcenter.com] got around?

Grade: A+++

Other applications (0, Redundant)

jdbarillari (590703) | more than 10 years ago | (#9205783)

A 6-point computerized grading system. Hmmm. Moderators, watch out -- your jobs are about to become obsolete!

This says more about "trained readers"... (4, Insightful)

Phurd Phlegm (241627) | more than 10 years ago | (#9205788)

than it does about the software, methinks. I imagine it gives helpful hints like the ones I always turn off in Microsoft Turd. Any construction that deviates from the norm in a boring business document apparently triggers the "grammotron" or whatever they call it. A human reader has some appreciation for style and may actually accept something a little different for the sake of variety and sparkle.

Not that there's anything in this post that serves as an example. I guess that's because I was graded by humans. Seriously, I don't recall getting any encouragement in writing back in the '70s in high school, and not much in college. I guess it wouldn't have been any worse if the Grade-O-Vac was inspecting my papers instead of my mostly-marginally-literate teachers. There were several exceptions, but they focused much more on reading than on writing. I suspect they had a lot greater effect that way--I know they had a great effect on me.

No way this is sound (1)

Hays (409837) | more than 10 years ago | (#9205789)

I cannot believe they are doing this. It is undoubtedly a deep AI problem to properly grade an essay. You have to be able to understand if statements connect logically to form an argument, you have to understand analogy, you have to understand sarcasm, etc... this is so far beyond our current capability.

As implemented, I'm sure there are easy ways to scam this system by writing gibberish. Who sold them this idea? I want to know the person responsible for this abuse of computing.

Students (3, Funny)

Gettinglucky (655935) | more than 10 years ago | (#9205790)

Now all the students need is e-writer so that they can just type in the subject and the score they want to achieve and then e-reader will grade it accordingly!!!

Can students get this program? (4, Insightful)

Theovon (109752) | more than 10 years ago | (#9205793)

If I were a student, I'd want to get a copy of this software and use it to pre-grade my papers so that I could find out what's wrong and fix it before I turned it in.

dangerous dangerous dangerous (1)

tplayford (308405) | more than 10 years ago | (#9205815)

Especially for a subject like English. Over the years teachers will begin to understand how the computer makes and they'll tell their students what style will get the beast marks.

It becomes a study of the marking algorithms, not a study of English.

I suppose they could make alteratoins every year... but then it could become unfair.

India vs. Indiana (4, Funny)

the eric conspiracy (20178) | more than 10 years ago | (#9205830)

Wouldn't it just be cheaper to grade the tests at call centers in India? What are those Indians doing when there are no incomming calls? Just slacking off??? They could be grading tests.

Is this a joke? (0)

Anonymous Coward | more than 10 years ago | (#9205832)

e-rater will generate an advisory if it has difficulty scoring or identifying some or, all, of the writing sample.

I wonder if e-rater flagged that incorrectly placed comma.

Seriously, does anyone remember a program for Apple II circa 1985 called "Babble"? You'd give it a couple of keywords and a number of paragraphs, and it would churn out pages of grammatically valid, completely readable nonsense about any topic.

From the FAQ:

I don't see how a program can be much more than a super-sophisticated grammar-checker. As the FAQ says, e-rater doesn't actually "read" anything. I wonder how it would score the output of Babble.

How would e-rater would judge essays written by, say, Hemingway, or another writer who has a sophisticated and unique style. How does a machine judge sarcasm or humor in general? How does it judge accuracy or depth of thought? How would a well-written but totally racist essay be graded?

From the FAQ:

e-rater compares the new essay to samples of essays previously scored by faculty readers-looking for similarities in sentence structure, organization, and vocabulary.

Great. The more you conform with your peers, the higher your score.

Last point-- writing an essay to be read by a human and one to be processed by a machine is, I suspect, a different psychological exercise. To write for a human is called "communication". To write for a machine is called "programming" or "data entry".

Makes life easier (1, Redundant)

Dracolytch (714699) | more than 10 years ago | (#9205834)

Since I know a real person won't be reading the paper, all I need to do is come up with an AI to write the paper.

2: Sell it to my classmates
3: Profit!

~D

Completely offtopic papers (2, Interesting)

fulldecent (598482) | more than 10 years ago | (#9205848)

...and how would one ever devise software capable detecting papers that are
-1 Offtopic
while completely gramatically correct?

For grading grammar, a computer would be unbiased (1)

tmundar (587769) | more than 10 years ago | (#9205855)

I wrote many essays when I was in school, and I remember being graded down because the teachers didn't like my ideas. The quality of my writing was acceptible, but how I wrote the topic was not. I didn't write anything offensive, but I occasionally took creative liberties with the point of view (i.e. I changed the gender of the person who wrote the article, and then interpreted it from that point of view). Maybe I write an essay about atheism, but the grader is intensly religious and has a bias against my point of view.

I just want to point out that having a computer grade a paper using rules would be much more 'fair', because the computer won't have any personal bias about the content. I am sure that the computer would grade an essay against computerized grading identically to an essay promoting computerized grading.

This will create more techies... (1)

MoeMoe (659154) | more than 10 years ago | (#9205865)

As many have said regarding this topic so far, the kids will try to find out what makes this system tick and find an exploit... Having said this, wouldn't that end up making atleast a few of these kids smarter through trying to "crack the code"?

It's just my opinion, but I say if you can figure out how to use the tricks of the trade, whether on some grading system or a complex business scenario, you deserve the grade you set for yourself... That's what learning (and to some extent grading) is about, right?
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?