Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Computers Could Grade Essay Tests Better Than Profs

timothy posted about 3 years ago | from the mckittrick's-behind-this dept.

AI 323

An anonymous reader writes "Robot essay graders could be the answer to grade inflation. New software being tested turns over the task of grading to computers — this article has an interactive demo of the software. One professor says the computer is far fairer than human graders, who get tired and become inconsistent, or play favorites."

cancel ×

323 comments

Sorry! There are no comments related to the filter you selected.

Playing favorites (0)

Anonymous Coward | about 3 years ago | (#37014854)

But then who will be the teacher's pet?

Re:Playing favorites (3, Funny)

Farmer Tim (530755) | about 3 years ago | (#37014916)

Not who, what [oldcomputers.net] .

Re:Playing favorites (4, Insightful)

Opportunist (166417) | about 3 years ago | (#37014958)

The best student. Duh.

Re:Playing favorites (1)

kanto (1851816) | about 3 years ago | (#37015498)

Kids with Apples of course.

Re:Playing favorites (0)

Anonymous Coward | about 3 years ago | (#37015872)

The Asian kid.

Play favorites? I believe it (4, Interesting)

sandytaru (1158959) | about 3 years ago | (#37014868)

I once got an F on a paper from a TA who wrote in the margins "How dare you try to say what Shakespeare was thinking!" Um, that's what literary analysis IS, to some extent. You try to place someone's written works within the context of their culture and society at large and reconstruct their thought processes and views on the world. But that TA was an asshole and had it out for me, and many of us complained about him bitterly for years afterward. The only person who got an A in that entire section was one cute girl.

As long as the robo-grader also includes a plagiarism check, I'd be okay with it. My husband is a professor and most of his failed papers are a result of TurnItIn.com catching outright plagiarism.

Re:Play favorites? I believe it (0, Flamebait)

Anonymous Coward | about 3 years ago | (#37014890)

I once got an F on a paper from a TA who wrote in the margins "How dare you write nothing but bullshit" Um, that's what literary analysis IS, completely.

FTFY

Re:Play favorites? I believe it (2, Interesting)

hal2814 (725639) | about 3 years ago | (#37014950)

The purpose of literary analysis is to analyze literature, not the author writing it. It was an asshole way to put it, but the TA was correct. It doesn't matter what the author thought or even intended. The only thing that matters is what the author wrote and what we can analyze from that.

Re:Play favorites? I believe it (0)

Anonymous Coward | about 3 years ago | (#37015166)

I whole heartedly disagree. As one of my engilsh professors put it: the only way to truly know a man is through his work and the only way to truly know a man's work is through the man.

Re:Play favorites? I believe it (1)

Anonymous Coward | about 3 years ago | (#37015550)

Your professor is an ice cream koan....

Re:Play favorites? I believe it (0)

Anonymous Coward | about 3 years ago | (#37015228)

This is what every English professor I have ever had has hammered into my skull.

Re:Play favorites? I believe it (4, Insightful)

damienl451 (841528) | about 3 years ago | (#37015236)

This is why most people don't take literary analysis seriously. There is a real human being who took the pain to write a 400-page long book. Presumably, he wanted to convey *something*. But apparently, we have to act as if the book came down from heaven and we can't try to discover what the author wanted to say?

The worst manifestation of this is when some literary theorists seem to argue that *even the author* cannot interpret what he wrote better than anyone else. He's just another reader!

This sounds ridiculous to me. Even if the author writes an essay saying "this is what I meant when I wrote this", we're supposed to ignore that and simply focus on the words of the work because this is all that matters in literary criticism?

Re:Play favorites? I believe it (2)

digitig (1056110) | about 3 years ago | (#37015416)

This is why most people don't take literary analysis seriously. There is a real human being who took the pain to write a 400-page long book. Presumably, he wanted to convey *something*. But apparently, we have to act as if the book came down from heaven and we can't try to discover what the author wanted to say?

No you don't, but you can't say what Shakespeare thought, you can only say what you think Shakespeare might have thought. That's valid interpretation, and if you can back it up with the work of other critics then you're heading towards a supported academic position. If you claim to know what Shakespeare thought then everybody knows you are bullshitting because nobody does.

The worst manifestation of this is when some literary theorists seem to argue that *even the author* cannot interpret what he wrote better than anyone else. He's just another reader!

This sounds ridiculous to me. Even if the author writes an essay saying "this is what I meant when I wrote this", we're supposed to ignore that and simply focus on the words of the work because this is all that matters in literary criticism?

The author (possibly) knows what [s]he intended to communicate. You find what [s]he actually communicated in the words on the page. They're not necessarily the same.

Re:Play favorites? I believe it (1)

mooingyak (720677) | about 3 years ago | (#37015560)

The author (possibly) knows what [s]he intended to communicate. You find what [s]he actually communicated in the words on the page. They're not necessarily the same.

What was actually communicated is entirely subjective and will usually vary from person to person. What was intended is singular.

Re:Play favorites? I believe it (2)

digitig (1056110) | about 3 years ago | (#37015752)

The author (possibly) knows what [s]he intended to communicate. You find what [s]he actually communicated in the words on the page. They're not necessarily the same.

What was actually communicated is entirely subjective and will usually vary from person to person. What was intended is singular.

What was intended might be singular but it rarely merits academic study and is inaccessible (even if the author is on record about it, how do we know that they were accurate and honest?)

The whole point about the humanities is that they are inherently subjective (there's a bit of a clue in the name), so learning to deal with subjectivity is an important part of a humanities course. What one person thinks is of some limited interest. The similarities and differences between what different people report finding in a text and the possible reasons for those similarities and differences do tend to be worthy of academic investigation.

Re:Play favorites? I believe it (1)

FoolishOwl (1698506) | about 3 years ago | (#37015878)

It's hard to know, out of context, whether the grad student grading the paper was making an insightful criticism or a stupid one. Stupid grad student tricks abound, but stupid undergraduate tricks are even more common. It rather depends upon the work under analysis and what the student actually wrote.

With Shakespeare,, most of what we know about what he thought is from literary analysis of his plays and sonnets, supplemented with inferences drawn from general historical knowledge of the period, and what we know of his patronage and business dealings. When a student writes, "Shakespeare believed in the Great Chain of Being; therefore, he must have meant it was wrong for Brutus to assassinate Julius Caesar", the student is getting things backwards, because most interpretations of Shakespeare's beliefs about aristocracy are based upon what he wrote in Julius Caesar and the various history plays.

Re:Play favorites? I believe it (0)

Anonymous Coward | about 3 years ago | (#37015418)

If we wanted to know what the author was thinking, we could ask him. This is a trivial exercise left to the reader. However, what his/her work means to society is a much more interesting question that the author would probably not be able to answer adequately.

Re:Play favorites? I believe it (1)

obarthelemy (160321) | about 3 years ago | (#37015670)

Next time you run into Big William, please ask him what he meant by "to be or not to be" for me !

Re:Play favorites? I believe it (4, Insightful)

calmofthestorm (1344385) | about 3 years ago | (#37015430)

This is a bit of a strawman. In high school English, it was explained to us as: humans write literature, and sometimes they have something to say. This doesn't mean that they are the final word on what broader meaning their work has, but it does mean they have a deep insight into it. So no, don't ignore authors, but don't expect appeals to their authority to be viewed as anything but a fallacy in and of itself.

Or put another way: Read Robert Frost's "Stopping By Woods On A Snowy Evening" (http://www.iment.com/maida/poetry/frost.htm#stopping), (it's short). The dominant interpretation of this poem is that it is an allegory of old age and death. Frost, however, insisted that this poem was about nothing more than taking a ride through a wood on a snowy evening. Who's right? It's not an either/or. In literary analysis there are right interpretation*s* and wrong interpretations, but it's not like there's just one right answer.

Or at least that's what I remember from my last literary analysis class taken, which was in high school many years ago.

Re:Play favorites? I believe it (0)

Anonymous Coward | about 3 years ago | (#37015554)

I have to agree with Frost. It reads just like the title of the poem.
If I could write poetry, I wish to think I'd written something similar about my stroll alone to a frozen lake in Hokkaido years back (sans horse).

Re:Play favorites? I believe it (4, Insightful)

Anubis IV (1279820) | about 3 years ago | (#37015836)

I remember reading a quote from Tom Stoppard (playwright for Rosencrantz and Guildenstern are Dead, among others), in which he basically said that writing is like packing a bag that he later takes to the airport. The people at the airport opt to inspect his bag and they start finding all sorts of things that he didn't remember putting in the bag, but he can't deny that they are there.

There is an argument to be made that we should take art as it was intended, but oftentimes the benefit of something is other than what was intended (case in point: movies that are so bad they're good), so there shouldn't be a reason why we deny that line of thinking as well. That said, there should be a limit. Some people, particularly the sort of liberal arts folks we all love to lampoon, try to insert their own things into the bag, rather than finding things that were legitimately there in the first place. But if they're simply discovering additional, yet unintended, depth to a classic piece of literature that can help us appreciate it better? Yeah, I see no problem with that. It may be unintended, but that doesn't mean it's not there. Even so, we shouldn't ascribe more meaning to it than it's due.

Re:Play favorites? I believe it (1)

epine (68316) | about 3 years ago | (#37015672)

The only thing that matters is what the author wrote and what we can analyze from that.

Taking that position, it makes it hard to understand why so many academics heap scorn on Wikipedia, which operates under a similar proscription: you can only write what you can source, even if what you wish to say is so obvious that no reputable source bothers to spell it out, in direct terms.

You can not state the insidiously obvious on Wikipedia. I once tried to add a footnote to a term from computer science that has long fallen out of fashion, as to why the term has fallen from fashion (it was always stupid deep down), but it was so out of fashion nobody with anything useful to contribute to the field had commented on the term for fifteen years; while many in the active profession would have immediately agreed with me, my contribution was scratched. Newcomers to the field become aware of the taxonomic term, and take it too seriously, thinking it conveys professional insight. Wikipedia would be a great place to spread the news that the term has died and not gone to heaven: you sound like a neophyte for taking it seriously. Every profession has these lingering embarrassments, just ask Freud.

Concerning my scratched contribution: my bad. You have to play by the rules. What's unfortunate about Wikipedia culture is that once the rule is decided, awareness of where or how it breaks down is not encouraged. Here again, Wikipedia mirrors life in the academic setting. No surprise to me that contribution wanes to precisely the degree that academic virtues wax.

The textual conceit of literary criticism doesn't hold up to Kolmogorov-Chaitin complexity theory. Not much does. Practical problems abound in theories of infinite potency and nil applicability. All the results of K-C theory live in the event horizon between finite and the infinite: where the gravitational mass of the energy of computation sucks you backwards into a black-hole before the (finite) computation terminates.

K-C theory (as it exists in my intuition) would not sharply discriminate between the author and his text; neither would it sharply discriminate one author from the next, but heavily factor out the common font of all memes, both genetic and cultural, if you fed it plenty of surrounding context.

Restriction to analysis of the text is an institutional pretext: the institution of literary criticism requires this (unless you've got Marshall McLuhan hidden behind a handy placard a la Annie Hall, which is funny, because never was a man less capable of tilting the balance toward clarity).

If the author can't explain himself (where "him" is the first metasyntactic variable of lambda elided), you've got some explaining to do about explanation. The big bang doesn't exactly "explain" itself either, but we do happen to occupy a strange physical regime where physical theory is possible and thus amenable to K-C compression. Scientists ponder why physical law is possible (it doesn't have to be). It's just something we observe in the universe we happen to inhabit. (At this point otherwise sensible people start dividing by zero by contemplating universes we don't inhabit; yes, you can make our existence contingent on a prior that divides by zero, but I don't see where that gets you, precisely--except out of the frying pan, into the fire).

If you're striving for credit in an institutional setting (especially the common coin of undergraduate credit, where you tip your end product into the round file immediately upon its award), you're not going to earn much by speculating on the mind of the author explicitly.

Of course, our capacity to engage in textual criticism presumes a cognitive architecture where we do precisely that, but then we erase our workings at the boundary of collectivisation. Oh, the humanities.

The first thing an alien galaxy will do with our emanating (but not eminent) symbol stream is construct a theory of human mind. Warning space alien: don't show up for literary criticism class, two heads and six eyes stalks will guarantee you a failing grade (though you might prompt a revision of the list of acceptable metasyntactic pronouns in deference to your undefinable otherness).

Alien: Ugly bags of misty alcohol never let up on this girl/boy thing.

Humanities instructor: That's not textual. F-

Re:Play favorites? I believe it (1)

professionalfurryele (877225) | about 3 years ago | (#37015750)

Ah yes, New Criticism, because Leavis and his idiotic ilk took a look at the downfall of logical positivism and said "if only our discipline could show such absurd hubris with even less justification". The second you make normative statements about criticism, is the second you look like a moron.

Before you mount a defence of your absurd position I have two words for you I consider highly appropriate given the context, assassination, Macbeth.

Re:Play favorites? I believe it (5, Interesting)

WindBourne (631190) | about 3 years ago | (#37014956)

Back at NIU, I had a lit class in which the female prof on the tenure track nuked all 3 guys in the class. I mean we scored D and F. ALL of the women scored much higher. I got fed up with this and one of my dorm mates gave a paper of his that had earned a A+ from the head of the lit program. It got a D-. After the semester was done, we took all of my papers including the purposely plagiarized one and went to the head. Showed it to him. Apparently, a major investigation was done, and she was released after that. My grades were adjusted up to a B after the head had re-graded all of the men's paper (I gave the head the paper that I had done and let him grade it).

Sometimes, things do work the way that it should.

Re:Play favorites? I believe it (2)

NecroPuppy (222648) | about 3 years ago | (#37015068)

I've seen worse, though not in Lit.

At Carolina, the Engineering program had a professor who, one semester, failed -everyone- ...except for the guy who was from the exact same part of India that he was.

Everyone there tromped down to the Dean's office and showed him the facts. They all got regraded, and the prof was not retained.

But worse than that was the Thermodynamics prof who graded entirely on the curve. As in, the Bell Curve.

The first test was six definitions and one problem involving steam. All but one person in the class used the Ideal Gas Law to solve it. And he marked us all wrong, because -he- hadn't taught us the Ideal Gas Law yet. (Never mind that you had to have two semesters of Physics to take Thermo...)

So that first test the class average was a 34 (the one guy pulled it up that high), with a standard deviation of around 17... so the 30s that most of us got was a... C.

Second test, the class average was a 92 with a standard deviation of around 19. So my 100 was a... C.

The final, the class average was a 100. There was no standard deviation. So my 100 was a... C.

Needless to say, a lot of pissed off people in that class.

Re:Play favorites? I believe it (0)

Anonymous Coward | about 3 years ago | (#37015270)

That reminds me of a physics class I took. Our professor was from Canada and would, at random intervals, switch to French. This physics course had no math prerequisites (despite the heavy use of trig and calculus) and the only one who had taken any was me. I did so well that with a zero as my final, I still got an A+; the one classmate who spoke French got an A- (if I remember correctly, but it was somewhere in the A range); everyone else got a C.

Well, everyone complained that the French guy got the only A (as I never discuss my grades and they all assumed I failed because I didn't show up for the final). The professor, in his defense, pointed out that I had the highest grade, so that it obviously wasn't favoritism. The dean required me to go to his office and then asked me if I spoke French too. I responded with, "no, but that is kind of secondary to the mathematics." Suffice it to say, no one was my friend in the second semester of that class as the grades were not adjusted; but, that course now has math prereqs.

Re:Play favorites? I believe it (1)

Wansu (846) | about 3 years ago | (#37015564)

At Carolina, the Engineering program had a professor who, one semester, failed -everyone- ...except for the guy who was from the exact same part of India that he was.

Do you mean South Carolina?

Not just favorites (1)

Nemyst (1383049) | about 3 years ago | (#37015174)

I had a teacher that was reviled by just about the entire campus such was her utter lack of competence. I had initially thought it would be possible to go through her exams by bullshitting, but then I stumbled on something that made me understand some of it.

I had an appointment with her for reasons I do not recall and she was grading exams still. As I waited, she started discussing with another teacher, careless about me and many other students listening in. After the discussion ended, she decided to raise some grades by nearly 10%! Thing is, she did not back up to recalculate exams she had already evaluated, she just changed the grades of a few students and moved on...

Considering her evaluation was based off a handful of vague keywords with grades written besides them (little to no useful margin notes or anything of the sort), you'd think she just rolled some dice and picked the grades from there.

And that's notwithstanding errors made by teachers that, if you don't press, can hurt a lot. I won as much as 20% on an exam due to faulty evaluation from careless teachers.

Re:Not just favorites (1)

mooingyak (720677) | about 3 years ago | (#37015644)

Not really related, but I remember a chem test where there was a 4 part question, with each part supposedly graded independently from the others. Each part was worth 2 points.I got the first part wrong, and then you need to use your answer from part 1 in part 2, and so on. Having the first part wrong, naturally the numbers written for the others parts were wrong, even though the steps taken to acquire those numbers were completely correct. He docked points for each part that had the wrong number as the answer. I went to him afterwards and asked what was wrong with parts 2 - 4, what should I have done differently to get full credit. He told me that he couldn't give full credit for wrong answers. So I asked him if part 1 was worth 5 points and all the rest worth 1, since that's what getting part 1 wrong cost me. We got into a fight over which got me nowhere but pissed off. The guy as OK overall, but he tended to do poorly with any alternative approaches to solving a problem. He'd give credit if it were right, but he tended to deride you for not going the "orthodox" route.

Re:Play favorites? I believe it (0)

Anonymous Coward | about 3 years ago | (#37015406)

As someone who made the mistake of taking an opinion on the GRE essay portion (which is what you are supposed to do on one of them), and who further made the mistake of having conservative views, I would welcome a computer that can grade essays (assuming they can actually make it work well).

Re:Play favorites? I believe it (1)

nbauman (624611) | about 3 years ago | (#37015740)

You learned an important lesson or two. Your TA was an asshole. Just because he's a teacher doesn't mean he's right. Cute chicks win (you did read Anthony and Cleopatra, didn't you?).

You wouldn't have learned that from a computer.

After school (4, Interesting)

digsbo (1292334) | about 3 years ago | (#37014870)

I had a prof in literature who only graded well if you made your critical essay about sexual imagery. At one point I gave up trying to "be me" and went whole hog, way overboard, almost parodying the over sexualized essay. And I scored an "A" for the the first time. Lesson learned? Sometimes it's OK to tell the boss what he wants to hear and do it his way, as long as it doesn't cost you anything, and nobody gets hurt. And, of course, life's not fair.

Re:After school (0)

Anonymous Coward | about 3 years ago | (#37014904)

And, of course, life's not fair.

That is, essentially, what the whole education system is about anyway.
It teaches you stuff, then tells you that you are useless and you won't need those skills because you will be forced in to a crap job, unless you have a good teacher and suck up to him/her.
Hurray for education!

Re:After school (-1, Flamebait)

0123456 (636235) | about 3 years ago | (#37015120)

That is, essentially, what the whole education system is about anyway.

No, it's about providing cushy, well-paid jobs for Marxists who would never get a job in the real world.

Re:After school (0)

omz13 (882548) | about 3 years ago | (#37015238)

No it's about employing hard-core environmental fascists who push their cause down your throat... even if the course you are doing has little to do with environment.

Re:After school (0, Funny)

Anonymous Coward | about 3 years ago | (#37014934)

Sometimes it's OK to tell the boss what he wants to hear and do it his way, as long as it doesn't cost you anything, and nobody gets hurt.

That's what she said...

Re:After school (5, Funny)

Opportunist (166417) | about 3 years ago | (#37014998)

The lesson you probably learned is that buttkissing gets you further in today's society than delivering good work.

Who said school doesn't prepare you properly for your adult life?

Re:After school (4, Interesting)

furball (2853) | about 3 years ago | (#37015172)

The lesson you probably learned is that buttkissing gets you further in today's society than delivering good work.

At the same time, if a customer tells you what features he wants and you keep not building it, are you surprised when he's unhappy with what you delivered?

Feedback is a valuable tool. What you do with said feedback is up to you.

Re:After school (1, Insightful)

Velex (120469) | about 3 years ago | (#37015294)

Ideally studying at a university isn't an exercise in delivering to spec. I'm not paying to produce a product for other people. I get paid to produce products for other people.

Eh, what the hell do I know. I stopped taking classes after I couldn't come up with a good way to echo the feminist sentiment that all women really do have penis envy and that I'm so lucky to have been born a guy without completely imploding. Somehow a body that causes you pain and discomfort one month out of the week is worse than a body that causes you pain and discomfort 12 times a day without ever letting up, and having a period is the equivalent of being raped. Oh, and long hair is the equivalent of being raped. So is putting on make up. In fact, just being female is like being continuously raped. At least that's all I learned in college.

Anyone whose knee jerks and mods me troll is actually modding feminism troll.

Re:After school (1)

Velex (120469) | about 3 years ago | (#37015308)

one month out of the week

Strike that. Reverse it.

I swear it was the right way around in the preview.

Re:After school (4, Interesting)

Phat_Tony (661117) | about 3 years ago | (#37015436)

Similar experience here. I got very good grades on my college papers. Later on I had a sociology class and got a bad grade on my first paper. I knew it was a much better paper than that. I talked to the TA and she tactfully went over some things I could improve that sounded mostly like BS she was trying to make up because she didn't know what she could tell me. After a pause I said "it is my concern that I can not get a good grade in this class without agreeing with the professor's opinions," and she replied:
"That would be my concern also."

On the first paper we were allowed to re-write it and resubmit it, and rather than picking a new topic, I simply re-wrote the same paper from the opposite stance, parroting back the professor's (in my view entirely wrong) opinions. I even included some egregious BS about how I'd learned so much and realized how right he was. I worried it might be over-the-top with the sarcasm, but I couldn't help myself. Anyone without an ego problem would have seen through it, that a college student isn't likely to have a total change of heart and (in this instance) change from being basically a libertarian to being a socialist overnight because their professor was so brilliant that they showed them the error of their ways. I was a little scared he was going to notice and call me into his office for submitting a sarcastic paper.

I got an A. The rest of the class was a disgusting piece of cake. There was no reason to bother with hard work, insightful points, and original analysis. It wasn't even necessary to read the material (although I generally did for my own benefit.) I just typed whatever opinions the professor espoused in class, with fidelity that was borderline plagiarism, and it was an easy A every time.

Accuracy (5, Interesting)

Wowsers (1151731) | about 3 years ago | (#37014880)

I can't say if a computer is better than a human at marking, but in my engineering subjects, when my name was on the test papers I did not get very good grades (actually at least grade lower than expected). But as soon as all the students were given anonymous numbers the grades went up. Conclusion, the staff could no longer decide to give better grades to their pet students. So in theory, there could be many students who get better grades because there is no more favouritism.

Anonymous numbers? (0)

Anonymous Coward | about 3 years ago | (#37015010)

That's a pretty good idea. Wish my teachers followed that in school. Then again they probably could have told it was me by handwriting, but I guess they use computers for that now.

Re:Anonymous numbers? (1)

Sir_Sri (199544) | about 3 years ago | (#37015352)

We tried to use student numbers on papers at one point at a school I was at previously. Unfortunately it became pretty easy to tie a student number to a student, especially in a small class or lab.

You could come up with a unique ID for each submission, or if all submissions are done online you don't even need that, it's just a bin of assignments to grade with names removed (they still have a unique ID in the database it's just not exposed to markers).

Though I think with essays you get to know peoples style. I suppose it depends on the essay.

Re:Accuracy (1)

TheRaven64 (641858) | about 3 years ago | (#37015488)

I had to mark some student essays this year and, in common with the rest of the coursework, even though it wasn't anonymous, I had no idea who most of the students were. Most of them had been in my lectures, but even most of the ones who had asked questions or come and talked to me after the class never actually told my their names, so they were effectively anonymous. I had no idea who the guy who got 99% was until he asked in the lecture why he had a mark deducted.

As to whether a computer can be more fair, if it can then you shouldn't be setting an essay. A computer can tell if you've listed a set of bullet points correctly, but it can't judge your understanding of the subject. For example, one of the titles my students could pick was 'Give five design patterns for concurrent programming and suggest when each would be appropriate'. Students got a reasonable mark if they showed me that they understood the materials I'd covered in the lecture. They got a really good mark if they showed me something I hadn't covered in the lecture and demonstrated that they understood it. How would you program a computer to make that call?

Re:Accuracy (1)

nbauman (624611) | about 3 years ago | (#37015858)

As to whether a computer can be more fair, if it can then you shouldn't be setting an essay. A computer can tell if you've listed a set of bullet points correctly, but it can't judge your understanding of the subject. For example, one of the titles my students could pick was 'Give five design patterns for concurrent programming and suggest when each would be appropriate'. Students got a reasonable mark if they showed me that they understood the materials I'd covered in the lecture. They got a really good mark if they showed me something I hadn't covered in the lecture and demonstrated that they understood it. How would you program a computer to make that call?

I agree. If you look at the example of computer grading in TFA, you'll see that it is in effect a set of bullet points.
http://chronicle.com/article/Can-Software-Make-the-Grade-/128505/ [chronicle.com] It's like a short-answer test with the numbers and paragraph marks deleted and run into a paragraph.

If you're going to use an essay-marking program like this, you might as well give short answer tests.

It's nice to get the date of birth and death in an essay, but today teachers are de-emphasizing facts that you can look up on Wikipedia in favor of more insight.

Lazy Professors - surprise surprise (0)

Anonymous Coward | about 3 years ago | (#37014884)

What's wrong your Grad Students too busy playing Farmville to grade papers?

Re:Lazy Professors - surprise surprise (0)

Anonymous Coward | about 3 years ago | (#37015332)

And once again the value of a degree dies just a little. Where was that article I read here recently about the masters degree being the new bachelor's? No, it's not ALL due to recession. But then again what do you expect from a nation of fat, self absorbed entitled pricks who just can't wait to get home to watch sit-com re-runs while binge eating? Sure, automate your grading and teach your students to pass the algorithm. In the meantime people in other countries will actually try to understand the subject material.

Re:Lazy Professors - surprise surprise (2)

Sir_Sri (199544) | about 3 years ago | (#37015432)

As a grad student. My supervisor isn't allowed to ask me to grade work for him, or prepare lecture material for him. Some of them do that, but we have a union that allows us to push back against it. I am paid as a teaching assistant as well (which is not guaranteed for all grad students), for that I am to work no more than 140 hours in a semester, and supposed to be roughly 10 hours per week. Part of being an instructor (which I have also done) is setting assignments that can be graded in the hours you have available to you. If you have 1 TA, yourself, and 25 students, don't set 100 page papers. I taught a graduate/4th year computer science course, so the TA needed to be trained up (that counts against his 140 hours), we had meetings he had office hours (all counts against his hours), and he took care of some stuff with IT (counts against his hours). In the end he had about 70 hours for marking. 5 assignments per student + an exam. So he had about half an hour per student per assignment to mark. That's neither good nor bad, it's just a matter of not setting material that cannot be graded that fast.

Professors are usually 40/40/20. 40% teaching, 40% research, 20% administration. Sometimes they are a bit more teaching or research. Of that teaching they usually do 4 or 5 courses, which then means they are supposed to be spending about 10% of their time on the one course you see them in. In practice it's more like 60/20/20 or 70/20/10 but depends on the department/school/personal ability etc.

Play favorites indeed (4, Interesting)

hal2814 (725639) | about 3 years ago | (#37014886)

My essay grades in college humanities courses were terrible until I started trying to figure out the political slant of my professor (or TA if the TA is the grader) and wrote papers supporting those views (and to be fair, those views weren't always left-leaning ones). I went from a C paper student to a low-A paper student in the blink of an eye.

Re:Play favorites indeed (2)

hort_wort (1401963) | about 3 years ago | (#37015204)

My essay grades in college humanities courses were terrible until I started trying to figure out the political slant of my professor (or TA if the TA is the grader) and wrote papers supporting those views (and to be fair, those views weren't always left-leaning ones). I went from a C paper student to a low-A paper student in the blink of an eye.

That sounds like an excellent humanities lesson in itself.

Re:Play favorites indeed (1)

Dunbal (464142) | about 3 years ago | (#37015358)

Eh, what difference does a grade make in the humanities anyway. You're still not getting a job at the end of it all.

Re:Play favorites indeed (1)

hal2814 (725639) | about 3 years ago | (#37015566)

So what degree did you get that didn't require humanities courses?

Re:Play favorites indeed (1)

Anonymous Coward | about 3 years ago | (#37015366)

That must have been rough - and they must have though you were crazy. One paper you'd be a left leaning atheist and then the next paper you were a hard Right Christian fundie? Maybe it wasn't because you found their political preferences.

I can just see it now:

TA: "Professor, I really think we must give him good grades! He's leftist one time, on the right the next, then back again .... I don't know!"

Prof: "Yeah, he's definitely unhinged! Start giving him 'B's and 'A's and lets hope he doesn't go off."

Playing favorites? ORLY? (3, Interesting)

DavidR1991 (1047748) | about 3 years ago | (#37014888)

Consistency is a fair point, but playing favorites? Isn't this what anonymous marking codes/IDs are for? (Or at least, that's what happens in the majority of universities in the UK)

Re:Playing favorites? ORLY? (1)

Tacvek (948259) | about 3 years ago | (#37015570)

In the US with the exception of tests/exams, it is almost unheard of to not have your name directly on an essay. A few Universities or programs may do that, but the vast majority do not.

On standardized tests/exams the essays never have the authors name.

On professor generated exams, the grader usually can see the name, but professors setting up exams such that graders do not know the student's name, while rare, is not unheard of. If such an exam contains an essay, then the essay would be graded without knowing the student's identity, (unless the identity can be inferred from the essay's content of course).

Re:Playing favorites? ORLY? (1)

hedwards (940851) | about 3 years ago | (#37015662)

It's a tough problem to solve. Ideally the instructor would be giving out rubrics when assigning the paper and sticking to it. At least that way folks have some meaningful idea as to what they need to do to get the grade they want. And can ask questions if they follow it and aren't given an appropriate grade.

The times that I get freaked out are when I've got an assignment due that's worth half my grade and the teacher hasn't bothered to spend any time explaining what exactly one needs to do to get a good grade.

Great idea (4, Interesting)

WindBourne (631190) | about 3 years ago | (#37014894)

but it really needs to check for plagiarism. I saw a load of it up at Colorado State.

In addition, it would ideally be able to handle lab books. I remember grading micro-bio 201 lab books back in the 80's, and I was getting tired after the first 30. The second 30 was a pain. The last 30, well, we finished the grading at a pizza joint over beer. I suspect that was how grade inflation happens.

Graduate Record Exam (2)

ub3r n3u7r4l1st (1388939) | about 3 years ago | (#37014910)

The GRE exam uses software to grade the essay portion for quite a while, along with a human grader. If these two scores different by a point or more, then it is forwarded to another human grader and the final score will be the average of the three entities.

That cuts the cost of running the exam, considering the cost of incurring an extra human grader.

It will soon pop up everywhere at university level, when the budget cuts are everywhere.

Re:Graduate Record Exam (1)

maxwell demon (590494) | about 3 years ago | (#37015826)

The GRE exam uses software to grade the essay portion for quite a while, along with a human grader. If these two scores different by a point or more, then it is forwarded to another human grader and the final score will be the average of the three entities.

So if the first human grader gives you zero points, the software gives you full points, and the second human agrees with the software, you still get only 2/3 of the points ...
I think it would be more fair to use the median of the three values.

easyer to cheat / keyword jam (0)

Joe_Dragon (2206452) | about 3 years ago | (#37014918)

easyer to cheat / keyword jam also auto grading systems can miss the point and fail a good paper or pass a bad off topic one.

Obviously? (1)

adosch (1397357) | about 3 years ago | (#37014936)

Hasn't that always been the case? I can recount dozens of personal examples in undergraduate/graduate (high school was too distant, sorry! But nor did I really take that seriously) where outside the multiple choice or true-and-false realm, there is always that element of human favoritism and non-neutral judgement involved. Certain people would get a lower/higher grade on a paper/research project that had really close ideology, thoughts or facts, that matched the next person (all cheating trolls stay in your cave). More of the educator's time is then spent 'justifying' their grade than the time it took to grade the item to begin with at that point, IMHO.

It would be a very logical feat to have a knowledgeable, computer system be educated enough to look at styles, patterns for topic(s) 'xyz' than it would would be worth just to remove the human judgmental element factor. IBM Watson [ibm.com] , I presume?

IBM system give a F for say toronto is in canada (2)

Joe_Dragon (2206452) | about 3 years ago | (#37015024)

And the IBM system give a F for saying Toronto is in canada. I say let the computer help but make it so there is no AUTO FAIL and make a real person review at least some flagged papers.

no it cant (0)

Osgeld (1900440) | about 3 years ago | (#37014962)

My wife just got done with a course, and it was online, it graded her wrong cause she clicked the edge of the button instead of the dead center. it doesn't matter she clicked the right button, it only matters that she did not click it DEAD PERFECT CENTER

so no these systems cant grade better cause they are made by teachers who don't proof read their own shit (for example there was many times she would be taking a quiz and the answer would be in 2 chapters ahead) made by flunkie CS 101 students that just got introduced to the concept of a bounding box via web image mapping.

Re:no it cant (0)

Anonymous Coward | about 3 years ago | (#37015212)

Idiot

What's up with the mass media headlines? (5, Interesting)

Co0Ps (1539395) | about 3 years ago | (#37014964)

What's up with the mass media headlines? Reading the summary actually makes me dumber. It talks about "computers" like they are sentient and grades the tests instead. Having professors first strictly defining the rules, entering them into software and having a computer evaluate those rules is still "professors grading the essays". It's self evident that the grading is better if it's more strictly defined.

Wow, I can build a house faster with this hammer. Headline: Hammers Could Build Houses Faster Than Construction Workers (In Cyberspace)

Fairer vs. Better? (4, Insightful)

fuzzyfuzzyfungus (1223518) | about 3 years ago | (#37014978)

Unless they've made some impressive advances in natural-language interpretation in the past few years that haven't trickled out into other products, I'm a bit puzzled as to how this scheme is supposed to work.

Even the (comparatively much easier) tasks of spelling and grammar checking result in a fairly steady stream of mistakes from computer systems. I can't exactly summon much optimism for the likely outcome of such a system trying to distinguish between a paper with a well supported thesis and a paper that contains some declarative statements, a few quotations, and the word "therefore" at intervals.

On the plus side, it should be pretty trivial to get the machines to do the same lousy job without the slightest consideration of the student's name/status/cuteness/willingness to flatter the professor; but what use is purely objective execution of lousy work?

Re:Fairer vs. Better? (0)

Anonymous Coward | about 3 years ago | (#37015776)

If you look at the examples in the link provided in the TFA, the computer is just doing buzzword analysis of the essay.

Re:Fairer vs. Better? (1)

fuzzyfuzzyfungus (1223518) | about 3 years ago | (#37015900)

Bloody hell. If I were paying for college and learned that Dr. GREP, LSB, POSIX. was doing the grading of my work I'd be seriously pissed.

An essay graded by buzzword analysis is really just a fill-in-the-blanks where you have to supply your own connective grammar...

more classes need to move away from the written te (1)

Joe_Dragon (2206452) | about 3 years ago | (#37014984)

more classes need to move away from the written test and to a more hands on / maybe even no test class.

That fixes 2 things the people who just cram for the test and pass but I have little to no idea about the content or how to use it. As well as people who know what they are doing and are bad test takes / not that good at witting essays. Also cut's down on the people who pay for paper / essays witting services.

Re:more classes need to move away from the written (1)

digitig (1056110) | about 3 years ago | (#37015478)

more classes need to move away from the written test and to a more hands on / maybe even no test class.

Good luck implementing that approach for journalism and creative writing courses! In some fields being able to write well is a vital skill. Even in sciences and engineering, the person who can communicate their ideas clearly and persuasively in writing will have a big edge over one who has similar technical skills but can't write well.

Re:more classes need to move away from the written (1)

hedwards (940851) | about 3 years ago | (#37015700)

You've got that backwards. Written tests are of absolutely no value in Journalism or art. You might do a quiz covering something like ethics, but in terms of the actual practices you're not going to test that in any sort of useful way using a written test.

Another Grad Student (1)

portraitofsanity (870052) | about 3 years ago | (#37014996)

The grading work in sciences (chemisty) can be extremely easy or hard. If you figure it partial credit and do all the calculations with their number they screwed up on step 2. With teaching load at some universities, this is not a viable approach (yes spreadsheet this, do that, everyone finds a different way to mess doing things up). My grades tended to rise almost a letter grade as I got towards the end and would basically just check that they had it within an order of magnitude (Freshman classes, easy reports). The 20 page monstrosities I was guilty of scanning the "smarter" students finding an error giving them a 95 and let it be, I admit. I never targeted anyone for lower grading but almost the opposite effect of the freshman thing had happened as each lab is more or less the same and I'd peruse the smart kids first, the concept(s) they skipped/didn't understand were obvious, so the last student far far more likely to get a bad grade.

Too Late (2)

sgt scrub (869860) | about 3 years ago | (#37015032)

As someone who never effected the curve or caught the affection of a teacher, I welcome our new digital grader overlords.

Writing for your audience just got harder (1)

Opportunist (166417) | about 3 years ago | (#37015050)

Let's be honest here, everyone had at least one teacher where it was obvious what you had to argue for or against to get a good grade. My German teacher was an ex-army officer. Take a wild guess what position you should take when the topic is the role of the armed forces in the history of the nation.

It was easy to get a good grade. Why? Because you knew, no matter how harebrained or outlandish your arguments were, if it was what he wanted to hear, a good grade was your reward. Simply and plainly. Once you learned that it's not your job to argue with your teacher but to write what he wants to hear, you have a much easier life.

And it transfers to work as well. My life was a living hell while I tried to achieve security when everyone just wanted compliance. Since I dropped the idea that we should be secure, everything's running a lot smoother, everyone's happy and when (not if) the shit hits the fan, I even get to say "told you so".

Shallow accuracy only (0)

Anonymous Coward | about 3 years ago | (#37015058)

While you can get better than blind luck accuracy out of robo-graders, they are still easy to fool. They can play with coherence, but none of them has a chance at the semantic level, so you can replicate basic essay structure with nonsense content and they won't know the difference.

A robo-grader that grades only based on length is nearly as good as a human grader, and takes far less investment.

Post-structrualism (0)

Anonymous Coward | about 3 years ago | (#37015102)

I would love to see this grade a literary essay, especially a post-structuralist one, something that takes creative thought to comprehend.

Wasn't Aware (2)

paleo2002 (1079697) | about 3 years ago | (#37015134)

I wasn't aware we now have access to AI this advanced. Spell check and (maybe) grammar check are reasonable, but how does a computer assess a student's understanding and mastery of a topic? How does the computer recognize originality, creativity, or intuitive leaps? Can the software recognize an effective argument, a convincing solution?

I'm a geology and earth science professor. When I give writing assignments, I'm usually more interested in the content than the mechanics. I'll tolerate a few spelling and grammar mistakes if the content of the essay or paper demonstrates that the student understands concepts presented in class and, even better, is THINKING about the implications.

For intro. writing classes, where grammar and structure are the point of the assignment, computerized grading is understandable; especially if your school has you teaching classes with more than 50 students (which is another issue entirely). But, in my experience at least, proficiency at writing is not always directly correlated with proficiency at class material.

Re:Wasn't Aware (4, Informative)

tgv (254536) | about 3 years ago | (#37015206)

I'm pretty sure no program is capable of this (and I've got a PhD in natural language processing). They might be able to check for a couple of easily scored factors, such as number of words, and consistency between paragraphs, but I'm pretty sure that there is no program that could distinguish between an essay and the same essay messed up to base reasoning on false assumptions. I think someone left out a pretty important assumption: such programs might be able to score fairer (meaning: with less bias!), provided the students did their best.

Re:Wasn't Aware (2)

Animats (122034) | about 3 years ago | (#37015788)

Take a look at SAGrader [sagrader.com] and see what it is doing. It's not grading "essays". It grades answers to narrowly focused essay questions. It's looking for key phrases. The student''s correct strategy is thus to repeat, exactly, the language of the textbook.

I would believe that the grades are more objective (2)

brokeninside (34168) | about 3 years ago | (#37015148)

But the objectivity of the grades has nothing to do with the problem of grade inflation. Professors intent on inflating grades will simply reduce the weight of tests as part of the overall grade and count class participation, homework, etc. more /or/ add a flat number of points across the board to the results of the computer scored tests.

Grade inflation, after all, isn't simple bias. We're not speaking of professors grading up people (or views) that they like and grading down people (or views) that they dislike. Rather we're speaking of professors that systematically give higher grades than they ought for one reason or the other. Some do this for ideological reasons. Others do it because they're tired of fighting students (or parents) that complain. The end result is an 'A' no longer means 'excellence in performance' but is pretty much the default grade for anyone that do a moderate amount of work.

Doesn't help grade inflation (2)

jaroslav (467876) | about 3 years ago | (#37015184)

I'm not arguing that this is a good or bad idea, but it won't do anything to change grade inflation. In my experience (as a TA for a number of different classes), college professors look at the point totals at the end of the semester and determine the letter grade cutoffs by hand so that they have the grade distribution they want. I'm not saying they're going through and making sure specific students get a particular grade, just that they want, say 50% A's 30% B's and 20% C's and they'll put the cutoffs where they need to be for that to happen. Just because the essays are graded tougher doesn't mean they can't still give half the class an A.

Robotic Teachers - Robotic Students... (2)

Lohrno (670867) | about 3 years ago | (#37015216)

AIs to grade the papers I would assume would result in some folks developing AIs to create the papers...

Complete nonsense (0)

Anonymous Coward | about 3 years ago | (#37015218)

You only need to look at the job done by automatic translators to see how primitive current systems are at understanding the structure of a sentence, let alone its meaning, or how it relates to other sentences.

Automated grading systems look for keyword and "key statements". If you write A=B in one sentence and A!=B in another, an automated system will tend to either grade that as "correct" or, at most, "neutral". A human grader will (correctly) evaluate it as "worse than wrong", because giving the wrong answer could be the result of a simple mistake, but giving two contradictory answers is proof that the author didn't even understand the basic concept he's talking about.

The reason why grades (especially college grades) are inflated is that education is big business. Flunk a student and he might leave college. Pass a bad student and your college gets next year's tuition money. I give seminars as part of a couple of graduate courses, and I've been pressured to do just that (though I never caved in).

The only way to fix education is to make sure that every student pays the same and every teacher is paid the same. It's the same basic principle applied to scientific studies (eliminate all other variables so you can focus on the actual performance of each student and teacher), but somehow people think it's "evil communism"...

I was one of those graders (4, Interesting)

artor3 (1344997) | about 3 years ago | (#37015256)

First, for those who didn't read TFA, computers play only a small role on a handful of essays. Most of the article is in reference to having a 3rd party grade anonymized tests, rather than leaving it to the professor or TA. During college, I had a job as one of those graders.

We worked for five hours a day in the evening, though we could leave early and get the full pay if we finished all our papers. Most of the tests would be on general topics, but occasionally we'd get tests that required specific knowledge. In those cases, only qualified graders could review them, and we were given cheat sheets to make sure we didn't make factual mistakes. Essays were generally graded on a 1-5 scale (or a 0 if the essay was a blank page or similar). Each essay would be graded by two people, with a third breaking the tie in the event of a disagreement. However, we trained to be extremely consistent in the grading, so disagreements were rare and never more than a one point difference.

A few times a day, we would get fake essays intended to test our grading skills. For example, an essay that was supposed to be a perfect example of a 4 would be given to you with all the rest. If you gave it a 4, you get +1 point. Give it a 3 or 5, you get zero points. Give it a 2 or less, and you lose a point. If you accumulate a lot of points, you get a bonus up to 50% of your pay. If your total score goes too negative, you get fired.

It was a pretty good job, as crappy part-time "work your way through college" jobs go. The best part was whenever we got to grade essays by little kids. They were harder to score accurately -- it's hard to look past the abysmal handwriting and frequent misspellings. But they were frequently adorable and unintentionally hilarious.

Re:I was one of those graders (0)

Anonymous Coward | about 3 years ago | (#37015908)

I would like that job, as far as crappy part-time "work my way through college" jobs go :)

Fully automated grading is (still) impossible. Semantics.

Bias (1)

p51d007 (656414) | about 3 years ago | (#37015316)

Most likely due to the fact that HUMANS, no matter how hard they try, will be biased in reviewing. Most college professors are liberal. Not saying that is a bad thing or a good thing, it is just the reality of the situation. If a professor is grading papers and comes across some student that instead of keeping quiet about his/her political inclination, is outspoken in class, that teacher will even if they try not to, will be harder on a conservative student, than a liberal student. It's just human nature. A computer on the other hand, will not have that problem.

It comes down to feelings (1)

NicknamesAreStupid (1040118) | about 3 years ago | (#37015318)

Have written a few things, not including all the crap I spew here, I can tell you that any judgements beyond syntax, grammar, and semantics are purely emotional. That is because all human reasoning boils down to feelings. If that bothers you or intrigues you or bores you, then there you go.

When machines are used to judge people's essays in subjective ways, either their state machines will be unemotional and miss much of the contextual meaning of writing, or they will be patterned after some designers' emotional criteria or sets of emotion-like heuristics and largely be non-evolving until their algorithms improve to adapt as we do. The latter would probably violate at least one of Asimov's three laws, if not all of them.

Slashdot (0)

Anonymous Coward | about 3 years ago | (#37015346)

Slashdot: stop deleting my comments, I'm bringing up fair points. I am a lit major and these are things I've gone through and reasons why I'm skeptical about a computer grading any assignment for anyone in my major as any literary criticism doesn't require merely an understanding of grammar, syntax or algebraic relationships of ideas, but being able to comprehend the synthesis of (often times unrelated) ideas.

Here's one example, that I will re-post. Last year a friend of mine in gothic literature wrote an essay on The Monk, for which he received an A. His thesis was that by the end of the novel, the reader "becomes" Ambrosio, the character who narrates, while the real "narrator" of the story is the main temptress/antagonist; which implies that this temptress has more critical distance and knowledge of the actual story than Ambrosio, the supposed narrator.

Here's another, more common, example. One scholarly essay we had to read described the function of the veil in Ann Radcliffe's novels. While I can't remember every facet of this essay, a point I remember was that the veil was used as a way to build temporary tension between superficial and obscured "truths" (ie a dead body), which was really among the first times that an author started to use a veil in this manner (historically).

I'd really like to know how a computer would be able to see the synthesis, let alone appreciate it, in essays like these. I just do not believe it can. It would require something close to actual artificial intelligence, which, to my knowledge, we do not have. This is not like Watson, who hears a word, looks it up, and repeats its most common factor; this is not like spell-check, that compares a database of spelling and grammatical rules to a paper and points out inconsistencies between that paper and the database. This requires deep thought.

If these machines CAN do just that, then I will be taken aback.

Are grades really meaningful? (1)

FoolishOwl (1698506) | about 3 years ago | (#37015374)

My partner is just starting an MA teaching program, and she's been ranting a lot about the utter uselessness of grades and standardized testing. Apparently, there are decades of research establishing that standardized tests fail to measure anything but performance on standardized tests, and grades measure little besides conformism, self-discipline, and a lack of creativity. (And self-discipline is not always a good thing [alfiekohn.org] -- why are you working so hard at doing things you don't really believe are worth doing?)

My first reaction to the headline was that, if computers are better at grading than people, and we know many of the essays are plagiarized from essays found through Google, why have any human participation in the process?

More seriously, one learns to write well through reading a lot, writing a lot, and occasionally listening to criticism. I think we'd do a better job teaching writing by having students in a class read each other's writing and make comments, and simply pass those who participate and fail those who don't, with no further assessment than that.

Of course, that presumes an education system designed to help people learn to become fully participating members of a community and to lead rich, fulfilling lives. As things stand, mass education systems seem designed to produce some dubious justification for burying most people alive, while selecting a conformist and quiescent minority for middle-class careers.

Re:Are grades really meaningful? (1)

ADRA (37398) | about 3 years ago | (#37015676)

So your ultimate solution is what now? I mean the sun causes cancer, but unfortunately for everyone, its essential to live. Why is this relevant? Well, if standardized testing and general schooling is so bad, what do you presume we replace it with? Rich people could get personal tutoring by experts in fields that they find meaningful. The rest of us should just learn the job our parents taught us because we get really really good at that one thing. Globalization would die because hell, if you can't generally gauge the performance of someone from half way around the world, why would I want to hire them? There are dozens of situations where the lack of standard testing would cause the end of the known world, but I'll save my breath.

If your teacher friend thinks that standardized testing is so broken, then they should get off their asses and come up with something better. I think you'll find a large track record of people trying to make changes here or there in order to make things better, and I can't see why your friend couldn't do the same.

Re:Are grades really meaningful? (0)

Anonymous Coward | about 3 years ago | (#37015712)

>Apparently, there are decades of research establishing that standardized tests fail to measure anything but performance on standardized tests, and grades measure little besides conformism, self-discipline, and a lack of creativity.

Absolutely true. And yet, governments around the world continue to rely on them as their number one indicator of student progress.

>More seriously, one learns to write well through reading a lot, writing a lot, and occasionally listening to criticism. I think we'd do a better job teaching writing by having students in a class read each other's writing and make comments, and simply pass those who participate and fail those who don't, with no further assessment than that.

You just described my junior and senior high classes. At the beginning of the term, students are told that they need to complete five types of assignments: short story, creative non-fiction article, persuasive speech, persuasive essay, and term paper. Beyond that, they choose the text to use in their assignment and they choose when to hand it in.

Students type up assignments on a forum I've set up on a private server over the course of a semester. Other students peer edit. All of this is anonymous, as much as working with junior/senior high students *can* be anonymous. Once the semester is done, I sit down with each student and all the work they've done and negotiate a final grade - no piecemail grading of individual assignments, but an overall grade that we're both happy with. Grades are meaningless to me: I can see what they've done without needing to assign a two-digit number to each assignment. Some students care about their grades, as they need a specific number to get into university, so that drives them. Others could care less about grades, and for a lot of them they're just enjoying themselves. The difference between "I give an assignment, you do the assignment" and my current system is night and day - kids actually care about what they're doing because they're writing for each other instead of writing for me. They take risks and come up with some beautiful pieces of writing.

So, tell your partner that she's on the right track, and there are others out there who agree with her. Have him/her check out the Cooperative Catalyst (http://coopcatalyst.wordpress.com/) -- more educators out there who are sick of teaching like it's 19th century Prussia and who are trying to make education more meaningful.

Revealing (0)

Anonymous Coward | about 3 years ago | (#37015394)

This is a very revealing post.

I had good grades and no complaints, but now I write poorly in a mechanical sense, but far better in a wisdom sense.

This post gives the impression essays are strictly mechanics, which would be a horrible inditement of education!

Who's smarter, someone sharp with arithmetic or calculus! This is rediculous.

It must be a neural net or similar algorithm with a training set. Boy, what a sorry state we're in if a unforeseen insight on an essay problem cannot win beaucoup points.

No number of unoriginal ideas can add up to a original idea in real life. There are tons of people who just don't grasp the concept of why the music band with a new sound wins and the copycats don't. All the copycats are walking around thinking life is not fair.

worthless essays (1)

sustik (90111) | about 3 years ago | (#37015462)

Based on what I know about the current state of AI, the essays that computers can grade fairly are NOT worth writing.

Students and professors should try doing something useful... Something that pushes human intellectual boundaries and imagination, explores human emotions, discusses ethics and moral issues. These are not topics that today's artificial intelligence can handle.

How are professors supposed to get laid... (0)

Anonymous Coward | about 3 years ago | (#37015496)

If they can't improve the grades of the cute coeds????

From Degrading to De-Grading by Alife Kohn (1)

Paul Fernhout (109597) | about 3 years ago | (#37015730)

http://www.alfiekohn.org/teaching/fdtd-g.htm [alfiekohn.org]
"... The preceding three results should be enough to cause any conscientious educator to rethink the practice of giving students grades. But as they say on late-night TV commercials, Wait -- there's more. ..."

Key points:
1. Grades tend to reduce students' interest in the learning itself.
2. Grades tend to reduce students' preference for challenging tasks.
3. Grades tend to reduce the quality of students' thinking.
4. Grades aren't valid, reliable, or objective.
5. Grades distort the curriculum.
6. Grades waste a lot of time that could be spent on learning.
7. Grades encourage cheating.
8. Grades spoil teachers' relationships with students.
9. Grades spoil students' relationships with each other.

This is too simplistic. (2)

Dr_Ish (639005) | about 3 years ago | (#37015756)

As a professor, I can attest that the diagnosis of the problem here is too simplistic and the proposed 'solution' here is unnecessarily complicated. While it is the case that TAs and insecure professors will often inflate grades as they are scared of student appeals, the solution is to employ most experienced professors. There are also relatively simple methods that can be used to prevent grades becoming skewed. For instance, it is easy to grade anonymously. Just ensure that identifying details only go on the first page and turn the work over and grade from the back. One can also compare class mean and median scores (and SDs) with the scores from other sections of the same class. Such methods can ensure fair and consistent grading, without grade inflation. I always use such methods to great effect.

what about staff evaluation schemes?? (2)

nerdyalien (1182659) | about 3 years ago | (#37015772)

Aren't they a culprit too in grade inflation debacle ???

I was a TA in a far east university in an Engineering department. Generally I consider my self a tough marker, as I expect students to arrive at answers with right logical reasoning. Having said that, I usually had a partial blind eye for students who has genuine drive towards studies -- post grad research types --, because their future shouldn't be eclipsed by a one bad grade. Also I highly control the grade distribution, such that only 5-10% of the class will get A-grade.

First time when I marked the maths assignments, the feedback was horrible. I was told off by the lecturer for marking strictly, and then he increased marks of everybody by some percentage. Then I was instructed "not to go through the workings" and "give full marks if you see the answer". Since then, more than half the class gets A-grade.

The problem here is, lecturers are evaluated every semester by handing out questionnaires to students (in that university). Bad feedback can kill lecturer's x-mas bonus to getting a promotion in the department. So him (and many others) end up pleasing students not to hurt his career as an academic.

On a separate note, most of engineering course work are now done in software level. As a consequence, hardly any hardware related experiments and report writing. Downside of all this is, it is impossible to catch plagiarism; as all experiments in a software produces same outcome, more or less. Unless all students get it wrong, everybody ends up getting A-grade.

In my time, all course work (labs, assignments) has to be submitted as a report. Highest I ever got was 8/10... mostly 7/10. In one assignment I submitted, marks were slashed for no zooming in a graph (still it covered 90% of the page). In another report, few marks were removed for not using a ruler to draw a circuit diagram. Having few bad grades eventually costed my first class, which became a major issue in my post-grad entry. Considering those days, I think college kids are having easy time now. In a way, I can understand why people in the working world pay little to no attention on college performance.

Next step... (0)

Anonymous Coward | about 3 years ago | (#37015844)

Why stop at computer grading! Its time to make the computer do all of our writing too!

Life skills lost? (0)

Anonymous Coward | about 3 years ago | (#37015912)

In the real world picking one's battles and tailoring one's message to match the expectations of one's audience is often more important than the content itself.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>