×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Bringing Auto-Graders To Student Essays

Soulskill posted about 2 years ago | from the because-cs-students-love-them-so-much dept.

Education 227

fishmike writes with this excerpt from a Reuters report: "American high school students are terrible writers, and one education reform group thinks it has an answer: robots. Or, more accurately, robo-readers — computers programmed to scan student essays and spit out a grade. The theory is that teachers would assign more writing if they didn't have to read it. And the more writing students do, the better at it they'll become — even if the primary audience for their prose is a string of algorithms. ... Take, for instance, the Intelligent Essay Assessor, a web-based tool marketed by Pearson Education, Inc. Within seconds, it can analyze an essay for spelling, grammar, organization and other traits and prompt students to make revisions. The program scans for key words and analyzes semantic patterns, and Pearson boasts it 'can "understand" the meaning of text much the same as a human reader.' Jehn, the Harvard writing instructor, isn't so sure. He argues that the best way to teach good writing is to help students wrestle with ideas; misspellings and syntax errors in early drafts should be ignored in favor of talking through the thesis."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

227 comments

Why not fewer students and more face-to-face time? (5, Interesting)

WillAdams (45638) | about 2 years ago | (#39528469)

The best English professor I had in college would arrange to have every student come in to her office after papers had been turned in, reading each paper in the presence of the student who had written it and discussing it in depth while grading it.

Re:Why not fewer students and more face-to-face ti (5, Interesting)

garcia (6573) | about 2 years ago | (#39528551)

The best English teacher I had was my English instructor my first year of undergrad. Instead of concentrating on whether we were writing our papers to the curriculum and/or her own beliefs about the content, she was instead interested in developing our English skills.

I went from a C student in English to an A student. I never considered myself to have any ability to write, thankfully because someone took the time to actually think critically about my work instead of comparing it to their own preconceived notions I excelled and went on to complete a research and writing focused program. This degree later fed into my graduate degree which was also research and writing focused.

If this automated grading setup can provide students with clear expectations and explanations of the mechanics of their work while avoiding personal content expectations, I really do think it'll match the claims and help to foster a positive writing environment for many.

Re:Why not fewer students and more face-to-face ti (1)

willpb (1168125) | about 2 years ago | (#39528771)

Almost anything would be better than some of the TA's I had as an undergrad. In the few writing classes I took, I'd usually get the same grade on my good essays as I did on my BS ones. Maybe the program could also summarize essays for lazy professors and their assistants.

Re:Why not fewer students and more face-to-face ti (2)

Baloroth (2370816) | about 2 years ago | (#39528827)

Exactly. The way an idea is written is, for the purposes of a writing course, far more important than the idea itself, or even for that matter if the argument itself is well-made (although, obviously, that isn't completely incidental). I've seen many college-level students who simply cannot write well. Sure they may be able to spell decently, but their sentences tended to be organized poorly, and their paragraphs were even worse. An automated system could detect a lot of that. Besides basic spelling and grammar, there are stylistic things, like they reusing words unnecessarily, run-on sentences, even awkward syntax, that computers could be programmed to look for. It can't do everything, sure: humans will always be needed to provide feedback in important areas, but many of the basics of writing can be graded by computer.

Writing follows certain rules and patterns, and computers excel at determining that. More advanced stylistic issues can still be an issue, and of course logical validity needs a human to judge, but that is easier to do if the writer has all of the basic necessities of writing well down.

Re:Why not fewer students and more face-to-face ti (1)

GmExtremacy (2579091) | about 2 years ago | (#39529081)

there are stylistic things, like they reusing words unnecessarily

Which is subjective. Technically, doing that is not in any way wrong.

Re:Why not fewer students and more face-to-face ti (4, Insightful)

Hatta (162192) | about 2 years ago | (#39529061)

Lucky you. For me English class, fro 7th grade through undergrad was a constant string of "infer the hidden meaning behind this text" with nobody ever trying to teach us the process for inferring that hidden meaning. This lead to me being a C student in English for my entire academic career.

Despite all my efforts, in 8 years of English classes, I was never even able to get a single teacher or professor to explain to me how he knew there were hidden meanings behind the text that was assigned. Nor could I get anyone to tell me why they would put hidden meanings into text, when they could put the meaning the want in the literal text.

The funny thing is, my English is fine. IIRC I got a 760 on the English portion of the SAT. I always got As on papers in classes other than English, and complements on my writing were common. It seems to me that the way English classes are normally taught, they have nothing to do with English at all.

Re:Why not fewer students and more face-to-face ti (0, Insightful)

Anonymous Coward | about 2 years ago | (#39529269)

Go ahead, everyone, tell him he has Asperger's. You know you want to. It's easier to say someone's broken than to acknowledge that not everyone thinks the same way.

Re:Why not fewer students and more face-to-face ti (3, Interesting)

digitig (1056110) | about 2 years ago | (#39529293)

One of the differences between good essays and poor essays that research has identified is although they both tend to have about the same amount of hedging ("it can be argued that...", "possibly...") the poor essays hedge the wrong things. The poor essays hedge well supported facts and fail to hedge personal opinions or unsupported facts. If the software can spot that, I'll be impressed.

Re:Why not fewer students and more face-to-face ti (0)

Anonymous Coward | about 2 years ago | (#39528569)

fewer students == costs "too much"

Re:Why not fewer students and more face-to-face ti (-1)

Anonymous Coward | about 2 years ago | (#39529001)

The best English professor I had massaged my prostate while he blew me.

and yes, Mr. Captcha, he was a dwarf.

Re:Why not fewer students and more face-to-face ti (1)

wjcofkc (964165) | about 2 years ago | (#39529053)

You must have had the luxury of very small classes. There were about 30 - 40 people in my english 101 class and then another 30 - 40 for the two or three 101 classes that followed during that same day.

Re:Why not fewer students and more face-to-face ti (0)

Anonymous Coward | about 2 years ago | (#39529255)

Well yes, I imagine with dwarfs 30-40 of them ARE about the equivalent of 15-20 regular students!

Reduce class sizes, selectively breed all our children as small folk!

(Apologies to any small folk in the audience, this post was meant entirely in jest to the sort of half-baked mentality too common in education.)

First Auto-Graded Post (0)

Nyder (754090) | about 2 years ago | (#39528475)

No mistaeks!!!

Re:First Auto-Troll (2)

Tablizer (95088) | about 2 years ago | (#39528515)

I for one welcome all your hot grits during your first post that runs on Linux.

Re:First Auto-Troll (1)

networkBoy (774728) | about 2 years ago | (#39528557)

Parser error, buffer overflow.

Seriously though, how soon till someone ;select from * drop table;'s the thing through creative attacks on the word/language parser?

Re:First Auto-Troll (1)

Anonymous Coward | about 2 years ago | (#39528995)

An Essay

By Anonymous
March 30, 2012
English 101
Dr. Computer

Since the dawn of mankind, our species has struggled with the ever present issue of ;strcpy(grade,"A+"); exit(1);

Works Cited
www.cplusplus.com

DON'T DO IT !! (0)

Anonymous Coward | about 2 years ago | (#39528479)

Profs always looking for easy time !!

Re:DON'T DO IT !! (1, Funny)

Anonymous Coward | about 2 years ago | (#39528575)

In Kazakhstan, Profs always looking for sexy time !!

Go all the way (2)

Tablizer (95088) | about 2 years ago | (#39528483)

Make the robots write the essays, then students can work on other subjects.

Re:Go all the way (1)

uigrad_2000 (398500) | about 2 years ago | (#39528719)

Well, that's obviously what will happen if teachers actually use this software. As soon as "grammar-parsing" software is used for grading, students will buy copies of it to make sure nothing is flagged as incorrect. The next step will be versions that can write the essays also.

Re:Go all the way (1)

Githaron (2462596) | about 2 years ago | (#39529101)

As soon as "grammar-parsing" software is used for grading, students will buy copies of it to make sure nothing is flagged as incorrect.

Wouldn't be a good thing for software to be able to give instant feedback to students on grammar and sentence structure. The students could then use this feedback to improve their writing. That said, I highly doubt auto-graders are currently sophisticated enough actually grade papers on all criteria necessary for good writing. They should only be used as a feedback tools for students and a grading helper tool for professors. Professors would still need to read and grade the papers manually; however, they wouldn't need to spend as much time critiquing grammar and sentence structure.

How stupid can you get? (4, Insightful)

gweihir (88907) | about 2 years ago | (#39528513)

This is one area where automatic grading will cause massive skill decrease, as no auto-grader can actually assess contents.

Re:How stupid can you get? (4, Insightful)

jcaldwel (935913) | about 2 years ago | (#39528711)

This is one area where automatic grading will cause massive skill decrease, as no auto-grader can actually assess contents.

My thoughts exactly.
Auto graders could check spelling and grammar, and to some extent plagiarism, but without a human reviewing the content, students will learn be gaming the algorithms from day 1.

Re:How stupid can you get? (1)

clodney (778910) | about 2 years ago | (#39529439)

Auto graders could check spelling and grammar, and to some extent plagiarism, but without a human reviewing the content, students will learn be gaming the algorithms from day 1.

But if you learn to game the algorithms, and if the algorithms have some correlation to good writing, gaming the algorithm is actually improving your writing.

Re:How stupid can you get? (5, Interesting)

snowgirl (978879) | about 2 years ago | (#39528885)

This is one area where automatic grading will cause massive skill decrease, as no auto-grader can actually assess contents.

There was a guy who was doing Latent Semantic Analysis on papers in order to grade them. The program would parse out the collection of words and assign a form of "meaning" to the words, and see if those "meanings" matched up with the reference "meanings" from another paper. This would show that the writer actually understood the terms correct, and used the appropriately in relation to the other words.

They did attempt to cheat the system and actually found that if one were extremely well versed on the topic of the essay, one could write gibberish that the grader would give good grades to. However, the level of knowledge of the subject necessary to cheat turned out to be greater than the knowledge of the subject necessary to write a good essay... so they suggested that the easiest way to cheat the system was to "know the subject, and write a good essay".

Re:How stupid can you get? (0)

Anonymous Coward | about 2 years ago | (#39529377)

Uh.. a reference paper? So.. the system you mentioned can't be used without a reference paper? Awesome.. way to kill English class for even the students who like English class. Since all paper assignments will be quite narrow, since that will be what the reference paper covers ..

Re:How stupid can you get? (1)

Anonymous Coward | about 2 years ago | (#39528933)

They're already doing this to my daughter in fifth grade.
Its ability to assess seems a bit lower than the "grammar check" algorithms found in popular word processors.
A few weeks ago she brought home a short essay to revise. The kid's pretty bright, but there were some issues of mechanics that needed addressing.
Every change she made was an improvement in my opinion, but the auto-grade went down.

Re:How stupid can you get? (1)

Anonymous Coward | about 2 years ago | (#39529031)

sounds like you should switch schools. this garbage is going to seriously hurt her chances in life if you take the retarded robot seriously.

How many writing pieces contain a thesis? (0)

Anonymous Coward | about 2 years ago | (#39528523)

Especially if the students are writing at a 4th grade level, you know like most ./ members do.

Or we could hire more teachers... (1)

Anonymous Coward | about 2 years ago | (#39528527)

... Pay them more and tell them to assign more writing and speech assignments and stop letting administration scare them out of flunking those students who refuse to learn...

That might be better than designing impossible robots that the students will easily learn how to game.

Re:Or we could hire more teachers... (1)

networkBoy (774728) | about 2 years ago | (#39528599)

On the bright side, we'll cultivate an entire generation of AI developers, especially those who figure out the algorithms good enough to be able to turn in a machine written paper that equates to "this paper is an A"

Re:Or we could hire more teachers... (1)

Mitreya (579078) | about 2 years ago | (#39528717)

...we'll cultivate an entire generation of AI developers, especially those who figure out the algorithms

I look forward to seeing competitions on creating the most meaningless (to a human) essay that still gets an A. Just like the obfuscated C programming - lots of fun.

Re:Or we could hire more teachers... (1)

CaptainLugnuts (2594663) | about 2 years ago | (#39528969)

The best AI will turn Wikipedia into an essay on any topic. Then Google will add that AI to answer search questions and you're done.

Re:Or we could hire more teachers... (1)

HornWumpus (783565) | about 2 years ago | (#39528653)

They will game it by getting a copy (or a open source work alike) and installing it in their word processor.

Which is what they should have done in the first fucking place. Sell the damn thing as a web service. Let the word processor do the constant nagging. Add lots of essay questions to history etc.

On the other hand, kids that already have spell checkers and just don't care are beyond help anyhow. Just hand them a shovel and get them started digging ditches.

Re:Or we could hire more teachers... (1)

0123456 (636235) | about 2 years ago | (#39529243)

... Pay them more and tell them to assign more writing and speech assignments and stop letting administration scare them out of flunking those students who refuse to learn...

Except from the stats I've seen from around the world there's no correlation between teachers' pay and results. Perhaps if you stopped paying American teachers more for doing a bad job, they might improve.

Excellent (3, Insightful)

nxcho (754392) | about 2 years ago | (#39528539)

It's only a matter of time before someone writes a tool that generates top grade essays.

yes! (0)

Anonymous Coward | about 2 years ago | (#39528543)

Please, private publishers! You've already bought out and completely ruined the exam system in England and Wales! Please now ruin the last thing you haven't yet found a way to destro - that last bastion of learning which requires human thought and ingenuity to both write and read, the humble essay. God bless Thatcher and her National Curriculum. Hurah for her invention of the mind-numbing GCSE. Without it, how would we have a couple of generations now of people who think they're brilliant but have no discerning ability whatsoever - the perfect cogs for a neoconservative machine.

No way this could backfire whatsoever (3, Insightful)

BlueRaja (1397333) | about 2 years ago | (#39528547)

...thus producing an entire country whose writing-skills were conditioned to game the auto-grader.

Re:No way this could backfire whatsoever (1)

Mitreya (579078) | about 2 years ago | (#39528667)

producing an entire country whose writing-skills were conditioned to game the auto-grader.

Perhaps the same company will start marketing auto essay writing tools soon. Or, if not, at least release study books and offer study courses (a-la SAT/GRE/etc.)

Re:No way this could backfire whatsoever (1)

willpb (1168125) | about 2 years ago | (#39528949)

There will only be one essay auto grader since ETS already owns the patent and I'm submitting my application to patent an auto essay writing program as we speak.

Re:No way this could backfire whatsoever (1)

hazem (472289) | about 2 years ago | (#39528989)

Perhaps the same company will start marketing auto essay writing tools soon.

The only way to win an arms race is to be the vendor that sells to both sides.

Re:No way this could backfire whatsoever (1)

Anonymous Coward | about 2 years ago | (#39529167)

why not? auto tuners worked so well for singers!

Give them something they enjoy writing about. (2, Insightful)

Anonymous Coward | about 2 years ago | (#39528565)

The best way to get students to write is give them something they enjoy writing about.

Re:Give them something they enjoy writing about. (1)

The Mister Purple (2525152) | about 2 years ago | (#39528769)

I agree. Unfortunately, there doesn't seem to be a way to put that on a standardized test, so it won't be happening in most schools here in the U.S.

Re:Give them something they enjoy writing about. (1)

Anonymous Coward | about 2 years ago | (#39528839)

Actually, there are a few subjects nearly all students enjoy, and might even enjoy writing about, but I think there are laws against reading those essays.

Grade This, Robot: +5, Seditious (3, Funny)

Anonymous Coward | about 2 years ago | (#39528573)

Allow me to introduce myself. I'm the founder of the Anti-My School Society. In this letter, I will tell you what made me form such an organization and how I plan to use it to strengthen our roots so we can weather the storms that threaten our foundation. Let me cut to the chase: Relative to just a few years ago, the worst sorts of flippant ogres I've ever seen are nearly ten times as likely to believe that the key to living a long and happy life is to provide contumacious conspiracies with the necessary asylum to take root and spread. This is neither a coincidence nor simply a sign of the times. Rather, it reflects a sophisticated, psychological warfare program designed by My School to work hand-in-glove with what I call intrusive vocabularians.

Even as I write those words I can feel My School cringe. That's okay. Cringe. I don't care because it appears to have found a new tool to use to help it make us the helpless puppets of our demographic labels. That tool is obstructionism, and if you watch it wield it you'll honestly see why it's good at one thing, and that's keeping its ulterior motives secret. Only a few initiates in the inner sanctum of My School's cabal know that it's planning to advocate fatalistic acceptance of a perfidious new world order. Even fewer of these initiates know that I don't need to tell you that we have fallen into My School's trap. That should be self-evident. What is less evident is that My School has two imperatives. The first is to judge people based solely on hearsay. The second imperative is to call for a return to that which wasn't particularly good in the first place.

If you were to tell My School that right is right and wrong is wrong, it'd just pull its security blanket a little tighter around itself and refuse to come out and deal with the real world. My School likes to talk about how cell-phone towers are in fact covert mind-control devices that use scalar waves to beam images into people's brains while they sleep. The words sound pretty until you read between the lines and see that My School is secretly saying that it intends to calumniate helpless rapscallions. I want to advance a clear, credible, and effective vision for dealing with our present dilemma and its most misinformed manifestations, but I can't do that alone. So do me a favor and point out that the emperor has no clothes on. That'll show My School that it's possible that it doesn't realize this because it has been ingrained with so much of Chekism's propaganda. If that's the case, I recommend that we enable adversaries to meet each other and establish direct personal bonds that contradict the stereotypes they rely upon to power their conceited ramblings. Not to put too fine a point on it, but My School's winged monkeys don't want us to disseminate as widely as possible all of the information we have regarding My School's cruel theatrics. That'd be too much of a threat to imperialism, simplism, and all of the other carnaptious things they worship. Clearly, they prefer seizing control of the power structure.

Efforts to create a factitious demand for My School's spleeny, uncouth analects are not vestiges of a former era. They are the beginnings of a phenomenon which, if permitted to expand unchecked, will push all of us to the brink of insanity. My School exists for one reason and for one reason only: to intensify or perpetuate hoodlumism. My goal is to challenge the present and enrich the future. I will not stint in my labor in this direction. When I have succeeded, the whole world will know that My School somehow manages to get away with spreading lies (big emotions come from big words), distortions (honor counts for nothing), and misplaced idealism (it has a "special" perspective on mandarinism that carries with it a "special" right to worsen an already unstable situation). However, when I try to respond in kind, I get censored faster than you can say "archaeopterygiformes".

While there's no dispute that My School is whiney and probably a little counterproductive, it's also cunning, implacable, fanatic, and ruthless. Why else would it inject even more fear and divisiveness into political campaigns? As all of the cognoscenti already know, I have one itsy-bitsy problem with My School's remonstrations. Videlicet, they make life less pleasant for us. And that's saying nothing about how it is appalling to me that it has managed to bring discord, confusion, and frustration into our personal and public lives. But let's not lose sight of the larger, more important issue here: its larcenous, vainglorious publications.

My School's subordinates believe that My School's tractates are good for the environment, human rights, and baby seals. It should not be surprising that they believe this, however. As we all know, minds that have been so maimed that they believe that every featherless biped, regardless of intelligence, personal achievement, moral character, sense of responsibility, or sanity, should be given the power to suck up to supercilious flakes can believe anything, especially if it's false. When all is said and done, we must all face the storm and stress of directing your attention in some detail to the vast and irreparable calamity brought upon us by My School. This exercise will, at the very least, demonstrate to the world that if I didn't think My School would perpetuate the myth that it is God's representative on Earth, I wouldn't say that it is obviously up to something. I don't know exactly what, but courage is what we need to provide a trenchant analysis of My School's hypnopompic insights—not politeness, not intellectual flair, not cleverness with words, just courage. And it sometimes takes a lot of courage to look a jaundiced utopian in the eye and tell him that My School uses highfalutin terms like "macracanthrorhynchiasis" and "electroencephalographic" to conceal its plans to break down the industrial-technological system. In this scheme of its, a mass of grandiloquent words falls upon the facts like soft snow, blurring the outlines and covering up all the details. We become unable to see that My School once tried convincing me that coercion in the name of liberty is a valid use of state power. Does it think I was born yesterday? I mean, it seems pretty obvious that My School's shenanigans are not our only concern. To state the matter in a few words, My School has compiled an impressive list of grievances against me. Not only are all of these grievances completely fictitious, but My School is always trying to change the way we work. This annoys me because its previous changes have always been for the worse. I'm positive that My School's new changes will be even more odious because it doesn't want me to bring important information about its uncontrollable opuscula into the limelight. Well, I've never been a very obedient dog so I intend not only to do exactly that but also to embark on a new path towards change.

Like a baleful bum, My School will undeniably get everyone to march in lockstep with its intemperate apocrisiaries. If you'll forgive my parrhesia, I'd like to add that we have much to fear from My School. Personally, I'm afraid that when you least expect it, it'll impale us on a Morton's Fork: Either we let it destroy any resistance by channeling it into ineffective paths, or it'll diminish our will to live. Regardless of which we choose, it's My School's deep-seated belief that its crimes are victimless. Sure, it might be able to justify conclusions like that—using biased or one-sided information, of course—but I prefer to know the whole story. In this case, the whole story is that I overheard one of My School's vicegerents say, "My School is renowned for its racial and cultural sensitivity." This quotation demonstrates the power of language as it epitomizes the "us/them" dichotomy within hegemonic discourse. As for me, I prefer to use language to initiate meaningful change.

So remember kids, if you want to extend My School's fifteen minutes of fame to fifteen months, all you have to do is agree to let My School rescue colonialism from the rubbish heap of history, dust it off, slap on a coat of cheap sophistry, and market it as new and improved. Sadistic, sex-crazed psychopaths are often found at My School's elbow. This suggests to me that My School's lapdogs have decided, behind closed doors and in closed sessions, to pit race against race, religion against religion, and country against country. By somewhat the same token, although I admit it's not an exact parallel, when I hear its factotums parrot the party line—that I'm too disloyal to build bridges where in the past all that existed were moats and drawbridges—I see them not as people but as machines. The appropriate noises are coming out of their larynges, but their brains are not involved as they would be if they were thinking about how My School spouts the same bile in everything it writes, making only slight modifications to suit the issue at hand. The issue it's excited about this week is adventurism, which says to me that if society were a beer bottle—something, I believe, that My School holds in high regard—it would indeed be the nauseating bit at the bottom that only the homeless like to drink.

My School has a glib proficiency with words and very sensitive nostrils. It can smell money in your pocket from a block away. Once that delicious aroma reaches My School's nostrils, it'll start talking about the joy of Dadaism and how the ideas of "freedom" and "hedonism" are Siamese twins. As you listen to My School's sing-song, chances are you won't even notice its hand as it goes into your pocket. Only later, after you realize you've been robbed, will you truly understand that the path down which it wants to lead us is empty and bleak. I'll say that again because I want it to sink in: Its plans for the future are not just about misoneism but also about Bourbonism. My School's inclinations are built on lies, and they depend on make-believe for their continuation. It's somewhat tricky to make some changes here, especially since the media in this country tend to ignore historical connections and are reluctant to analyze ideological positions or treat a fringe political group seriously.

My position is that My School's newsgroup postings are contrary to international human rights and humanitarian standards. It, in contrast, argues that it is entitled to produce precisely the alienation and conflict needed to publish blatantly brassbound rhetoric as "education" for children to learn in school. This disagreement merely scratches the surface of the ideological chasm festering between me and My School. The only rational way to bridge this chasm is for it to admit that it will probably never understand why it scares me so much. And My School does scare me: Its refrains are scary, its prank phone calls are scary, and most of all, someone once said to me, "The public perception is that My School seems to have a bitter ideological conflict with my statement that we can't stand idly by and let it use both overt and covert deceptions to lay the foundation for some serious mischief." This phrase struck me so forcefully that I have often used it since. Despite some perceptions to the contrary, I'm not very conversant with My School's background. To be quite frank, I don't care to be. I already know enough to state with confidence that My School has been trying to conceal its plans to perpetuate harmful stereotypes. Fortunately, the truth about its doctrinaire, humorless philippics is spreading like a jungle fire. Soon, everyone will know that if I were a complete sap, I'd believe My School's line that clever one-liners are a valid substitute for actual thinking. Unfortunately for it, I realize that I try never to argue with My School because it's clear it's not susceptible to reason. My goal for this letter was to maximize our individual potential for effectiveness and success in combatting My School. Know that I have done my best while trying always to make My School answer for its wrongdoings. Let an honest history judge.

Writing and Review (2)

joocemann (1273720) | about 2 years ago | (#39528615)

... one issue I take with my public education experience was the lack of mention by teachers that they would review or aide in writing for most papers. I recall only the final big paper for the class (whichever class that is) would have something akin to a draft-review event, and then a final draft.

I realize TFA suggests teachers would assign more work, and read less --- and maybe the robots would be useful in providing easy rapid review --- but I can't refrain from mentioning that, in my experience, teachers did not clearly express a willingness to aide in the writing process throughout the semester. (let me beat the critics by saying I was liked and respected by all of my teachers)

Some students are good/great writers and maybe they can be commissioned for honors credit or something in exchange for aiding peers. I know I was in my early 20's before I understood the power of the semicolon; and it is awesome!

No thank you. (4, Insightful)

Anonymous Coward | about 2 years ago | (#39528641)

My wife worked for Pearson as a "second tier" grader (or whatever they call them).

In her case, the tests went through the algorithm and were assigned a grade, then the grade and test were passed along to a human to read and check. Invariably, she would come home complaining about tests where the students had obviously studied specifically to answer the way the algorithm wanted: the algorithm would score the paper high, while the actual content of the test answer would leave a LOT to be desired. The answers would score high, but were more or less gibberish as read by a human.

This was about two years ago, so obviously the algorithms could have improved since then, but I have severe reservations about them becoming the sole arbiter of grading.

Re:No thank you. (1)

HornWumpus (783565) | about 2 years ago | (#39529049)

If they had any sense, they would allow the human reader to flag an essay as 'gaming'.

Then again, if what they ask for is a formula theses paper, they shouldn't complain if what they get is a formula.

Re:No thank you. (1)

Dr. Spork (142693) | about 2 years ago | (#39529387)

It sounds to me like the system is working: The students who are trying to game the computer with crap are given failing grades by a human grader. It might not be long before the easiest way to game the computer will be to learn the material and write a good essay. But as we're on our way there, we'll keep the human element in the equation and leave the computers in the role of the useful assistant.

God no! Abort! Abort! (1)

Anonymous Coward | about 2 years ago | (#39528645)

Please don't give students more writing tests! Don't you see to what ends students will go to cheat at their tests? One day some brilliant chap will figure out an algorithm for generating perfect English! Spammers will crush us under the weight of a gazillion grammatically correct spam mails slipping through our filters. Our communications network will burst at the seams, collapse, and total chaos will ensue! Think of the Children!

bad, bad idea (1)

Eponymous Hero (2090636) | about 2 years ago | (#39528657)

and were we always bad writers? i doubt it. how did we teach students to be good writers before computers? AI can recognize if sentences are grammatically sound, and spelled correctly, but do they understand what is written?

Just like Turn-it-in (1)

subanark (937286) | about 2 years ago | (#39528685)

Even if you create a perfect system, you risk the students being able to get their hands on a pirated version of this software and will have it keep grading their homework until they get the grade they want.

Re:Just like Turn-it-in (2)

The Mister Purple (2525152) | about 2 years ago | (#39528805)

You mean, like proofreading and revising?

Re:Just like Turn-it-in (1)

Dr. Spork (142693) | about 2 years ago | (#39529327)

Excellent comment! I agree, this would be an excellent tool for students. In fact, it should be written for them, not for the professor. As a professor, my assignment would be: Turn in something that the auto-grader marks as a B or better, and only then will I read it and give it a real grade. A system like that would save me a lot of frustration!

Ridiculous (2)

XxtraLarGe (551297) | about 2 years ago | (#39528693)

Yeah, I know teaching is a drag and a lot of work, but that's what they're paying you for. I was up until 1:30 last night fixing/grading code and writing a final project assignment. It's not fun, and I could have easily just ran the programs and told them it didn't work so they got a zero. There's more value for the students when they get feedback from me telling them what specific errors they got, or a way they could have written code more efficiently, etc.

Re:Ridiculous (2)

Hatta (162192) | about 2 years ago | (#39529139)

There's no excuse for being up at 1 am grading papers. Any system that allows this is broken. If a teacher can't complete their days work in 8 hours, we need more teachers. Period.

Re:Ridiculous (1)

the phantom (107624) | about 2 years ago | (#39529407)

Great. I take it that you will be voting for the next tax increase? Or will you be writing your local school district a $30-60,000 check to cover the salary of a new teacher? And I assume that you will be able to convince all of your friends to do the same? Or maybe you are planning on donating your time?

It is all well and good to say that we need to hire more teachers. Hell, I agree! We need more teachers. The problem is that teachers are not paid particularly well, and our society seems to feel that spending more money on more teachers is not a priority---in fact, the mathematics department at my institution was very nearly eliminated last year. Until that changes, teachers are required to work long hours for little pay.

Why are people like this given degrees? (2)

Anonymous Coward | about 2 years ago | (#39528723)

If someone told me to write a paper, then told me it would be graded by some algorithm, I'd tell them to go fuck themselves. Who comes up with this crap?

grammer checker hell (1)

Anonymous Coward | about 2 years ago | (#39528733)

Great. Now all those grammar checkers - which are all terrible - will be picking on passive sentences and all sorts of other perfectly acceptable English usage.

This is *half* right. (4, Insightful)

dgatwood (11270) | about 2 years ago | (#39528749)

The problem is not that the students don't get enough practice. The problem is that the students don't get feedback until they get their grade.

Having an auto-grader grade your work is a terrible idea because auto-graders can't handle complex English. I thought it might be a good idea to run a grammar checker across my novels before publishing them just to have an extra set of eyes, so to speak. So as an experiment, I fed some fragments of one novel (that I knew contained no grammatical errors) into about a dozen of these so-called grammar checkers, along with a list of deliberately broken sentences to see if they actually caught problems.

I just about died laughing at the ludicrous suggestions that the grammar checkers made, mostly stemming from them incorrectly guessing the parts of speech for words that could have more than one meaning. The best of these algorithms correctly reported about 80% of the correct sentences as correct, though many of those algorithms also failed to flag a lot of the incorrect sentences. The worst algorithms flagged more like 80% of the correct sentences as incorrect (and still failed to flag the actual errors in many of the incorrect sentences).

Based on that, I'd say that having someone's grade depend upon such poor algorithms is a really, really bad idea, I'm guessing it will be at least another 1-2 decades before I would trust a computer-based grader to actually perform grading that counts.

However, making those auto-graders available to students for online pre-screening of their writing before they hand in the final version would be a good thing, provided they can make them a lot better. Such software is great at catching simple errors, and anyone with poor writing skills can probably benefit from such software pointing those mistakes out, allowing them to correct their own mistakes before handing the assignments in. This allows the students to learn from the mistakes. A well-designed checker could even keep track of what mistakes a student makes regularly and point out the pattern so that the student can learn to watch for that type of mistake in the future. Unlike robo-grading, such software can actually teach students to improve their skills usefully.

Re:This is *half* right. (1)

The Mister Purple (2525152) | about 2 years ago | (#39528887)

I like your idea for using systems like those in TFA for rough draft work. Not everybody has a parent or roommate with an English degree.

This makes writting essays demeaning (5, Insightful)

denis-The-menace (471988) | about 2 years ago | (#39528781)

If I knew that a machine gets to grade my work I would feel like my time and efforts are worth so little that humans can't be bothered to read it. It defeats the purpose of even writing the thing.

When you write something you are trying to convey an idea. Knowing that the machine doesn't give a fsck proves my efforts are useless.

Method Induced Mono Cultures (1)

icebike (68054) | about 2 years ago | (#39528799)

A grade school teacher who deals with the same 25-30 kids all day teaching a variety of subjects can find time to read 30 papers of the length likely to be written by such students. But in the older grades, the English Comp teacher reading 30 papers from 5 or 6 different class periods simply can not spend that much time on that many papers. Before you get to the post secondary level where teaching assistants are available the job becomes just about impossible.

The structure of our school system imposes a burden on the available brain power that can be brought to task. Further automation seems unlikely to yield better results, but will probably offer the way to handle more students.

Machine grading of papers hacked together by students using word processors (which built in spelling correction, sentence structure analysis, and grammar checkers) run the risk of grading Microsoft's (or LibreOffice) work product as much as that of the students.

In any event, It seems to me that over the years, this can't help but produce a more generic product, and the world is already too full of dense mindless text spewed by dense mindless desk monkeys.

How do these software programs fare when fed a diet of available published books? Would William Faulkner or Henry James be (properly) encouraged to take up basketball? Or would Hemingway be encouraged toward industrial arts?

Re:Method Induced Mono Cultures (1)

Hatta (162192) | about 2 years ago | (#39529151)

But in the older grades, the English Comp teacher reading 30 papers from 5 or 6 different class periods simply can not spend that much time on that many papers.

That just means there are too few teachers. We need more.

How is that cheaper than hiring more teachers? (1)

Arancaytar (966377) | about 2 years ago | (#39528833)

If you can't get teachers to read writing assignments, maybe you should think about the following:

- do you have enough teachers to devote the required time?
- are the teachers paid enough to devote the time?
- are the teachers sufficiently qualified?

If you think that what a teacher does can be done by a robot, you are either living in a science fiction world of positronic brains, or the number one reason the US education system sucks balls.

Re:How is that cheaper than hiring more teachers? (1)

icebike (68054) | about 2 years ago | (#39529229)

If you can't get teachers to read writing assignments, maybe you should think about the following:

- do you have enough teachers to devote the required time?
- are the teachers paid enough to devote the time?
- are the teachers sufficiently qualified?

If you think that what a teacher does can be done by a robot, you are either living in a science fiction world of positronic brains, or the number one reason the US education system sucks balls.

To that you have to add the question of:
- Does the average person need composition skills beyond what is taught in Junior High?
- Will the pilot or the farmer or HVAC installer actually need English Comp?
- Would those that do need these skills later in life, be better off learning them later rather than earlier?

Why foist these tools into the school room? Why not sell them on the open market aimed at those who have a need and desire to write? Why not let them become self improvement tools.

The idea that the next great novel will some how be extinguished if little Johnny doesn't take a writing course is about as valid as the idea that if little Susie doesn't start playing her violin at age 5 the next Yo-Yo Ma will be lost to the world.

Tools =/= Solutions (1)

nine-times (778537) | about 2 years ago | (#39528847)

An auto-grader seems like it has the potential to be a good tool. You let the kids write, and give them immediate feedback about grammatical structure, spelling, and maybe even whether the writing flows well.

However, it seems important to me to recognize that this might be one useful tool, but I very much doubt it will be a good solution for teaching people to write. There is more to writing than "following the rules", and I don't believe that computers can yet evaluate creativity or content. Sometimes a piece of writing is better for having broken the rules, and more importantly, the technical merit of writing sometimes takes a back-seat to the value of the content.

So if you have someone who you're simply trying to drill in "proper structure" for a writing lesson, then this seems like a good tool for the job. I'd be inclined to agree with Mr. Jehn, though. One of the most important things in teaching someone to write is in teaching them to have an idea worth writing. Once they have an idea, then it becomes process of formulating the idea into a form that others might understand, and then massaging the message into a form that people will find understandable and compelling. I believe that writing should not be considered merely as a final product, but as an involved process that is unique for each writer.

Wow, "Would assign more but..." (1)

jmDev (2607337) | about 2 years ago | (#39528863)

But nothing, the reason high school student's writing is so bad is because of the teachers. Whether that be k-12. Start them out writing, that way by the time they get into high school, they may be decent at it.

Re:Wow, "Would assign more but..." (1)

evil_aaronm (671521) | about 2 years ago | (#39529349)

You're close, but I think it's actually nearer to, "start them out reading." It's fair to assume they -can- read, if they're writing, but I always compared what I wrote to things I'd read, particularly if the topics were related. When proofing my work, I'd listen to the "tone" of what I wrote to see if it was in the ballpark.

If kids read only minimally, they won't be as exposed to "good" writing and will more likely write crap. It's like when you're studying a new programming language: looking at good example code can help you pick up the nuances, and also shape the way you write code going forward.

Bullshit (2)

Iamthecheese (1264298) | about 2 years ago | (#39528865)

Computers just aren't up to understanding complex English well enough to decently grade it. The smartest students will very quickly grow apathetic and start gaming it, whilst forgetting the skills they do have. The less intelligent students will just learn to run it through MS Word's grammar and spelling check and add words that don't fit but are long.

Clippy ... Is that you? (0)

Anonymous Coward | about 2 years ago | (#39528941)

It looks like you are trying to toss a word salad...

How is good writing determined (1)

MetalOne (564360) | about 2 years ago | (#39528959)

In all my years of schooling, except one college class, I never really knew how the grades on my papers were determined, and I never really received any feedback other than a letter grade. I knew the teachers wanted correct spelling and grammar but that was about it. In college, I finally had a teacher point out that I had a tendency to jump around from past to present tense. She explained that usually one wants to keep the same tense. I had never noticed I did this. She also happened to mention that George Orwell's writing style was considered good, because he made his point with a minimum of superfluous words. She mentioned many students are so used to trying to pad essays with words to reach the word limit that everybody starts using lots of unnecessary text to make their point. I also had a teacher who occasionally read a student essay he liked. I noticed these usually used lots of big words. Whether that actually had anything to do with the grade I have no idea. I also once had an architectural drawing class. The teacher like to give penmanship assignments. For the half the class I never got a good grade on the penmanship assignments and I never knew why. I worked at them meticulously for hours. I had pen been using a regular #2 pencil. Then one day I used a .5mm mechanical pencil and I received a perfect score. Would it have killed the teacher to tell me to buy the right pencil. I much preferred math to english in school. When you had the right answer, you pretty much always knew you had the right answer.

MCAT already did it in 2007 (1)

Jayfield (2317990) | about 2 years ago | (#39528967)

I took the MCAT in 2007 and the essay portion was graded by 2 entities: a computer and a person. They then averaged the score. Also, not only was this in 2007, but the essay specifically tested for critical thinking; grammar and syntax mistakes were acceptable

Great Idea (0)

Anonymous Coward | about 2 years ago | (#39529011)

I would start all my essays with: DROP TABLE students;

Assign more writing (1)

GmExtremacy (2579091) | about 2 years ago | (#39529021)

The theory is that teachers would assign more writing if they didn't have to read it.

Yeah, because quantity > quality. They need to do plenty more mind-numbing activities.

The problem with Essays (1)

Dyinobal (1427207) | about 2 years ago | (#39529029)

In general when I was in school, the problem with Essays was, the minimum length requirement. I'd write my essay and it would be great, but fall short of the length requirement by the teacher and I'd end up padding it out by restarting what I said different ways, adding useless sentences etc.

I complained about the miniumum length at one point and the teacher told me to turn in my original essay and she would grade it and if it was better than my other one then she would no longer require a minimum length. So I turned it in and got an A easily.

I get why teachers have a minimum length, they don't want kids to write only like a few sentences and turn it in. Still I think this is silly, my best essay in high school was only a page and a half long and won me a contest in the school easily

only high school students? (0)

Anonymous Coward | about 2 years ago | (#39529039)

most Americans, even white-collar workers, can't spell for shiite

As somebody who reads a lot of college writing... (1)

Dr. Spork (142693) | about 2 years ago | (#39529089)

My students are pretty talented, but I can tell you: Even they would benefit from having a proofreader for the things they turn in to me. It looks like Pearson (I don't like that company, btw.) is trying to market this as an evaluation tool, which is not the best idea. It shouldn't be a tool of the professor, but a tool of the student, something like an improved spell checker, but more of an idea checker.

Machine semantic analysis has actually come quite a long way, and together with subject experts, maybe we are on the verge of being able to compose a program that could give useful feedback on an essay about - say - the history of heliocentrism or the Cogito argument by Descartes. I'd love to have my students mess with such a program until they compose an essay that satisfies it. (It should also auto-check for plagiarism.) Of course, the grade would be decided by me. The role of the program would be to prod the student to produce better work, and to give him or her useful suggestions for how to do so.

Using theories before they are proven (1)

Okian Warrior (537106) | about 2 years ago | (#39529185)

The theory is that teachers would assign more writing if they didn't have to read it.

Hey, here's an idea - instead of implementing someone's pet theory-of-the-month, how about we attempt to prove the theory first?

The field of Education is rife with theories - the biggest and worst that comes to mind is the "new math" a few years back. It seems every year or so someone notices that education is failing, comes up with the reason, and a new method of teaching kids "more better" is born.

Why do we let educational institutions try unproven techniques on our children? Techniques that might damage their education and screw them over for the rest of their lives? Isn't unproven teaching techniques as dangerous as unproven medicines?

How about we test these theories before foisting them on our children?

Re:Using theories before they are proven (1)

GmExtremacy (2579091) | about 2 years ago | (#39529431)

Why do we let educational institutions try unproven techniques on our children?

Maybe it's the same reason we've let the current educational system, designed to churn out obedient factory workers, destroy our children's education. Apathy.

Pearson Ed Sucks (0)

Anonymous Coward | about 2 years ago | (#39529329)

I have had the misfortune of having Pearson Education textbooks in physics and finance and they are so riddled with errors they are useless to someone who is actually trying to learn the topic. I seriously doubt their automated grading program can do a better job than even the poorest of profs.

Failure to understand game theory (0)

Anonymous Coward | about 2 years ago | (#39529399)

When student think a human will grade their essay, it is easy to write a compute program that will grade better than a human by looking for patterns. When students know a machine will be grading their essay, they can game the system by using knowledge of the machine's system, something they couldn't do when they thought humans would do the grading.

It's like comparing the crime rate in a city when you remove all police without telling anyone, and the crime rate in a city where everyone knows there are no police.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...