×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Typingpool: Human Audio Transcription Parallelism

timothy posted about a year ago | from the dove-eyed-in-conk-her dept.

Software 41

theodp writes "Silly rabbit, parallel processing is not just for Big Data! Building on techniques outlined by Andy Baio back in 2008, Wired writer and 20% Doctrine evangelist Ryan Tate has released Ruby-based software called Typingpool to make audio transcriptions easier and cheaper. 'Typingpool chops your audio into small bits and routes them to the labor marketplace Mechanical Turk,' Tate explains to his reporter pals, 'where workers transcribe the bits in parallel. This produces transcripts much faster than any lone transcriber for as little one-eighth what you pay a transcription service. Better still, workers keep 91 percent of the money you spend.' Remember to Use the Force for Good, Tate adds."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

41 comments

Excuse Me While... I Kiss This Guy. (5, Insightful)

cognoscentus (1628459) | about a year ago | (#42376121)

Seems an interesting idea for long recordings needed in a hurry, but the transcriber will be losing possibly important contextual cues by reducing the length of the utterance. Also, there is a great overhead per-person in terms of manual labour (waiting for audio to buffer, HCI interaction, etc). On the upside, it might be less dull to listen to shorter, more varied recordings than one long one. But it would suck to transcribe half a murder-mystery.

Stop discussing ang go buy Northland ! (-1)

Anonymous Coward | about a year ago | (#42376171)

If you want more games on Linux, please buy Northland ! Northland [desura.com]

Training (4, Interesting)

girlintraining (1395911) | about a year ago | (#42376173)

The biggest market for audio transcription I'm aware of is in medical transcription, followed by legal. Many of the terms used are not normal english; In fact without a basic understanding of medicine, you could easily confuse one thing for another, with potentially tragic results. This is fine for everyday english, but not in industries where terminology is used that isn't. And that's a lot of it. This would be more useful for something like Siri -- no doubt Apple has humans to process some of the unknowns in the background, and would find a service like this useful, if they don't have something similar already.

Re:Training (2)

MLBs (2637825) | about a year ago | (#42376295)

This would be more useful for something like Siri -- no doubt Apple has humans to process some of the unknowns in the background

Not so sure about that [youtube.com].

Re:Training (1)

runningduck (810975) | about a year ago | (#42377143)

To be fair it took me several times before I could figure out what he was saying. Radiolingua really needs to create a "Coffee Break Scottish." This might even need to be a prerequisite to their other instructional series.

Re:Training (1)

Cederic (9623) | about a year ago | (#42377685)

Really? Interesting. It was apparent on the very first attempt, when his accent was at its broadest.

He needs to get himself out of Scotland though. An accent like that'll win him popularity across the world.

Re:Training (3, Informative)

iluvcapra (782887) | about a year ago | (#42376359)

I suspect distributing even small, redacted portions of a medical or legal dictation would violate the many confidentiality laws in force in these industries.

I'm a sound editor and from time to time I've toyed with sending certain extremely cretinous jobs to Mechanical Turk, things like cutting silence out of audio recordings (can't always automate this), identifying and synchronizing numerous takes, or going through a scene frame by frame and identifying every frame with a gunshot. It's technically possible but if your project is anything more complicated than the tiniest FunnyOrDie video you're going to be breaching the producer's confidence.

As information technology makes things like Mechanical Turk easier to implement, it makes the information you would send to MT all the more valuable and dangerous to release.

Re:Training (1)

Urza9814 (883915) | about a year ago | (#42376577)

That may be the biggest market right now, but that doesn't mean there aren't significant areas where this may be useful. Independent podcasts would be a great one -- currently most of those don't have any transcriptions available, but there are a number that I personally would follow if I could read them instead of listening. Conferences are another great one. Generally you'll get partial transcriptions of keynote speeches and that's about all they can do. A service like this might make full transcriptions of all sessions possible (I've been to numerous conferences where they record hundreds of sessions but don't transcribe anything at all.) News reports is also a great one. I frequently read transcriptions from Democracy Now!. Those could probably be done faster and cheaper than with this, and they're usually rush transcripts anyway and tend to contain a few minor inaccuracies, so that's not a huge problem.

Perhaps part of the reason audio transcription is such a limited market is precisely because it's so expensive and difficult. This kind of idea could change that and open up entire new market segments.

Re:Training (0)

Anonymous Coward | about a year ago | (#42376979)

Build in parity by overlapping chunks. Yes, it decreases the speed of transcription, but it would allow the application to compare overlaps for differences then start a contextual check for larger coherence.

Not sure this is a good idea (4, Interesting)

CRCulver (715279) | about a year ago | (#42376179)

Five or six years, I transcribed podcasts for Mechanical Turk. Their audio files were already split into shorter passages (3 or 5 minutes, for example). Split them even further, and the transcriber might miss out on the context, which is often vital to knowing what exactly is being jabbered about.

That said, I'm not sure who is the target demographic for this kind of work anymore. Many of the podcasts were on subjects of interest to me, and I was getting about $10/hour from Mechanical Turk, which wasn't bad considering that I was often doing the work from backpacker beach havens around the globe where a couple of hours of work would pay all my expenses for the day. But the last time I had a look at Mechanical Turk, the amount they pay had been heavily reduced. Who wants it now? Even if you are in a cheap third world country, if you have the English skills for such transcriptions, you can surely find better and more dependable work elsewhere.

Re:Not sure this is a good idea (2)

eugene ts wong (231154) | about a year ago | (#42376251)

I don't see how you made $10/hour. The assignments that I got were only a few cents. You say $20 would pay for your expenses for the day? I thought that room and board would be more than that.

When I did it, I made about $1/hour, and sometimes less, because the voices were garbled, and sometimes my work would be rejected. It was silly.

Re:Not sure this is a good idea (1)

eugene ts wong (231154) | about a year ago | (#42376257)

I forgot to add that I don't like what the summary said. 1/8 of the wages is not much at all. If it was $8 before, then the new price is $1. Being able to keep 90% of that is $.90. I'd rather have 50% of $8.

Maybe I misunderstand the summary.

Amazon Charges 10% Fee, Worker Gets 90%+ (1)

theodp (442580) | about a year ago | (#42376403)

How are the fees calculated? [mturk.com]: "Amazon Mechanical Turk collects a 10% commission on top of the reward amount you set for Workers. For example, if a HIT reward is set to $0.20, Amazon Mechanical Turk collects $0.02 for each assignment." So, the worker gets 91% of the total amount paid ($0.20/($0.20+$0.02).

Re:Amazon Charges 10% Fee, Worker Gets 90%+ (1)

eugene ts wong (231154) | about a year ago | (#42376523)

That part I understand. It's the lower price that bugs me. From the summary, I get the impression that they had to pay $10, for example, for an ordinary contract, but now, they can pay $1. In other words, it sounds like a worse deal for the labour force. The percentages might be better, but the total money at the end of the day is worse, if I understand correctly.

Re:Amazon Charges 10% Fee, Worker Gets 90%+ (0)

Anonymous Coward | about a year ago | (#42380075)

Minimum wage doesn't usually apply to people that are considered to be self employed. So, minimum wage doesn't come into effect. Mechanical Turk is only affordable as a source of income if you live in the 2nd or 3rd world, for those in the 1st world you're better off getting a real temp job if you need extra cash.

Re:Amazon Charges 10% Fee, Worker Gets 90%+ (1)

Rich0 (548339) | about a year ago | (#42380675)

I'm pretty sure that whoever creates the HITs assigns their value. Market forces essentially dictate the price.

If somebody wants to pay you $20/hr they can, or if they want to pay $2/hr they can do that instead. If they offer too little then nobody will work on them, in theory. However, it is a global market.

Re:Not sure this is a good idea (2)

CRCulver (715279) | about a year ago | (#42376389)

I don't see how you made $10/hour. The assignments that I got were only a few cents. You say $20 would pay for your expenses for the day? I thought that room and board would be more than that.

I take it you've never been to India, Nepal, Thailand, Indonesia, Turkey, etc.? Once you leave Europe and North America, you can often find a comfortable and clean bed for the night for under US$10, and even back in 2007 free wi-fi could be found a nywhere there were tourists. For cheap meals, just eat at places that local people eat at -- you think poor people spend more than US$10 on food per day?

I don't see how you made $10/hour. The assignments that I got were only a few cents.

There was a scoring system in place. If you maintained a good score, higher-paying jobs were made available. Of course you still had to do a search for jobs paying over $X to make them show up on your screen, but there was a pretty steady flow of them.

Re:Not sure this is a good idea (1)

eugene ts wong (231154) | about a year ago | (#42376557)

No, I haven't been to those countries. Thanks for clarifying. :^D I've been to China, though, so I should have known. I think that part of my problem is that I did most of my travelling in Canada.

Re:Not sure this is a good idea (0)

Anonymous Coward | about a year ago | (#42376645)

Yeah, sorry -- I don't want to compete as labor for American companies with other labor in places around the world who pay less for rent than I do for a week's worth of milk. These are things outside of my control. I can't be "cheaper", because rent, food, gas, insurance, and other costs are firm necessities that I can't change and I don't think American companies should be able to force me to become a citizen of a foreign country just to be able to compete for employment with the American company.

Re:Not sure this is a good idea (0)

Anonymous Coward | about a year ago | (#42376719)

I think you missed the point, he wasn't competing for work, he was on a never-ending vacation working just enough to get by. You make this sound like he forcibly offshored himself.

Re:Not sure this is a good idea (1)

icebraining (1313345) | about a year ago | (#42376421)

You say $20 would pay for your expenses for the day? I thought that room and board would be more than that.

It depends on where you live; it should be enough in cheap countries, like Cambodia.

Re:Not sure this is a good idea (0)

Anonymous Coward | about a year ago | (#42376749)

You say $20 would pay for your expenses for the day? I thought that room and board would be more than that.

When I was backpacking in Australia 10 years ago, backpackers charged between A$7-A$12 in most places and only on the East Coast during peak season did I ever have to pay in the A$15-A$25 range. I'd estimate the average price I paid over the year was A$10 per night because I spent most of my time away from the East Coast. These are Australia Dollars too, so at the time it would be about US$5-$10 for the hostel.

I certainly didn't live like this all the time, but I did have a period of a month where I lived very frugally and spent about A$10 per day on accommodation and 3 simple meals a day, so about US$6 per day and spending all day on the beach reading books.

Re:Not sure this is a good idea (0)

Anonymous Coward | about a year ago | (#42377243)

Go check reddit.com/r/beermoney people have pushed passed minimum wage by working 8 hours a day on there. I myself have made 140 over the year for the random MP3 or Humble Indie Bundle, easy pool of cash to send a few game keys from come Christmas/Birthday times. Never got past 4-5 an hour even pre-election when there were a lot of $1.00 5-10min surveys. Never touched the transcription posts but as someone who keeps an eye on mturk and other beer money style things, 10/hour is certainly feasable especially of the postings are paying out more (as that does vary with time).

Re:Not sure this is a good idea (0)

Anonymous Coward | about a year ago | (#42376781)

Screw Mechanical Turk. It is an Internet era sweatshop.

Re:Not sure this is a good idea (1)

elucido (870205) | about a year ago | (#42376889)

Five or six years, I transcribed podcasts for Mechanical Turk. Their audio files were already split into shorter passages (3 or 5 minutes, for example). Split them even further, and the transcriber might miss out on the context, which is often vital to knowing what exactly is being jabbered about.

That said, I'm not sure who is the target demographic for this kind of work anymore. Many of the podcasts were on subjects of interest to me, and I was getting about $10/hour from Mechanical Turk, which wasn't bad considering that I was often doing the work from backpacker beach havens around the globe where a couple of hours of work would pay all my expenses for the day. But the last time I had a look at Mechanical Turk, the amount they pay had been heavily reduced. Who wants it now? Even if you are in a cheap third world country, if you have the English skills for such transcriptions, you can surely find better and more dependable work elsewhere.

Where can I find these backpacker beach havens? Seriously!

Re:Not sure this is a good idea (1)

Kergan (780543) | about a year ago | (#42380367)

Much about anywhere, really.

Stay clear of major tourist centers, mix with locals extensively, and don't hesitate to ask if the latter have a room available for a few nights. Nowadays it's also easy to find a cheap bed through couch-surfing websites, short-term flatsharing websites, and Facebook.

A Deepness in the Sky (0)

Anonymous Coward | about a year ago | (#42376337)

This reminds me eerily of the term, "zipheads". This is not a racial slur. Those who have read "A Deepness in the Sky," will likely recognize it too.

Use the force for good? (0)

Anonymous Coward | about a year ago | (#42376515)

Maybe I just lack imagination, but how could you use it for ungood?

Examples?

Re:Use the force for good? (1)

papafox_too (883077) | about a year ago | (#42376607)

Ask Mark Zuckerberg

No, you mean Eduardo Saverin, Traitor to the US. (1)

sethstorm (512897) | about a year ago | (#42385271)

He thought that he was too good for US citizenship.

Perhaps the US should pursue on that additional $62 million by returning him and said assets to US jurisdiction. In this case, the military and intelligence used to effect his return would be a force used for good.[

Re:Use the force for good? (0)

Anonymous Coward | about a year ago | (#42376625)

It's a meaningless part of the blurb - just like the idiotic commercial "silly rabbit" comment. What do you expect from "theodp"? He just constantly posts utter crap.

That's how anime fansubs are done (4, Interesting)

Animats (122034) | about a year ago | (#42376559)

The volunteer process by which Japanese anime is subtitled within hours of release works a lot like this.

Re:That's how anime fansubs are done (1)

Anonymous Coward | about a year ago | (#42376743)

And it's how we end up with mass naked child events.

Re:That's how anime fansubs are done (0)

Anonymous Coward | about a year ago | (#42380039)

More details please? Would love to see translations for some of my other favorite Japanese shows, like 'world ranking'. Right now my wife translates for me but it's unpleasant to watch - need to pause the show, translate, unpause, repeat process.

Re:That's how anime fansubs are done (1)

drinkypoo (153816) | about a year ago | (#42381049)

More details please? Would love to see translations for some of my other favorite Japanese shows

It's different for different groups. Many of the groups have or have had forums where they would announce releases and talk about whatever. A bunch of people capable of transcribing each take a section of the episode and typically do a transliteration, then often one person produces a translation, then one or more people time the subs which doesn't really take all that long and then they encode the video and then they upload to their distribution point and you know how it works from there I assume.

Subbing used to be a PITA but now there's pretty nice programs for it on most platforms. And before there was a good one on Linux you could use some to most of the best ones from Windows in Wine.

I talked about this idea for transcripts for podcasts not so long ago on Google Plus. I would like to see it implemented as a sort of social site. People would link to things they want transcripted, and people would vote on which things they found most interesting, and then you could upload transcripts (or segments thereof, which was the whole point) to the site and get points or something about which you could be smug. Perhaps the points could be used to cause your transcript requests to bubble to the top of lists so that people would see them. Users vote on the quality of your transcripts. It seems like it would be easy to implement if you didn't care about scaling, which of course I do which is why I haven't tried :)

Most so-called AI is already done like this (-1, Troll)

Anonymous Coward | about a year ago | (#42376739)

Computer AI (semantics automatically arising from syntax) is, of course, impossible. All so-called AI programs depend on Human derived rules.

Google, for instance, uses Human workers to constantly add new rules to their search engine, while allowing the unwashed public to think the same is done by magical computer algorithms.

Many companies in the last 15 years have claimed to offer high quality translation of written and spoken Human language by computer, when what they really meant was that a computer system would send the data to Human operators for real-time conversion.

The 'Mechanical Turk' approach is the reality of so-called computer AI.

Of course, with voice recognition, or language translation, we should be disturbed by the lack of progress by computer software. Both of these tasks clearly do NOT need any concept of AI, since we already have full understanding of the language rules needed by the algorithms. Why then does software still perform so poorly? The researchers of the 1950s would be horrified at our lack of progress. They assumed it was just an issue of processing power and storage.

Not only does real AI not exist, we can't even successfully handle the obvious pseudo-AI problems like language processing.

Rethink the 'Mechanical Turk' concept. Shouldn't we all be able to input into a 'super' MT system to give that system the benefit of our innate understanding of key rulesets? How many rules does a speech-to-text engine need? If millions of Internet users were able to input rules into such an engine, would it rapidly become near perfect. Surely we would all volunteer to be part of a MT to create better speech recognition and language translation? And NO, Google's lousy "suggest a better translation" doesn't count, given how poorly it improves matters.

What About? (0)

Anonymous Coward | about a year ago | (#42376971)

What about when the whole is greater than the sum of its parts? This sounds great if what you want is a text dump transcription of a pod cast or some such. But, what of formatting and assembly of the whole document?

This sounds only slightly more useful/effective than automated speech to text transcription.

Sounds like shitty transcription (0)

Anonymous Coward | about a year ago | (#42377249)

I transcribe professionally, and this sounds like a really bad idea. Like others have said, context is really important. A lot of what I do is hard to understand. Often I will go back and fill in stuff that I couldn't understand in the beginning but am able to understand after hearing the speakers for a while. So if I were just doing random snippets from a bunch of different recordings, all with different speakers and audio qualities, accents, etc., I would do a horrible job. This also would make it more difficult to have a consistent style in terms of editing for nonverbatim transcription. Maybe this would be OK for crappy subtitles or quick podcast transcriptions or whatever, but it sure wouldn't replace any of the stuff that I get paid for.

Sounds like the Human Computers of WWII (2)

davidwr (791652) | about a year ago | (#42377255)

Human Computers [nasa.gov] describes the (mostly) women who did the mathematical calculations the engineers handed them to do for the war effort, freeing up valuable engineering time.

The article doesn't say how much parallelism there was in these "human computer pools" but I suspect there was a lot of it.

I guess it goes to show that even today, some problems are, for the moment at least, done cheaper, faster, and/or better (pick any two) with a person than an electronic computer.

See also: Logopolis [wikipedia.org], a Dr. Who serial from 1981 which features people (not humans) doing mathematical calculations rather than having computers do them.

It already starts in bad territory (1)

sethstorm (512897) | about a year ago | (#42385249)

Like any tool, Typingpool could probably be used that way. Please don’t use it that way!

You're using a service that generally does not attract people from the developed (First World) nations. It *is* being Used That Way.

Typingpool defaults to paying $0.75 a minute, and I often offer $1.00/minute, which produces transcripts very quickly, tends to attract better workers and is still roughly half the best rate I’ve seen for high-speed professional transcription. I have had success completing transcripts at lower rates, and Baio four years ago was able to find plenty of workers at $0.40/minute. But lower prices generally translate to slower transcription and lower quality.

You get what you pay for, but the service would have to have a builtin "No Third World" option defaulted to "on" for it to work well.

Also, bear in mind that just because you pay Mechanical Turk workers a lower rate through Typingpool than you’d pay a service doesn’t mean the workers are actually getting paid less or are worse off

However, the likeliness favors the conclusion that they are getting paid less/worse off.

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...