Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Google's Manual For Its Unseen Human Raters

timothy posted about a year and a half ago | from the do-this-stuff dept.

Businesses 67

concealment writes "It's widely believed that Google search results are produced entirely by computer algorithms — in large part because Google would like this to be widely believed. But in fact a little-known group of home-worker humans plays a large part in the Google process. The way these raters go about their work has always been a mystery. Now, The Register has seen a copy of the guidelines Google issues to them."

cancel ×

67 comments

like slashdot (5, Funny)

staltz (2782655) | about a year and a half ago | (#42105255)

"For relevance raters are advised to give a rating based on "Vital", "Useful", "Relevant", Slightly Relevant", "Off-Topic or Useless" or "Unratable"."

Hmmm, sounds like Slashdot. Anyone unemployed?

Re:like slashdot (-1)

Anonymous Coward | about a year and a half ago | (#42105335)

Alternative ratings:

Politically correct.
Beneficial to Advertisers
Beneficial to a particular political viewpoint.

Re:like slashdot (0)

Anonymous Coward | about a year and a half ago | (#42106679)

Do any of those preclude the worthiness and accuracy of the content being correct?

Re:like slashdot (2)

NatasRevol (731260) | about a year and a half ago | (#42105675)

Without a CowboyNeal rating, it's completely useless. Perhaps even unratable. Or unrateable.

Unpaid moderator here (1)

u64 (1450711) | about a year and a half ago | (#42110917)

Hey Slashdot! Why am i moderating for free, eh? Shoooow meeeeeee the monay!!!

Man, guideline #48 is creepy (5, Funny)

crazyjj (2598719) | about a year and a half ago | (#42105309)

"It puts a good rating in the bin or else it gets the hose again"

Work from home? (5, Funny)

Anonymous Coward | about a year and a half ago | (#42105329)

So I really can make $5000/month as a single mom?

Re:Work from home? (0)

Anonymous Coward | about a year and a half ago | (#42105379)

before taxes...and if you never do anything with your children.

Re:Work from home? (3, Insightful)

Anonymous Coward | about a year and a half ago | (#42105445)

Nowhere near that much. I know someone who was a rater. Pay rate was ok for someone in Idaho who needs part time work (something like $15 an hour), but there are limits on the number of hours you can work (both over and under), and you're often limited by the number of tasks available.

Re:Work from home? (3, Informative)

The Pirou (1551493) | about a year and a half ago | (#42105615)

You're talking about Lionbridge.
Leapforce isn't capped in the same way, but it has a lower rate of pay. Individual raters see limited hours at first, but as long as you perform well there is usually way more than 40+ hours.

This isn't news, as old versions of the General Guidelines have been leaked to the public before.

Re:Work from home? (0)

Sulphur (1548251) | about a year and a half ago | (#42105819)

Nowhere near that much. I know someone who was a rater. Pay rate was ok for someone in Idaho who needs part time work (something like $15 an hour), but there are limits on the number of hours you can work (both over and under), and you're often limited by the number of tasks available.

Small potatoes, and certainly not the potato on tour.

Re:Work from home? (5, Funny)

dietdew7 (1171613) | about a year and a half ago | (#42105491)

Maybe, send pictures and your hourly rate.

Re:Work from home? (0)

Anonymous Coward | about a year and a half ago | (#42105555)

Closer to a grand if the pay and hours are roughly what we got working on the same kind of project. If the metrics are anywhere near the same too, the combination of stress and boredom will be intolerable.

Re:Work from home? (5, Informative)

Anonymous Coward | about a year and a half ago | (#42105751)

I've done this work (for LeapForce; there is another company LionBridge that does the same work, but I have no experience with them). It's really, really mind-numbingly boring. I didn't last very long due to that. And it's worse than say a boring service job where you can interact with other people because you interact with nobody (I realize this may appeal to some). The other thing is that there is not always work available when you want to work. So, you may sign in and be ready to go, but there's no work there. This may cycle somewhat throughout the year with their hiring cycles. You may end up visiting sketchy websites, so having everything up to date is a must. They allow you to opt-out of adult ratings, but that doesn't mean that you will never come across an inappropriate site. You also have minimum performance requirements, and I think it would be difficult to maintain those requirements if you have lots of distractions going on while you work (kids, etc.). Some people are not able to attain the minimum at all within their required time frame (I had a couple of friends try and fail).

They do pay as agreed, but don't expect your check in a hurry. When I did work, you were able to submit an invoice on the 1st of the month for the previous month. They then paid net-30 on that invoice. That means, if you start December 1, then on January 1 you can submit your invoice for December. At the end of January you get paid for that invoice (for the work you did in December). So, if you need money quickly it doesn't work out so well, but once you get started it is a monthly income. They may have changed policies, so be sure to check it out before starting work.

That being said, the pay is good for a job you can do from home (I think around $13.50/hour). So, you certainly may be able to make $5000 a month, but that would be some insane hours (> 90 a week). If you need some extra income, try it out. There's a couple of tests to take before you are hired, and those are a good way to see what you think of the work. If you hate taking the tests, you will hate the work.

Re:Work from home? (1)

clickclickdrone (964164) | about a year and a half ago | (#42106161)

That means, if you start December 1, then on January 1 you can submit your invoice for December. At the end of January you get paid for that invoice (for the work you did in December).

But on the plus side, the other end you can put your feet up for a few weeks and still get paid!

Continued in alt.glass.half.full

Re:Work from home? (0)

Anonymous Coward | about a year and a half ago | (#42111571)

Google needs to get some raters in Google Groups, the flamewars, ridicule, and online pharmacy posts often take precedence over posts with actual useful content.

Re:Work from home? (0)

Anonymous Coward | about a year and a half ago | (#42108757)

Maybe, but you'll be involved in a little more than rating web sites.

The higher bar (0)

Anonymous Coward | about a year and a half ago | (#42105333)

I'd really like to see the algorithm that plays down the significance of pornography traffic. I mean what is the percent of traffic on the net related to porn and what is it's percent represented on Google?

Just wondering.

Re:The higher bar (2)

ciderbrew (1860166) | about a year and a half ago | (#42105427)

What are you looking for???

Re:The higher bar (1)

RaceProUK (1137575) | about a year and a half ago | (#42105523)

I mean what is the percent of traffic on the net related to porn and what is it's percent represented on Google?

While it's often repeated that porn makes up the majority of traffic, in reality it's an almost insignificant amount. I'd bet that traffic only from Google exceeds porn traffic by several orders of magnitude.

Re:The higher bar (1)

fahrbot-bot (874524) | about a year and a half ago | (#42107135)

I mean what is the percent of traffic on the net related to porn and what is it's percent represented on Google?

While it's often repeated that porn makes up the majority of traffic, in reality it's an almost insignificant amount. I'd bet that traffic only from Google exceeds porn traffic by several orders of magnitude.

From Scrubs [wikipedia.org] :

Dr. Cox: Listen Vanessa Janice Tiffany Amber Thiessen. I'm gonna go ahead and give ya a little something I call Perry's Perspective.

  1. If the guy in front of me in the coffee shop can't decide what he wants in the 30 minutes it takes for him to get to the register, I should be allowed to kill him.
  2. I'm fairly sure that if they took all the porn off the Internet, there'd only be 1 website left, and it would be called Bring Back The Porn.
  3. And most important, to be respected as a doctor, nay a man, you must be an ocean. You're born alone, you damn sure die alone, (looks over and speaks to a cadaver rolling by) isn't that right Spike? My point is, and you may want to jot this down... only the weak need help.

J.D.: I should have that tattooed on my neck.

Re:The higher bar (1)

osu-neko (2604) | about a year and a half ago | (#42108157)

While it's often repeated that porn makes up the majority of traffic, in reality it's an almost insignificant amount. I'd bet that traffic only from Google exceeds porn traffic by several orders of magnitude.

This, of course, depends on your definition of porn. To many of the people who oppose it, half of network TV content is porn...

Re:The higher bar (4, Funny)

Coisiche (2000870) | about a year and a half ago | (#42105527)

Hang on...

Does this mean that the raters can view porn and claim that it's on the clock?

Re:The higher bar (1)

Frosty Piss (770223) | about a year and a half ago | (#42106791)

...Missionary, doggy, doggy, doggy, missionary, blowjob, blowjob, tossed salad, missionary, small cock...

slashdot (-1)

Anonymous Coward | about a year and a half ago | (#42105353)

slashdot posts the most brain dead stuff these. every search result is produced by an algorithm but they use humans to tune such. jesus christ. THIS IS NOT NEWS.

Re:slashdot (1)

kasperd (592156) | about a year and a half ago | (#42110611)

Parent post is correct. Pages are not evaluated by people, but rather by an algorithm. And search results are not produced by people, but rather by an algorithm. But the algorithms don't magically appear. Those algorithms are written by people. But even smart people with good intentions cannot know if an algorithm is going to produce a good result or not. And this is where the human raters come into the picture. Their job really is to evaluate the variations of the algorithms introduced by the developers, to ensure that improvements to the algorithm make it through and other changes do not.

Could it be... (5, Interesting)

Baba Ram Dass (1033456) | about a year and a half ago | (#42105411)

First off, didn't read the article. Yeah, I said it. So if the article dispells this just ignore me.

What if Google actively uses the human ratings as a comparison/benchmark against which they measure those fancy algorithms? In other words, the users are rating the algorithms more than they are the websites. Makes sense they would improve search results algorithms, a highly technical and scientific method of ranking sites (which is of little use to a human in and of itself), by constantly striving toward an unscientific and untechnical (e.g. "quality") method... humans... which afterall is, you know, who uses the engine in the first place.

Amazon probably does the same to improve their suggestions model.

Re:Could it be... (5, Insightful)

mbkennel (97636) | about a year and a half ago | (#42105573)

This is almost certainly what is happening. It is impossible for humans to rate any significant fraction of searches/websites to be quantitatively useful for Google's search volume.

In machine learning, the name is "tags", a.k.a. ground truth for a supervised prediction/ranking model. Google gets zillions of weak, noisy, tag proxies in the sense of being able to measure when a user clicks on a link and then within a minute clicks on another link on the same search page, potentially indicating that the first link was undesirable.

These are the relatively expensive but highest quality "ground-truth" tags from which Google can calibrate the value and interpretation of the weak automatic tags and the algorithms themselves.

The final machine learning algorithms may be as simple as linear regression---performed on some rather complex features. These ground truth tags are used to calibrate and weight the importance of various features in making a final ranking.

Re:Could it be... (1)

Sulphur (1548251) | about a year and a half ago | (#42105689)

Amazon probably does the same to improve their suggestions model.

There is a suggestion bot?

--

It is now safe to turn your computer on.

Re:Could it be... (0)

Anonymous Coward | about a year and a half ago | (#42105897)

It should have been obvious that they would need to use humans to test the algorithms.

How else would they do it? You need a standard, and humans are it.

Re:Could it be... (0)

Anonymous Coward | about a year and a half ago | (#42106091)

You are correct and the author of the article is just confused. Google has many different feedback channels for scoring what they call "search quality." The outsourced armies of manual raters are just one signal among those many channels. Of course, the search quality feedback isn't used to rank anything in the search index (because the index is simply far too big for that). Instead, search quality feedback is used to assess and refine the algorithms that produce the scores in Google's index.

Re:Could it be... (2)

jovius (974690) | about a year and a half ago | (#42107227)

That's how it seems to be, according [searchengineland.com] to one rater:

So, you knew it was Google-related. At what point did you know that you’d be rating Google’s search results?

I knew before I got hired.

One thing I think the SEO community is missing is that this program has nothing to do with SEO or rankings. What this program does is help Google refine their algorithm. For example, the Side-by-Side tasks show the results as they are next to the results with the new algorithm change in them. Google doesn’t hire these raters to rate the web; they hire them to rate how they are doing in matching users queries with the best source of information.

Re:Could it be... (0)

Anonymous Coward | about a year and a half ago | (#42107699)

No wonder their search sucks and has been sucking for about 7 years now. No, No, No, I need technical data, not the best price on whatever I'm searching on, which actually isn't for sale. No Nextag can't sell me the Space Shuttle or much else for that matter. Then there's Alibaba and all the other misfits out there that automatically generate pages that have nothing to do with anything you are searching for. Does Google crack down on the them, no. They don't do a damn thing, and that is why it sucks. A simple letter would do the trick, "Quit spamming our search results or you will be permabanned! You have 1 day to reply.". Get it done if you want me to use your search engine again.

Re:Could it be... (0)

Anonymous Coward | about a year and a half ago | (#42112859)

Well Amazon has something similar as a service: Mechanical Turk [mturk.com]

Besides, machine learning is rather widely used today, but you need something for them to learn from.

My Mom (-1)

Anonymous Coward | about a year and a half ago | (#42105439)

My Mom makes 7k a month on her computer from home.

Re:My Mom (-1)

Anonymous Coward | about a year and a half ago | (#42105483)

While being fucked by Mandingo and Lexxington Steele while your cuck dad eats the cum dripping from her asshole.

Re:My Mom (-1)

Anonymous Coward | about a year and a half ago | (#42105657)

Your mom makes 7k a month on her knees from home. $5 sucky sucky, 1,400 dicks a month, 47 peckers a day, 6 pricks an hour. And no need to take a lunch break!

Re:My Mom (0)

Anonymous Coward | about a year and a half ago | (#42105947)

Psh. His mom can suck 12 cocks an hour minimum. 6 is amatuer hour speed.

Re:My Mom (0)

Anonymous Coward | about a year and a half ago | (#42107321)

To say nothing of other available orifices.

This is just standard expert-tagging, right? (5, Informative)

Anonymous Coward | about a year and a half ago | (#42105499)

It's widely believed that Google search results are produced entirely by computer algorithms...

This is only believed by people who haven't thought about it very hard.

At an abstract level, it makes no sense to think that computer code can be optimized to perform a task without any human intervention. The reason is simple: the task we want the code to perform is always something that a human cares about. So, somehow we need a human to instruct the computer about the goals. This can take the form of a programmer meticulously coding the entire thing, with a particular human-relevant code in mind. Or it can involve non-programmers providing feedback about how well the software is doing at its stated goal (depending on context, these people may be testers, evaluators, users, taggers, etc.).

More specifically, in the case of AI-software, a typical procedure is to have a store of 'pre-tagged' training examples. These are example of problem, with associated 'correct' answers. The training data is used to optimize the AI algorithm: the software can tweak its behavior in order to maximize accuracy of output on the training examples, with the hope that this will then generalize to general use. For something like web-search, where the goal is to make a human end-user happy with the quality and relevance of the results, of course you need humans to assess the quality of the algorithmic results. This is the only way to keep the results relevant. (For search results, this is a continual and iterative process, since the web constantly changes, people are trying to game the system, etc.)

Thus, it's probably better to think of these raters as providing input for evaluating and refining the search algorithms; rather than thinking of them as people who get to uniquely decide the rank of pages. Obviously they will have an influence on the rank of the pages they rate, but overall they are evaluating a rather tiny fraction of the web-pages in the Google database. Thus, when you perform an arbitrary web-search, chances are the results you are seeing are purely algorithmic (none of the listed results were manually rated/adjusted by anyone).

Re:This is just standard expert-tagging, right? (0)

Anonymous Coward | about a year and a half ago | (#42106995)

There's a difference between raters assessing the algorithm, and their ratings effecting the search results. Their ratings may make Google tweak the algorithm, but Google search results are still produced entirely by computer algorithms.

Re:This is just standard expert-tagging, right? (1)

Savage-Rabbit (308260) | about a year and a half ago | (#42110177)

It's widely believed that Google search results are produced entirely by computer algorithms...

This is only believed by people who haven't thought about it very hard.

At an abstract level, it makes no sense to think that computer code can be optimized to perform a task without any human intervention. The reason is simple: the task we want the code to perform is always something that a human cares about. So, somehow we need a human to instruct the computer about the goals. This can take the form of a programmer meticulously coding the entire thing, with a particular human-relevant code in mind. Or it can involve non-programmers providing feedback about how well the software is doing at its stated goal (depending on context, these people may be testers, evaluators, users, taggers, etc.).

More specifically, in the case of AI-software, a typical procedure is to have a store of 'pre-tagged' training examples. These are example of problem, with associated 'correct' answers. The training data is used to optimize the AI algorithm: the software can tweak its behavior in order to maximize accuracy of output on the training examples, with the hope that this will then generalize to general use. For something like web-search, where the goal is to make a human end-user happy with the quality and relevance of the results, of course you need humans to assess the quality of the algorithmic results. This is the only way to keep the results relevant. (For search results, this is a continual and iterative process, since the web constantly changes, people are trying to game the system, etc.)

Thus, it's probably better to think of these raters as providing input for evaluating and refining the search algorithms; rather than thinking of them as people who get to uniquely decide the rank of pages. Obviously they will have an influence on the rank of the pages they rate, but overall they are evaluating a rather tiny fraction of the web-pages in the Google database. Thus, when you perform an arbitrary web-search, chances are the results you are seeing are purely algorithmic (none of the listed results were manually rated/adjusted by anyone).

So... basically, you are saying that it will be a while until Google's systems become self aware and decide to exterminate humanity?

Re:This is just standard expert-tagging, right? (0)

Anonymous Coward | about a year and a half ago | (#42113849)

Often, the human involvement is already *in* the data. That's the idea behind the original Page Rank system. The links between the websites (created by humans) is already in the data, and the algorithms can use that to its advantage.

Machine Learning takes this to another level. If you have hundreds of thousands of videos (think YouTube), the human involvement is already there in the comments and video description. Jeff Dean and Andrew Ng accomplished recognition of "cats" this way in their neural net paper in the summer.

In any case, the humans mentioned in this article are certainly used for a benchmarking/quality measurement.

tl;dr The human map of reality cannot be recreated by a computer totally independently of humans. But often, the human map is already encoded into the data that we feed algorithms, and they can respond to human queries accordingly.

Not enough tech raters (4, Interesting)

poofmeisterp (650750) | about a year and a half ago | (#42105557)

Apparently, if this is the case (which is probably is because Google's algorithms aren't AI), the tech sector needs a lot better rating.

For instance, do a search for a particular model of laptop. The results you get are of course mad online retail shops, but you also get a BUNCH of sites that have nothing to do with the product you searched. They put the names / models in META tags and in hidden or font-size-reduced areas of the page, but the actual page contents itself is just a bunch of crap that has nothing to do with laptops or laptop parts. It's just a bunch of random crap.

Point being, these aren't weeded out very well. Unfortunately, I don't have an example right now, but I know of one that has been in existence for years and still ranks in the top 5.

Oh, and the above is dwarfed by software name / functionality searches 10-1!

Every good thing comes to an end. (0, Offtopic)

concealment (2447304) | about a year and a half ago | (#42105559)

"Don't be evil."

I always liked that motto, but I don't think it's that simple.

When something starts out, its goal is just to get good at what it does.

As time goes on, that role gets more complicated. Questions of profit, making employees happy, the legal department, marketing and monetizing come into play.

Over time every business becomes the same business when it gets to a certain size. It doesn't matter whether it intends to be evil or not.

It's not just limited to business. When your church, anarchist group, drug dealer network, friend group, gay softball league or underground terrorist cell reaches that same size, it, too, gets bloated and useless.

It's part of the human condition. The only thing that seems to be able to save it is charismatic, autocratic leaders like Steve Jobs.

Does Google have one of those?

Is it Google's time -- did Google get too big to not be evil?

The antitrust lawyers are closing in and the competition is getting ready to amp it up. I used Bing recently, and it was nearly as good as Google. I use DuckDuckGo on a regular basis, and it's about 1/2 to 3/4 of the way there.

Could it be the end of an era?

Re:Every good thing comes to an end. (0)

Anonymous Coward | about a year and a half ago | (#42106025)

Do the other search engines get better or is google just progressively getting worse?

Both, probably. (1)

concealment (2447304) | about a year and a half ago | (#42116555)

It seems to me that Bing has improved a lot, and I've watched DuckDuckGo improve quite a bit. Google may be dipping a bit as it tries too hard to localize and customize search results.

Re:Every good thing comes to an end. (0)

Anonymous Coward | about a year and a half ago | (#42106049)

you're trolling right? i really hope so.

Re:Every good thing comes to an end. (0)

Anonymous Coward | about a year and a half ago | (#42106121)

You mean like Dice quietly buying Geeky online properties? At least they put a disclaimer below but how long before all of our last bastion of hope websites become ad ridden crap.

Re:Every good thing comes to an end. (0)

Anonymous Coward | about a year and a half ago | (#42106337)

The only thing that seems to be able to save it is charismatic, autocratic leaders like Steve Jobs.

Does Google have one of those?

Nope. A super creep named Eric Schmidt.

Re:Every good thing comes to an end. (1)

osu-neko (2604) | about a year and a half ago | (#42108235)

You don't even mention human input to search rankings in your troll. Did you have it all typed up before this article was even posted, waiting for the first Google-related post so you could try to slip it in without appearing to be completely off-topic (despite the fact that your troll is, in fact, completely off-topic for this particular article)?

It's foolish to call a legitimate comment a troll. (1)

concealment (2447304) | about a year and a half ago | (#42116509)

You don't even mention human input to search rankings in your troll. Did you have it all typed up before this article was even posted, waiting for the first Google-related post so you could try to slip it in without appearing to be completely off-topic (despite the fact that your troll is, in fact, completely off-topic for this particular article)?

Trolling is a clear practice of trying to provoke response. If you look at my comment history, you see a series of contributions that stimulate discussion.

You calling them a troll suggests you're personally threatened by them. That's fairly typical internet behavior.

Addressing your passive-aggressive question of relevance, yes, this is relevant: Google is claiming it's an automated system, but it's using human beings to fudge the results, much like it prioritizes results like Wikipedia to ensure that an answer is always forthcoming.

That's not "don't be evil." That's outright deceptive.

I'm sorry that wasn't obvious to you but given that you called me a troll for it, I suspect PEBKAC on your end.

total crap article (0)

Anonymous Coward | about a year and a half ago | (#42105703)

from the biased article:

It's amazing how the image Google likes to promote - and politicians believe - one of high tech boffinry and magical algorithms, contrasts with the reality. Outsourced home workers are keeping the machine running. Glamorous, it isn't

i know two people who worked at google and met lots of their other co workers. this concluding statement of the article is total overgeneralization. this sounds like a writer who has NEVER been an engineer. he is painting a marketing slur on ENGINEERING OPERATIONS.

my two cents.

Re:total crap article (0)

Anonymous Coward | about a year and a half ago | (#42105725)

also this is not even news. i watched a video made by google i think about how they used people's responses to construct all their algorithm stuff. that was a special they did. this is a total crap article. im losing faith in slashdot picking up this crap up everyday.

Meh (0)

Anonymous Coward | about a year and a half ago | (#42105731)

Glamorous, it isn't. ®

Still, better search results...

Not new-s (1)

scubamage (727538) | about a year and a half ago | (#42105889)

This was actually listed last year on several black/grey hat SEO websites to help dissect how google functioned. The upside is that with this wider exposure, google may change its policies a little.

Raters gonna rate (5, Interesting)

Aeonym (1115135) | about a year and a half ago | (#42106103)

I've actually been a Google rater. I spent about 2 years total doing it--long enough to become a 'moderator' who ensures quality feedback from other raters--in between, and supplemental to, "real" jobs. Raters give feedback on lots of Google services but it falls into two buckets: ranking the quality of legitimate results, and learning to spot the "spam".

Legit results are easy. Spam is more interesting. For one thing, I didn't entirely agree with their definition of what spam was--that's part of the reason you still see spammy results in some searches. The other part of course is that the spammers are constantly changing tactics. But it was actually kind of fun learning to spot the various methods spammers can use, and know that I was helping to improve search results by getting them off the front page (and hopefully off the top 100 pages).

But I always assumed that rater feedback was used to judge and adjust The Algorithm rather than individual page results. The Algorithm has always been king at Google.

Re:Raters gonna rate (5, Interesting)

Kreplock (1088483) | about a year and a half ago | (#42106521)

I was a rater for 1 year some time ago. My impression was the rating was against results from updates they were considering for the production algorithm. Testing at the QA level. I found it boring and soulless, but a wide knowledge of obscure, otherwise-useless facts really facilitated the work. Sometimes a little-known double meaning for a concept would cause disagreements among raters, and once a moderator hated my opinion so much he had my home phone called several times to demand I change my rating.

FRIST STOP (-1)

Anonymous Coward | about a year and a half ago | (#42106149)

Mystery? Are you sure? (1)

someones1 (1580023) | about a year and a half ago | (#42106247)

"The way these raters go about their work has always been a mystery."

Not really. Anyone with half a brain could get to the second level of the work-from-home LeapForce exam, which is when they issue this guide. Nothing here is a secret or mystery.

They don't have to manually rate every page (1)

hey (83763) | about a year and a half ago | (#42106641)

If a page with a good manually set rating points to another page that other page should enjoy a good rep too. Perhaps for several "degrees of separation".

Are you sure that "relevance" is in there? (2)

whitroth (9367) | about a year and a half ago | (#42106893)

I constantly search for things, and a good half the time, *maybe* a third are relevant. Then there's the times where it completely ignores my conditions. For example, I've searched for a blazer with -ladies, because, duh, I only want men's, and I get hits that explicitly, in the title, say "ladies".

I won't even *mention* Target, who *always* claims to have whatever you're looking for in a sponsored ad on the side, and doesn't....

                mark

Re:Are you sure that "relevance" is in there? (1)

quixote9 (999874) | about a year and a half ago | (#42113949)

Exactly. Google search was amazing early on, when the comparison was to "no search." Now, with a near-infinite web and squillions of SEOs gaming the ratings, it's just half-baked, like all GOOG's products. Half-baked and gamed still brings in billions of dollars for them. Without effective competition that could take any of those billions away, half-baked is going to be all we get.

Re:Are you sure that "relevance" is in there? (1)

SnowZero (92219) | about a year and a half ago | (#42115125)

Try this query instead:
    https://www.google.com/#q=men's+blazers [google.com]
The entire first page is full of items that are exactly what you are looking for.

As the web and search engines both evolve, you may need to change the way you search to get the same information. Something that worked before may not work now, and the critical words or phrases to get the best results are still there but they aren't the same as what they were in the past.

In your particular example, the exclusion is far too weak, as "women's blazers" matches [blazers -ladies], and once you start using unusual queries (very few people will use exclusions) the search engine will tend to be more literal since it doesn't have the statistics of many previous searchers using those terms to go on.

raters (0)

Anonymous Coward | about a year and a half ago | (#42107007)

Well I gues...raters gona rate.

two quotes (0)

Anonymous Coward | about a year and a half ago | (#42107493)

"One thing I think the SEO community is missing is that this program has nothing to do with SEO or rankings. What this program does is help Google refine their algorithm. For example, the Side-by-Side tasks show the results as they are next to the results with the new algorithm change in them. Google doesn’t hire these raters to rate the web; they hire them to rate how they are doing in matching users queries with the best source of information."
Source: http://searchengineland.com/interview-google-search-quality-rater-108702

"Ideal candidates will be highly active users of Google's search engine and other products; use Google Play at least once per week; use Google+ more than once per month and have more than 11 people per circle and have a Gmail account with web history turned on."
Source: http://www.leapforceathome.com/qrp/public/job/32

So that's why I once had 15 minutes of fame... (1)

dpbsmith (263124) | about a year and a half ago | (#42110327)

There was a period of a couple of years when a web page hosted on my ISP's freebie 15 megabytes of web space was the top hit for a particular Google search. It was a good page--a lay discussion of a technical topic--and I enjoyed the ego boost, but I always wondered why since I was not aware of it's being linked from anywhere, let alone any high-traffic or high-creditibility page. Now I think I know.

(I have since contributed that page's content to Wikipedia. The article has evolved with contributions from others but is still very recognizably mine... and I recently received a the left-handed compliment of an angry email from someone who'd stumbled across my own web page and complained that I had plagiarized it from Wikipedia!)

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...