Online Reputation Is Hard To Do

Symblized writes "A new article from InformationWeek argues that not only does the Web need ways to verify identity, it also needs better ways to measure reputation . The article uses Digg, Wikipedia, and eBay as examples and muses whether their models could be applied more widely. There's also a profile of Opinity, a company that tried to introduce a reputation system and didn't make it. Choice quote from a source in the article: 'The idea of a transferable, semantic reputation is identity nirvana.'"

Trust is the currency

[Raindance]

"Trust is the currency of the participation age." -Jonathan Schwartz

This is the $64,000 question. Building a reputation/trust system is very difficult. Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

I'm volunteering at Citizendium, which is another possible datapoint. We're assuming that real names and respecting verifiable expertise will allow us to benefit in some fashion from existing scholarly reputation systems, and to build a more cohesive community.

Eventually, I think it'll be feasible to layer reputation and credentials (for sites that care) on top of a system like OpenID. People will be able to choose what reputation/credentials to share with which site. Information that you want to follow you (e.g., "I have a BA in Math from UCLA" or "I have excellent karma on Slashdot") will follow you across sites.

But yeah, it's a very difficult problem. Figuring it out is a big, potentially very lucrative issue.

And what do you buy with that currency?

[khasim]

It seems that everyone trying to "solve" this "problem" doesn't know what they're trying to achieve.

So what if you can make a perfect pseudonym identification system? What does that achieve for you? What do you accomplish beyond that?

Does it really matter to anyone else if your Slashdot 'nym can be verified to match your 'nym's on a dozen other boards? Who really cares if you have excellent karma on Slashdot?

Re:And what do you buy with that currency?

[gbjbaanb]

it matters because if you have good rep on slashdot, chances are you're not a complete mumpty. On the other hand, if you have a dreadful karma on slashdot, you'll be saying the same old pants on other sites too.

It really ensures that if you post good stuff somewhere you can be trusted to post good stuff on other sites too. What that means to a particular site depends on that site, for something like ebay that can matter quite a lot, at least it would allow good posters to be recognised as such, and then I think sites would start to implement policies on posting that restrict non-recognised 'nyms until they gain a good reputation.

Re:

[jlarocco]

I'm not convinced that's worthwhile. If I like trolling on web forums, how does that imply I wouldn't be a trustworthy seller on eBay? How does a good online reputation in any way guarantee that a person is qualified to write about a certain topic in Wikipedia? I don't see the connection.

Re:And what do you buy with that currency?

[Original Replica]

In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere. Just like if you act like an asshat at the company picnic, it effects you back in the office and possibly gets back to your friends at home. And yes, I think if someone is a carebear in WoW then they are a more trustworthy eBay seller, and someone with intelligent /. posts is more likely to contribute intelligently to Wikipedia.

Re:

[gronofer]

So in the future, acting like an asshat for fun in formus would relfect on you elsewhere. Just like if you act like an asshat at the company picnic, it effects you back in the office and possibly gets back to your friends at home.

These systems aren't going to achieve that completely. There's nothing to stop somebody creating a new id for their asshat sessions and reserving another id for anything connected with their real name. With a name like "Original Replica", you probably realise that already.

Even

Re:

[Anonymous Coward]

It's pretty funny how naive people are when it comes to anonymity online. What makes you think someone's behavior in an online video game would in any way reflect how that person might conduct business at an auction site? That's a dangerous and susceptible way of thinking, and I can think of more than a few scammers who banked on exactly the kind of presumptions you are unfortunately making.

Hell, anonymity is what leads to mixed behavior from the same person in the first place. It all depends on contex

Re:And what do you buy with that currency?

[Original Replica]

You don't think the fact that you occassionally like to "flame and act silly" or play "an annoying lowbie ganker" is telling ? It tells me that when you can hide behind the veil of Anonymous Coward, you don't have much respect for the people around you. Because let's face it, when you do those things you are causing real frustration for real people, and you do it for fun.

Don't be so gullible as to think someone's behavior is necessarily consistent no matter where they go on the internet.

This would be a case of adding "but ONLINE" and thinking it's something new and different. I don't think anyone's personality is 100% consistent as they go from one social setting to another, but it is all facets of the same actual person.

Re:And what do you buy with that currency?

[Sigma 7]

In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere.

I'll play devil's advocate here...

Let's say you comment on the theory concerning the use of demolition explosives on one of the world trade centers - pointing out that the collapse of the WTC doesn't look like other building demolitions [google.ca], or that the "symettric demolition" claim is incorrect.

However, the conspiracy theorists on the site are extremely fanatic about their theory (as opposed to a more moderate site that tries to investigate properly.) As a result, you receive a large quantity of negative feedback that attaches itself to your online reputation.

Other things that can affect you would be playing RTCW:ET, where you get kicked from a server for n00bism as you didn't dodge the three panzers that get fired into your local area (because another player thought you should have.)

And my personal favourite - just claim you support Bush. Your reputation would instantly tank.

Re:

[siriuskase]

However, the conspiracy theorists on the site are extremely fanatic about their theory (as opposed to a more moderate site that tries to investigate properly.) As a result, you receive a large quantity of negative feedback that attaches itself to your online reputation.

The data point isn't simply that i have excellent karma, it is that I have excellent karma on /. which itself has a reputation that is a factor in calculating my reputation. You can take this out another level and rate the reputation calcula

Re:

[Blakey Rat]

Then there's the Bush factor, certainly a killer for any reputations.

Are you seriously claiming it's a good thing to ruin somebody online reputation because of their political beliefs? Holy crap, man.

Re:

[siriuskase]

A lot of the weightings would be personal, my favorite rating services would get higher ratings from me. My weighting may not even affect anyone else, or there might be a system where all who use a particular service have their scores aggregated somehow. Seems a lot like social networking to me, friends and friends of friends, netflix friends, myspace friends, etc.

Re:

[Anonymous Coward]

I don't think it effects anything... I used to carebear a bit in WoW, and I'm a jerk.

Re:And what do you buy with that currency?

[Simon80]

Regardless of what kind of eBay seller you'd be, this system would let people shun you for being an asshat somewhere if you used an important identity to be an asshat with. This would relegate most asshattery to anonymous identities, which would mean that sites that want to eliminate asshattery would simply require that all participation come from an identity with a decent reputation. I'm not sure if this is a good thing or not.

Re:

[Jane_Dozey]

Well that brings up the problem of new people trying to not be an asshat and gain a good reputation. If no forum will allow people without a good reputation to join how do you get one?

Re:

[Simon80]

There's a number of problems it brings up, like privacy issues. Like I said, it's not necessarily a good thing. In this case, though, the user can start building their reputation on sites that allow new users to participate until they "level up", eventually gaining access to other parts of the internet. Thus, web 2.0 will be unwittingly turned into some sort of dystopian MMORPG. Seriously though, that would work, because plenty of places, like /., would allow users to participate even if they don't have

Re:

[rtb61]

The funny thing about this story, is who is going to fund research into the best way measure reputation. It is so wildly counter modern marketing techniques which of course is all about creating and sustaining a false reputation i.e. this cheap junk product in reality sucks, but we are going to convince you that it is wonderful, that you have to buy it and that it is really valuable.

Now I could believe them focusing research on how best to falsify reputation, or even to destroy truthful good reputation, b

Re:

[Simon80]

Wtf, since when is that a hallmark of being female? Or are you just another one of these paid trolls you speak of?

Re:And what do you buy with that currency?

[Achromatic1978]

You'll buy one. No, seriously. You'll see a whole host of forums, sites, etc, spring up where you either "have a good reputation with iDentify", or "Paypal/MC/Visa $15" for membership.

The merit of such a system is not particularly high. Neither is the probability of those sites making a profit.

And then you'll have reputation farms, like we now have link farms, that spammers will use to build reputation.

Re:

[gronofer]

Well that brings up the problem of new people trying to not be an asshat and gain a good reputation. If no forum will allow people without a good reputation to join how do you get one?

Forums are unlikely to disallow people with no reputation. I think the more difficult problem would be the advancement from no reputation to good reputation. In the worst case, it doesn't matter how insightful you make your comments, they will still be buried low down the pecking order since you don't have a good reputation

Re:

[mysidia]

For one thing, it says you are probably not a spammer; if you have a very good karma on slashdot. (You didn't just run by slashdot to post a piece of junk)

For the reputation system to really be useful for filtering purposes, a method needs to exist for your reputation to become "sullied" if you do spam somewhere, or abuse your reputation; in that case, your "reputation of good karma" on the pseudonym@SITE would need to have the bad part attached to it, I.E. if you spammed on some other site, and used

Re:

[siriuskase]

A good reputation system also needs to be resilient against people maliciously/frivolously attaching bad marks to other people's reputation, for example: claiming person X is a spammer, who never actually spammed. I.E. it needs a method of verifying or authenticating any credentials and negatives/qualifiers on those credentials, that get recorded and presented. One possibility is to require corroboration, and to quickly expire negative reports that are not confirmed by a sufficiently trustworthy source.

That

To clarify that ...

[khasim]

it matters because if you have good rep on slashdot, chances are you're not a complete mumpty.

So you're saying that it would help filter out a majority of the "complete mumpty".

That's a possibility. But it would be even easier to just use Slashdot's reputation/moderation system on your own site. That would solve the "complete mumpty" problem while also solving the problem of someone with excellent karma for his programming knowledge posting his conspiracy theories on your site.

And it automatically tunes itself to your audience.

It really ensures that if you post good stuff somewhere you can be trusted to post good stuff on other sites too.

Not really. Check back on the "creationism vs evolution" stories here.

What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).

You achieve all the same benefits without the problems just by having your own reputation/moderation system.

Re:To clarify that ...

[dkf]

What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).

You achieve all the same benefits without the problems just by having your own reputation/moderation system.

I suspect that there will be a lot of correlation between sites, as people who act like asses in one place are more likely to do so elsewhere. On the other hand, I also think that what will actually emerge is that the sites that correlate cluster in groups: if you have a good rep on site A, you're more likely to have a good rep on site B and a bad rep on site C, and you probably won't frequent D at all. Given all that, someone with a good search engine and a lot of cleverness will be able to start mapping out what these virtual communities actually look like and other sociological/anthropological stuff like that.

But will sharing reputation systems help sites? I think so, eventually, but not for many years yet. Until the reputation clusters have been found, sharing is as likely to introduce needless pollution of reputation systems as it is to enable reputations to be built up quickly. (Could there be a single reputation cluster? Maybe, but I suspect not; people are too inclined to divide the world into "us" and "them" for it to work out.)

One thing that might come out of reputation research is that it might become possible to use the reputation clusters to predict, from someone's interests and interactions, which sorts of sites they'd like to visit. OK, that does sound backwards, but it should guide people to where they won't want to make a total fool of themselves on a regular basis (yes, even the griefing pranksters; after all, when amongst the fools the foolish are sages and the wise foolish.) It may also eventually be possible to join the reputation clusters up, but using negative links (so reputations on sites for followers of Xenu who believe in ID and the supremacy of feng shui of placement of feeding bowls for their chihuahua will negatively reinforce reputations here) but I doubt that will help any time soon. There's a revolution waiting to happen here, but since it really involves lots of people, it'll take time to brew.

On the other hand, it is sensible to start working out what technological steps are required to enable specific bipartite reputation sharing, as well as looking at how to build sane single-site reputation systems. For example, slashdot's is pretty good in that it isn't easy to totally game the system while being mostly self-regulating, but can it be bettered without input of data from outside sites? If it can't be greatly improved, how difficult will it be to export the system to other sites? (It's late: I'm sure you can think of other aspects, but I can't right now.)

Re:

[Original Replica]

But it would be even easier to just use Slashdot's reputation/moderation system on your own site

Is /. now offering a "moderated forums that don't suck" site-design package? We have all seen too many forums with no good way to have intelligent discussion or accommodate branching discussions. There is a certain amount of groupthink here but it is also the only place online where I have seen people admit to being wrong or actually change their viewpoint based on new information. It's hard to say who much o

and...so what?

[Wabbit Wabbit]

I've got mod points, but I think I'd rather participate.

it matters because if you have good rep on slashdot, chances are you're not a complete mumpty. On the other hand, if you have a dreadful karma on slashdot, you'll be saying the same old pants on other sites too.

And so what? Is all of this really so important? I find it fascinating that so many people on so many sites care more about their "reputations" than what they post.

It really ensures that if you post good stuff somewhere you can be trusted to post good stuff on other sites too.

Does it? Sometimes I don't WANT my "good" reputation to follow me. I like acting like a goon on something awful and like a lolcat-loving ding-dong on fark and like a...well...never-you-mind-like-what on consumptionjunction and 4chan.

When (and where) I want to be serious, I am. Others see it quickly enough too. It doesn't take long at each site I join for people to realize that I'm a "good poster". Honestly, it isn't complicated. Stay on topic, write well, be helpful, and the rest follows. Such has been my pattern over the years at sitepoint, namepros, webhostingtalk, and even here.

\Perhaps it's because I'm old
\\And still use slashies
\\\(reversed because slashdot doesn't like 'em forward for some reason)

Re:

[lawpoop]

And so what? Is all of this really so important? I find it fascinating that so many people on so many sites care more about their "reputations" than what they post.

Personally, I like having the karma system at slashdot, so I don't have to wade through endless FP! GNAA! GOATSE! to get to the decent commentary. There are plenty of trolls and vandals out there, and a reputation system would automatically filter their junk out.

Does it? Sometimes I don't WANT my "good" reputation to follow me. I like acting like a goon on something awful and like a lolcat-loving ding-dong on fark and like a...well...never-you-mind-like-what on consumptionjunction and 4chan.

So have one username for one site, and other username where you do your GNAA postings.

Exactly

[Moraelin]

I was going to post much the same thing, so it's refreshing to see someone else thinking that way.

The thing about Slashdot's karma is that it creates groupthink. As you've said, too many people care about it instead of just posting what they think. So they post what they perceive to be the popular opinion, even if that's not what they really think, or even if it's contrary to what they really think.

Frankly, I think groupthink is a bigger problem than even the goatse links. Groupthink is where rational information exchange dies. If you look at the worst bible-thumping communities, or at rabid theocracies, or at the worst excesses of history, and some of its biggest mistakes too, almost all were based on groupthink. Take a million people who individually think "jeez, X is stupid and evil" and put them in a big group where they think that everyone else is fundamentally and rabidly pro-X... and watch them all start chest-thumping for the very thing they secretly despise. Just to get brownie points with the rest of the gang.

When a whole village went and cheered about one of them being burned at the stake as a witch (for bonus points when everyone knew it's a bogus charge and the real reason is something like: widow without sons inherits some land, some rich guy wants her land), that was groupthink. "OMG, I can't let the other ones even think I'm not a rabid fundie. Why, my popularity would go down."

At the risk of tempting Goodwin's law (although it's not a comparison): when a few million Germans cheered about invading the USSR, that was groupthink too. "OMG, I can't let the others think I'm not patriotic."

And in our own times, when you look at such things as bible-thumping communities, or at the broken high-school culture where being smart is uncool and being an airhead is the apex of fashion... guess what? That's groupthink too. Once the ball got rolling, even kids who do understand that their future job does depend on it... still go and insult the nerd, because that's what brings them karma points with the rest of the group.

So, to cut a long story short, I actually _don't_ want that kind of global karma. I actually _want_ people to come forth and say what they think, and not what they think would be popular in that community. I want people to actually come forth and say stuff like "this war is bogus" or "the PATRIOT act is unconstitutional" and not devolve into sheep thinking "OMG, I can't have it follow me for the rest of my life that I'm not patriotic or that maybe I have something to hide". Even if it's something as unimportant as a games forum, I actually want people to come forth and tell me the bad parts about it, so I can make an informed decision. I don't want more of them to think "OMG, if I say anything bad, I'll come out as a troll." Etc.

Reputation vs. anonymity

[mollog]

Reputation and anonymity are mutually competing goals. Early in the history of the 'net, anonymity was something that people at first exploited for a wide range of reasons; honesty without repercussions, viciousness without repercussions, etc. Eventually, technology caught up an much of the anonymity has become relative. If they want to find out how you are, they usually can.

I think that much of the recent alarm about loss of privacy is the result of us becoming accustomed to thinking we had anonymity a

Re:

[Achromatic1978]

And if you have good reputation on SomethingAwful? Encyclopedia Dramatica? Slashdot? Good reputation is contextual. You can get a good reputation on SA for being able to make humorous Photoshop images. ED, by trolling. /. by towing the Linux/Google "line". That means very little as to having a good reputation as an eBay seller, for example.

Rep Currency

[Z34107]

Microsoft's "Windows Cardspace" probably solves this problem already.

Granted, Microsoft is envisioning a system where an "identity provider" like a bank could look at your card and come up with a credit rating and whatnot, but using it for something less lofty, like a universal karma score, isn't that much of a stretch. Not much of a .NET guy myself, but it seems reasonable.

Re:

[Blakey Rat]

... except you can get a good reputation by being a shrill, annoying open source zealot/Microsoft hater on Slashdot. That would be a negative on the vast majority of discussion sites out there, and plain unwanted on discussions sites for (say) Microsoft software.

Slashdot's karma means nothing except on Slashdot, for no other community has the same biases Slashdot has. Hell, you can get your karma busted down here for saying you don't really like Nintendo games... that sure doesn't apply anywhere else on the

Re:

[JWSmythe]

Of course it'll rain tomorrow, we're in the middle of a hurricane! :)

Someone else had it more right. It really doesn't matter. Post a bunch of somewhat intelligent (i.e., not troll) messages, and you'll get a good karma.

But, if you're a Windows-loving, Linux-hating, Pro-War-Budget, No-Civil-Rights user, no matter how intelligent your posts are, you're going to have a very pathetically low karma. At least here. If you're posting on a Microsoft blog, you'd pro

Re:

[The One and Only]

As with real reputation, there should be a way for a bad person to be good (even excellent) and rebuild lost reputation.

Re:

[alphamugwump]

Oh, the irony. The utter irony. The one man who speaks truth about moderation, gets modded troll. Fortunately, I score trolls +2.

Getting excellent karma is indeed easy. I spent a whole weekend on slashdot, and now I have excellent karma. It's that simple. All you have to do is post something slightly different from the usual copyright arguments, gun-control arguments, and so on, and you'll get 5s all over the place. Even better: act like you know what you're talking about. I've made up complete bullshit abo

Re:

[Kadin2048]

Does it really matter to anyone else if your Slashdot 'nym can be verified to match your 'nym's on a dozen other boards? Who really cares if you have excellent karma on Slashdot?

It might. I think the best possible solution would be to let the destination site look at whatever pieces of reputation you choose to provide, and then weight them however it wants to.

E.g., let's say I set up a blog. I know most traffic is driven to it by people clicking on my site's URL in my Slashdot comments; therefore, it might

Re:

[JWSmythe]

Well, some of us have been lucky that we've managed to get our own names (or pseudonym in my case) almost everywhere. But some folks don't use quite as unique names.

I guess if there was a pseudonym reputation ranking facility somewhere, it would need to have every pseudonym for each resource listed. Ahhh, a tall order just got taller.

But, our pseudonym is such because we don't necessarily WANT everyone knowing who we are.

JWSmythe (my pseudonym) is kind of wel

Re:

[Tribbin]

This is a bit offtopic I must say but it just came to my mind because of your post.

A week ago I was impressed by somebody's (who I will not name) post so I read his previous post on different subjects. Although he almost got modded down as often as up, I was impressed by his honesty. He dare to say things that I'm quite certain other people would not say, just because they are afraid to offense people or be modded down for other reasons. His posts were more insightful than they appear.

The next thing that ca

Re:And what do you buy with that currency?

[Eivind]

Yes it matters. A lot.

Online, a lot of clues that are present in the real world are absent. We instinctively assign some level of trust to people we meet, for different purposes, based on a lot of variables, some of them we're aware of, others are subconscious.

You let your neighbour have your house-key to water the flower. But you wouldn't do that with *ANY* kind of neighbour.

You let someone babysite your kids. You let a friend borrow $50 'cos his credit-card is broken. You wouldn't to everyone, it's a matter of trust.

You trust someones judgement on some issue -- because you know that they are experts in the field and have a track-record of good judgement.

Being able to build trust in a pseudonym, and being able to prove that you are that pseudonym is very useful. It allows people to trust you who wouldn't otherwise.

To avoid abuse, it is nessecary that *you* have complete control over what aspects of your trust you share with which people and which companies.

So, what do you want to achieve ? World Domination offcourse ! *grin* No seriously, a million little small things, each of which may be unimportant, but the sum could be huge. Some examples used *today* include:

• If you've got excellent reputation, Ebay-buyers generally won't mind paying first, getting the item(s) afterwards. This would be quite risky -- except you know that the seller has sold 471 things before on similar terms, and -zero- of his customers complained.
• Hospitality-club use a trust-system to allow you to let complete strangers sleep over at your house, or vice versa, with a much reduced risk of any unwanted problems. Sure, *you* may not know this person, but it helps if 50 other people do -- especially if 5 of those are your friends.
• Slashdot use a trust-metric to let people with a track-record of sane comments be sligthly more visible.

In a universally networked world (which we're rapidly approaching anyway) with strong trust-systems, you could stop a complete stranger on the street and ask to borrow his car -- and he'd actually consider it. He wouldn't know *you* but, he'd be able to know a lot *about* you -- if you choose to share it with him.

Re:

[ajanp]

Trust in an online world is a tricky thing. Despite how noble the intentions might be, any information you reveal about yourself always has the possibility of being exploited when you reveal it to such a large audience (what do you reveal so people can verify your credentials, yet so you can still remain anonymous enough that it can't be exploited). With increased exposure, you open yourself up to a lot of people who might use that information to do harm.

I remember somebody posting a comment on /. about

Re:Trust is the currency

[ralphdaugherty]

I have some nasty comments about this, but first let me switch to AC.

Re:Trust is the currency

[zCyl]

Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

That's because Slashdot's system puts only minimal emphasis on individuals, and very high emphasis on selected adjectives of value. Comments do not simply get moderated up or down, but have to be moderated with a chosen adjective, such as "insightful", "informative", "funny", etc. This really helps keep people's heads on straight, especially with the presence of meta-moderation, because people then have to agree on what these words mean. The end result is that posts are usually moderated in close proximity to these labels.

The karma attribute is used only as an accessory to this content-based moderation, to provide some inertia to the community's character. It's not really a reputation centered system.

Re:

[UbuntuDupe]

Comments do not simply get moderated up or down, but have to be moderated with a chosen adjective, such as "insightful", "informative", "funny", etc

Yeah, it's not like you can just do an "overrated" or "underrated" that won't get reviewed.

Oh, wait...

Poster identity vs Post identity

[kahei]

It's also worth noting that the identities of /. posters are very weak. People reply to the post, not to the poster, and relationships pretty much never last longer than the thread they are in. I can't actually name a single /. user name except my own.

I used to have a .sig saying: "If you recognize me at all, you probably recognize me by this .sig" and I think it was true.

Compare this to sites like Fark where rivalries and stereotypes rule. Hey, time for Bevets to post how we're all going to hell! Time

Re:

[pla]

We're assuming that real names

You lost the present audience with that one statement.

Which, interestingly enough, makes a point of its own - You perhaps have a level of real-world credibility that you would like to extend a priori to your online life. Many of us would rather keep our "real" lives 100% separate, and accept the penalty of needing to prove ourselves for each online community in which we participate.

I submit myself as such an example - I have 3.5 degrees (1.5 of them in "hard" subjects)

Re:

[JWSmythe]

Pla,

I have no academic accomplishments, yet I am well spoken. I have life experience in many fields. That (and my razor sharp wit) has secured my 'Excellent' karma.

Reputation is in the eye of the beholder. Just because 10,000 people say I'm cool, cute, smart, funny, or an all around nice guy, does that mean that YOU will think so? Probably not.

I generally use one pseudonym, but there are others, and absolutely no ties between them. I don't tie

online resumes

[cashman73]

As mentioned in the article, online resumes are one easy way of verifying credentials. But even that's not perfect, as they can be faked quite easily. Heck, people have been fudging information in their resumes even before the internet was invented!

Maybe somebody like Google can use their search engine technology to develop an improved algorithm that would perform multiple searches across multiple websites and databases to come up with some type of score rating an individual's credibility. But even this h

Wikipedia needs reputation system

[lawpoop]

I think Wikipedia is a site that really needs to somehow integrate the reputation of it's contributors into the articles. I haven't kept up with the structural changes they've made in the past couple years, but a lot of the editing work seems to be undoing trolling and vandalism, and also participating in edit and revision wars. I could be wrong at this point.

But if wikipedia had a reputation system ( other than just being banned or allowed ), they might automate contributions from reputable authors ( and check on the actual contributions later), while authors who are less reputable may have their contributions queued for review before they are published.

Furthermore, a casual user would be able to have a more savvy understanding of the reputability of any article or section of an article if it is tagged with the reputation of its' author.

Reputable authors might be able to also tag the contributions of others, such that the text or information itself gets a reputation. That would help users make a judgement about the validity of information on Wikipedia.

Instead of pushing the mechanics of the actual editing of articles behind the scenes, and just presenting a 'final' article to the end-user, let's formalize the process and enfranchise users into the process of judging the validity of articles.

Hear, hear!

[mangu]

I think Wikipedia is a site that really needs to somehow integrate the reputation of it's contributors into the articles.

Indeed! Here's my own anecdote on that: I recently tried editing an article on a certain Posada Carriles, a man whom the Cuban government and Wikipedia call a "terrorist".

I was browsing the Cuban government site Granma [granma.cu] where they had a list of what they called evidence against Posada. One item was an AK-47 rifle, another item was a box of 5.56mm ammo for that rifle. It doesn't take much of gun expertise to know that NATO ammo doesn't go into an AK-47, and I tried to put that in a paragraph criticizing the accusations against Posada. I don't know the guy, for all I know he could really be a terrorist, but you aren't going to convict anyone in a civilized court of law with that kind of "evidence".

I was thoroughly flamed by someone about that. It seems that Cuban government sympathizers are carefully patrolling any critical statements about the dictatorship. If Wikipedia had a reputation system, the commies would mod me down for presenting a balanced view in their rant against Posada, but I would recover my karma through my other contributions. OTOH, fanatics would find it too troublesome to fake an interest in subjects other than their favorite and their karma would suffer from that.

Re:

[lawpoop]

That's a good anecdote, but I think wikipedia has a policy against 'independent research'. That means that you can't put original information or ideas on wikipedia; you can only summarize other information. So your information doesn't belong on wikipedia. Doesn't mean you needed to get flamed, though.

I don't know, maybe you could put up a page explaining the illogicality of this 'evidence', and then cite it. I don't know about wikipedia's policy about validity of references.

Re:

[Cyberax]

There ARE modifications of AK-47 and AK-74 for NATO ammo.

Re:

[owlnation]

As the parent indicates, the trust issues with Wikipedia are many and varied. The most worrying, and seemingly increasingly prevalent, is the strength of cabals.

As well as the example in the parent, Scientology and Ayn Rand examples could also be quoted here among many many other possibles. In fact if you are a political organization, pressure group, NGO, or a sinister cult, and you don't have your own wiki-cabal, you are failing in your mission.

There's simply no way to put reality into wikiality in

Wikipedia isn't scientific

[mangu]

You cannot make an original assertion directly on Wikipedia, even if you feel that your argument is logical. You should link to a reputable source making that assertion.

Well, let's see how all this happened, step-by-step.

I was browsing Granma, looking for news about Castro's health when I saw this link to evidence against "the terrorist". I had never heard about this guy, but the so-called evidence was very obviously fake. I looked in Wikipedia [wikipedia.org]. The opening sentence in that article today, 2007/06/03, says

Re:

[MaelstromX]

This is, in no way at all, an "original idea", it's a carefully constructed criticism with sources fully cited. As I said in my other post, this is a crude attempt at framing, based on that evidence alone, Posada Carriles would be acquitted in any civilized court of justice.

Of course it is original. As far as your posts show, YOU, and nobody else with any authority on the subject, are claiming that Posada was framed. While the evidence you have supplied comes from an external entity and it might make your c

Re:

[mangu]

Of course it is original.

Not under the Wikipedia guidelines [wikipedia.org] which state that "Original research (OR) is a term used in Wikipedia to refer to unpublished facts, arguments, concepts, statements, or theories." I presented two different publications.

YOU, and nobody else with any authority on the subject, are claiming that Posada was framed.

Correction: me and about 26100 other people [google.com] are claiming that. I presented two different sources, which were in clear contradiction, and noted that fact. The article as it

Re:

[MaelstromX]

Not under the Wikipedia guidelines which state that "Original research (OR) is a term used in Wikipedia to refer to unpublished facts, arguments, concepts, statements, or theories." I presented two different publications.

You presented zero publications which explicitly asserted that Posada was framed. Honestly, I hadn't seen the other reply to your post when I posted but he makes your misunderstanding abundantly clearer, much better than I was able to. Reread that other post, over and over if you have to, u

Re:

[gronofer]

I re-checked the photograph in Granma. I looked in a book, "An Illustrated Guide to Weapons of the Modern Soviet Ground Forces", Ray Bonds ed., Salamander Books, 1981, ISBN 086101 115 5. There in pages 136, 137 was a photograph of an original soviet AK-47 which was identical, to the last rivet, to the rifle pictured in Granma. The rifle in the photograph wasn't modified, it does not fire 5.56mm cartridges, the Cuban evidence against Posada Carriles was fake. I linked my criticism to the photograph in Granm

So, Wikipedia is a piece of shit?

[mangu]

A central tenet of Wikipedia is that it does not publish original research. The quality of that research doesn't matter. ... You have to link to someone making that assertion ... The policies on this are in fact crystal clear: Wikipedia does not accept original research. Wikipedia requires you to cite sources even if your claim is provable with mathematical certainty.

Look into my posts above, you'll find the sources I cited, one from a Cuban government source stating that a box of 5.56mm cartridges were to

Hmm

[Seiruu]

The only reason online reputation is hard is because online identification is hard. Once you're past the identification and privacy issues you could go Google: your single/central/one point rated identity, linked with all your accounts from all over the place which should give you some sort of a global and more specified ranking (karma on ./, trustworthiness on ebay, whatever rating/googlerank on google/amazon) for people to search for.

Web design

[Hao Wu]

Companies will often try to maintain "reputation" by using a rigid, non-interactive web design. With Flash, for example, a website becomes just a commercial to sell the perception of quality, or the illusion of branding.

(Isn't that right, all you "Hubsters"?)

What about LinkedIn?

[Realistic_Dragon]

LinkedIn has a reputation model that works in a limited sense. It tells you the people on the chain between yourself and an individual, up to a certain length.

Since you know (by definition) the first person on the chain you can ask them to make enquiries along the chain about the person you want to know about. It lists up to 3rd degree associations, ie your friend knows them or your friend knows a friend of theirs. Surprisingly effective for finding out about someone you want to hire, in a general sense at

Re:

[crAckZ]

that is a good idea. but one weak link will destroy the chain. what if one of those people gets hacked? malicous software is placed and then the whole chain gets it and it spreads. as of right now i see no viable way of doing this. there are to many factors and was to hide any identity or truth. if it was accomplished what if your information leaked? soemone signed you up for alot of questionable sites and your ID was trashed. then you have to take the time to rebuild and correct all that information.

Re:

[Phil Resch]

Regarding your P2P network of friends ... have you heard of WASTE? If I understand you correctly, it's more or less what you're suggesting. At any rate, its Wikipedia page is worth reading. It has some interesting information, and a lot of links to follow: http://en.wikipedia.org/wiki/WASTE [wikipedia.org]

Re:

[Mycroft_514]

Buzzz..... sorry, wrong answer.

Look, I have 3 kinds of links in my Linkedin lists. Those that know me professionally. Those I am reconnecting with (how much about my profession does a guy who was in my dorm in college know about?) and finally HEADHUNTERS. Yup, linkedin has already been taken over by headhunters. I have multiple HR people from consulting firms I used to work for pulling in hundreds of contacts. Yet these people know nothing of my current reputation.

Linking RL with the IL

[FoxNSox]

Even still, it is hard to rank someones reputation based on a numbers system. In alot of forums I post on, I am a regular poster with a high rank. However, alot of people have an issue with me because of my free speech (the beauty of the internet). Is there really a standardized way to determine reputation? It really has to do with the context. If you are on a programming forum, you may rank someone based on their aptitude for a specific language, or their problem-solving skills. Conversely, if you are on a political debate forum, ones reputation may be based on how fluently their opinions are expressed.

Re:

[DaleGlass]

I run a reputation system on Second Life.

My solution to that was to make it a graph. Scores are calculated depending on who you are and who you're looking at. So Alice likes Bob, who likes Carol, who likes Dave. Alice builds a tree, and adds up all the paths leaving to the same person.

This works well for me, the only problem is that it's very resource intensive. Cost increases exponentially with depth.

On the other hand, it's very resistant to attempts to disrupt it. Simply getting extra accounts doesn't wor

Instant Mashed Repuation!

[rueger]

The problem is that everyone wants to know right now who is trustworthy and who is not.

Building a reputation takes time, often a lot of time. Amazon's reputation is built on several years of good service, good web design, and overwhelmingly positive customer experiences.

Facebook and Digg don't have that track record, and until they do will not enjoy the same level of trust.

Any system designed to give a stamp of approval needs only one mistake to become untrustworthy. Unless it can be nearly 100% foolproof it won't be effective. And given the number of supposedly trustworthy businesses who are anything but, I'd say that rating reputation is not likely to happen soon.

The First Law of Cyberspace

[Hobbex]

There is no such thing as negative trust.

(Once you accept that, the rest isn't so hard.)

heatware

[mscdex]

Heatware [heatware.com] Most frequently trusted on many For Sale/For Trade forums because of their strong stance on scammers.

What about well-prepared people?

[CPE1704TKS]

I haven't used my name for posting on the Internet since 1997 when I realized that dejanews.com would keep my newsgroup postings forever (even that was with a somewhat random e-mail address). I literally don't have any internet presence with my real name unless it's inadvertent (ie. a news release from my employer) but the good thing is that my name is so common I would be hard to find anyway.

So in it's place, I created a whole shitload of false identities that I post under, one of them about 10 years old now. Mainly on forums and newsgroups for work purposes, etc. If you searched for this particular identity, you would probably fine hundreds of posts (including many on slashdot) some of them truthful, some of them fake, with various opinions of topics.

Every few years I will discard an identity or create a new one, for various reasons. I even have a fake lj blog that I've created just for the purpose of having that sense of "credibility", just in case I need it. I usually update that every few weeks, with something that I read on someone else's blog, but changing the words around just enough so that I can't be googled and exposed as a fake. I make sure each identity has a different way of typing, different levels of typos or capitalization, etc. I don't think you would be able to properly gauge the "credibility" of this person at all.

I doubt I'm unique and there are probably scores of people doing the same thing. As internet users get more and more sophisticated, how will internet credibility really be gauged unless you actually meet someone face-to-face? I was even contemplating getting a pay-as-you-go cellphone with no traceability (paid with cash at a store in a different city than where I live) just in case I needed to talk with someone offline. I'm doubtful you can really establish credibility to the point where it's better to just assume that everyone is lying and be on the guard all the time.

Re:

[Tatisimo]

Same here. I keep quite a few different characters, and shed the old ones in order to keep my privacy. Everybody I know does it to keep their past from haunting them. Funny how I look back on my old personalities and realize what a moron I was. It'd be horrible if people took the posts I made when I was 14 and use them as evidence against my character. People change in the real world, and the past is normally forgotten, but on teh interwebs, they stay the same, unless WE can help it. Also, it's great to see

Re:

[X-rated Ouroboros]

Never forget your past.

I've made a habit of going over the stuff I put out ten and fifteen years ago and recalling how enthusiastically certain I was about everything I wrote. And how amusingly simple and corrupted everything I "knew" at that time has turned out to be. I expect in another ten or fifteen years I'll look back at my 2007 self, shake my head, and think the exact same thing. Or maybe not, because, I hope, I can learn from my mistakes. People can change on teh intartubes, too, unless the

Re:

[DarkOx]

I doubt I'm unique and there are probably scores of people doing the same thing.

I am sure there are lots of people myself included that maintain a few "identities" on the net. But maintaining completely fake blogs, and managing the levels of typo and other errors per identity that is a whole special kind of paranoia.

Unless you are tring to evade some agency, I doubt there are people scrapeing the net for posts and comparing them to the degree required to determine different psudonyms are the same person.

Relax, speak your mind, keep two or three of those IDs, discard the rest stop was

Re:

[wall0159]

It seems to me that 'credibility' isn't so much about trying to connect an online avatar with a real world person, but rather be able to establish whether or not that online avatar is trustworthy. If you have a consistent online presence, then it doesn't matter that it can't be reconciled with your real-world life - because as long as others are able to obtain info about your avatar, then it's fine.

Of course, that's the difficulty, because if you were determined to swindle people, then it might be possible

Re:

[gronofer]

So in it's place, I created a whole shitload of false identities that I post under, one of them about 10 years old now. Mainly on forums and newsgroups for work purposes, etc. If you searched for this particular identity, you would probably fine hundreds of posts (including many on slashdot) some of them truthful, some of them fake, with various opinions of topics.

A waste of time, if you ask me. Anonymous opinions (or any opinions if you are not a celebrity in real life) are cheap, generally ignored, do n

Sucks to be you

[SuperKendall]

Some of us have posted under nothing but our real names for years, knowing that every post was only adding to our online metaprofile, and if the bulk of what we posted was helpful or useful or at least sensible, we were building up years of easily accessed credentials to help people understand they could rely on information from us a little more than Joe Q. Internet.

Why you would choose to be anon in a world where reputation is growing in importantance every day has always been beyond me.

Re:

[WilliamSChips]

He doesn't work for any organization that has initials.

RapLeaf

[AaronBrethorst]

What about RapLeaf [rapleaf.com]? Although it's centered around ratings for conducting transations, I have to believe their system would be pretty effective across a broad spectrum of reputation and ratings needs. Plus, they offer a set of APIs, which is always handy.

Advogato

[MSG]

Well, there's Advogato's Trust Metric [advogato.org] system.

I've been of the opinion for a while that a similar system could be devised using PGP or S/MIME certificates to combine identity verification with "web of trust" reputation evaluation. Under such a model, every user would import the public certificate of authorities that they trust. For example, consider a consumer review web site, where I decide to trust the site's admin. The admin trusts its editorial staff, and their certificates are signed by the admin. Any of the editorial staff may trust one of the site's frequent contributors, based on the quality of their work. That editor may sign the contributor's certificate. Now, my level of trust for that contributor can be established as a function of the proximity of that user to the admin in whom I placed trust. This differs from Advogato's system in that the "Master" certificates are simply those whom I've decided to trust.

The same thing can be applied to social networking sites, as well. I can trust my friends by accepting their certificates, and gain insight into social relationships by examining the signatures in their keys.

Re:

[DaleGlass]

The problem with PGP is that it's too complicated to understand for normal people. Now, I love it, but even the Windows GUI versions of it are hard to explain, because the concepts of webs of trust, fingerprints, signatures, and security are alien to most people.

I implemented something web of trust-like in Second Life, but without signatures, and without a root. Instead the server has a big table of user1/user1/rating rows. You are the root, to find a score the server builds your tree and adds up all the pa

Re:

[MSG]

...the concepts of webs of trust, fingerprints, signatures, and security are alien to most people.

Well, the trust relationships are the entire foundation of a reputation system, so we're going to have to engage in education on that front. However, just because it's complicated to use PGP (or S/MIME, more likely) directly doesn't mean that software can't use it as the underlying mechanism for a reputation system.

I implemented something web of trust-like in Second Life, but without signatures,

Second Life val

Re:

[DaleGlass]

Second Life validates users' identities, right? In that case you wouldn't need signatures. I envision a decentralized system, though, and that's going to need
cryptography.

Yep. Also when user-owned scripts contact my HTTP server, the SL server adds headers with the name and user ID of the script's owner. That definitely helps a lot making it friendlier. I was thinking that you were thinking of something implemented on one website, in which case you can also avoid the crypto.

Does a "web of trust" ever have a

Re:Advogato, not quite

[localman]

From the Advogato FAQ: "All the trust metric guarantees is that they really are who they say they are."

It's not a reputation system per se, but just a more reliable identification system. That is a prerequisite for having a reputation system, of course, but it's not the whole thing.

The next step would be some way to figure if the person identified is known to be beneficial or harmful to the community. That is a much, much harder problem, not the least of which is because we probably don't all agree on whi

Reward and punishment

[iminplaya]

Online, and in real life, we reward bullying and punish honesty. How do you make a reputation in that environment?

Reputation vs. identity

[btempleton]

The article claims to be about reputation but mostly talks about the various "identity" efforts out there. Yes, a reputation is associated with an identity, but most of the identity systems being promoted focus on real identity rather than pseudonyms which you can choose to associate with yourself or not.

There is a paradox to those systems -- the easier they are to use, the more they will get used -- and demanded. We'll go from a web where most web sites can be used casually, with no "sign on" (single or otherwise) to a web where far more sites demand you use the single sign on and thus have an account, because it's easy for them to ask.

This paradox is described at http://ideas.4brad.com/paradox-identity-management [4brad.com]

Online reputation? All reputation is hard to do

[Colin Smith]

It's why we have exams, professional organisations, CVs, brands, social networking etc etc etc.

We use reputation all the time and no-one has come up with a single reliable, coherent way of measuring it. You just try to get a decent builder.

 

Why bother?

[DynaSoar]

There is access via the web to some sources of information which exist independently and so are fairly spoof proof. For instance, someone wanting to research me could look for publications with my name on them in PubMed. Sources like that would be valid, web or no.

But as long as the web remains a place where anyone can say anything, rightly or not, anyone who relies on it for supposedly objective information that they can use to measure a person's reputation is coming in dead last in the old "Arguing on the

OpenID Not A Good Candidate For Trusted IDs

[aldheorte]

I've been doing some research on OpenID, which seems to be considered as a possible substrate for building trusted identities (OpenID, as its proponents are quick to point out, only establishes an identity token, not trust). However, I hope that trust does not get built on the OpenID model because the OpenID protocol for identification is very poor from a security standpoint, a blindness to which the OpenID proponents have (I think) because of the vocabulary used.

Basically, the way OpenID works is that you

Re:

[Ghworg]

The advantage of OpenID is not replacing typing in a username and password with a URL. It is not having to sign up with Joe Random's blog site in order to post a one line comment. I don't want to give out my email address to every site I want to contribute to. Sure I could make up fake info for the site, but most of the time even that's too much work to bother with.

OpenID has problems yes, but there are technical solutions for all the ones I know of, including the redirect/proxy one you highlight. I've

Rules of Reputation

[IL-CSIXTY4]

Back in March, I sketched out some "Laws of Reputation" (along the lines of Kim Cameron's "Laws of Identity") on my blog. As I thought about it, I came to the same conclusion. It's *hard*. eBay has a good system, but they only have one context to juggle and it's hardly free. I've seen people posting about the need for context & anonymity, which I touched on, but there's a big difference between saying "it needs to" and "it does". At this point...there hasn't been anything done to implement them, an

They're just blogs.

[Animats]

When I first saw the title, I thought this was about reputations for web sites or online businesses, but no, it's about reputations for, well, bloggers. Where it doesn't really matter all that much. It matters for eBay, but most of the sellers on eBay are businesses. It's been a long time since eBay was individual to individual.

Dating sites have struggled with this. True [true.com] wants to see an image of your driver's license. With the controversy over Myspace, we may see them going that way, at least for parents.

Wikipedia doesn't care much about identity, except as regards vandal blocking. Even admins and ArbComm members are anonymous. All Wikipedia needs is some way to slow down unlimited generation of new identities. I once suggested that one way to do that would be to require some easily available, no-cost, unique, verifiable physical token to register. Like an AOL disk.

One approach to identity verification, which I'd like to see used for domain registration, is simply mailing out a card by postal mail. When you register a domain, a letter should be sent to the address listed for the domain. When you get the letter, you type in the password printed in the letter postcard, and the domain registration completes. That would really improve WHOIS data quality and cut down on scams. The cost of sending out customized mailing pieces is under about US$0.50 each when you have a bulk mailing house do it, so it's quite feasible at current domain prices.

Collaborative filtering

[Colin Smith]

Reputations are relative. They depend on the person making the recommendation. If the person doing so is a numpty, the information the recommendation is based on isn't worth much. Expanding this to everyone would be what, an N^2 problem? Where N is potentially the population of the planet. Thankfully not everyone knows everyone.

It would require a centralised registry though rather than distributed in order to calculate the effect of the relationship to the person making the recommendation. And it would of c

This isn't just an online concern

[jollyreaper]

This is a problem with real life. There's a truism along the lines of "you never really know who another person is, you only know what they've shown you." This goes hand in glove with another truism: "perception is reality." Every one of us creates models of people in our life to better understand what makes them tick, how they operate. Once we feel comfortable enough in our understanding of that person we can say things like "Oh, X is going to love this!" or "Just wait until Y hears about this, I know he's

eBay's system doesn't work

[tekrat]

eBay's system of who you can trust and who you cannot doesn't work. Generally, even theives have a good feedback rating on ebay, and worst of all, the average joe who isn't an eBay powerseller or doesn't have his own ebay "store" is likely to get shafted because eBay only cares about fees, not feedback.

For example, a user with a feedback of 1 can buy something from a seller with a feedback of 450, and then complain to ebay. The user of with the feeback of 1 can have the seller's account suspended despite the vast chasm of difference between the seller's feedback and the buyers, and ebay makes NO distinction over which party has the better reputation.

It all comes down to who files the dispute first. Either that or eBay just assumes the seller is always at fault.

Ebay's dirty little secret is that you can create an account, buy something, and essentially get it for free from the seller because the only way the seller has to resolve the suspended account is to refund the payment. There is NO other recourse.

And "reputation" means nothing to ebay. You could have created the buyer's account a week ago, and take down a seller who has been a good seller on ebay for years.

And when Ebay gets wise to you, create a new email account on yahoo, start again, buy something, register a complaint and get it for free, because the seller has to refund your money.

So; how exactly does eBay's system "work"?

Reputation is easy

[richardtallent]

Identity is really the easiest part of the problem.

But reputation is easy as well. The problem with most proposals is that they are focused on organizational reputation rather than personal reputation.

Reputation, however, is relative and contextual. We don't need Slashdot vouching for us, we need people in our own address book / social network. Then we can vouch for our friends/family in various ways ("this person isn't a spammer", "this person knows a lot about cars", etc.).

But the real power of a personal reputation system is that it is transitive. If I trust that Alice is not a spammer, and Alice trusts that Bob is not a spammer, I can to some degree also trust Bob, and so can my friends, etc. A few degrees of Kevin Bacon there and you've got a real system.

Such a system allows for anonymity as well. I don't need to use my real name if I can generate some other identity and foster trust in some other community. As long as the identity token itself is secure, they don't need to know my name, they just need to know I'm not a troll, I'm insightful (hint hint), etc.

My vision of such a system would use SMTP as the transport mechanism for requesting and relaying trust between parties. Mail agents would handle the requests automatically, like calendar-enabled mail programs do now, and it is a fully-distributed system. Mail clients would also cache trust from their own "friends," like DNS, to better respond to requests.

This degrades well, since the emails can contain manual instructions for those whose mail clients don't have this feature. Or their Internet providers can help with server-based responses, so the mail client doesn't even need to be involved in most cases.

With such a system, spam would mostly be a thing of the past. I can limit incoming email to only people in my Address Book, people in theirs, etc. out to some limit of degree. Chances are, that will quickly encompass everyone likely to want to send me a legitimate email, and bounce away people with no legitimate friends (spammers). The system would self-correct when accounts are compromised or people unwittingly trust spammers, and if a friend of mine is too naive and adds spammers to his list constantly, I can stop trusting his list.

We really do need a ubiquitous identity-trust system, something that uses existing protocols to share trust and integrates with IM, email, online forums, auction sites, etc. But the problem itself isn't that hard.

Re:

[garbletext]

sloppy markup. that text has got an anchor tag around it with no href attribute.

Re:

[GuldKalle]

Hehe, I just thought my eyes were the problem. It's pretty late here, so it wouldn't surprise me.

Re:

[Watson Ladd]

Yes, but someone with a driver's license probably isn't a complete n00b at it.

Re:

[Rick Genter]

So the question then becomes: is the public qualified, as a lowest-common denomintator, to do determine who should have the reputation points?

The public is as qualified to determine who should have reputation points as they are qualified to determine who should hold elective office.

Take that as you will...

Re:

[gronofer]

The reputation system? Maybe something that, yes, we'd like try, but, this being the Internet, there always will be some sites somewhere that put out misinformation. Politically-motivated sites are the first ones that come to mind for me.

It's impossible to have a global reputation across venues, because what gives you a high reputation in one place may be considered despicable somewhere else.