Could a Reputation System Improve Wikipedia? 216
Acidus writes, "There is an excellent article in this month's First Monday about using reputation systems to limit the effects of vandalism on public wikis like Wikipedia. It discusses the benefits and weaknesses of various algorithms to judge how 'reliable' a given piece of text or an edit is. From the article: 'I propose that it would be better to provide Wikipedia users with a visual cue that enables them to see what assertions in an article have, in fact, survived the scrutiny of a large number of people, and what assertions are relatively fresh, and may not be as reliable. This would enable Wikipedia users to take more advantage of the power of the collaborative editing process taking place without forcing that process to change.'"
Easier said than done? (Score:4, Insightful)
I agree that they need to do something, but that is a fantastic challenge. Look at your major encyclopedias, they have a team of several thousand to do fact checking on a paid basis. I'm not saying people wouldn't fact check, but its a great challenge. How would you know that people aren't just saying its legit or not just for fun?
http://religiousfreaks.com/ [religiousfreaks.com]Re:Easier said than done? (Score:5, Insightful)
Re:Easier said than done? (Score:5, Insightful)
The first is that there are a lot of articles and sections of articles in wikipedia that are heavily edited without the facts changing much. This is mostly a good thing, cleaning up grammar, etc. But the if that is used as a basis for how reliable the information is, it could be misleading because the software won't know if the facts have changed, or just their wording
The other problem that I see with this is that it makes it easy for people who "disagree" with facts to make edits to the sections to reduce their rating without just deleting them. It just makes me thing of those people who say "yeah, but evolution is only a theory" to undermine it, I can see them making minor changes to wordings of things to make the facts seem less debatable.
Of course, if someone was doing that, it would be impossible to say if they were doing it because they wanted to supress facts by making them look less reliable, or if there were simply trying to contribute to the quality of an article.
Re: (Score:2)
It's a good point, but I think the edit frequency would be pretty substantialy different in those cases. I'm not really basing this off anything more tha
Re: (Score:3, Insightful)
Re:Easier said than done? (Score:4, Insightful)
So a reputation system is pretty useless because special interest groups can mobilize to skew reputation.
Want to have Intelligent Design show more favorably? Ok, get a bunch of like minded people to raise your reputation.
Heck, we even see it on Slashdot when a conservative or liberal viewpoint gets buried in moderation because people of a certain political belief gang up on the opposition.
A lot of what Wikipedia is dealing with is a direct result of the deep divisions on our society. And the fact that unlike World Book Encyclopedia, apparently *everyone* is allowed into the research offices. In a virtual sense of course...
Re: (Score:2)
seriously, do they
Yes, but worth the effort (Score:4, Insightful)
In the beginning, I'm sure this would just gather data & have little to no impact on the content. But over time, it could well become increasingly effective at improving content quality as its designers started to identify patterns & meaningful correlations in the collected data.
This isn't so different from SPAM filters that need constant training, or PageRank, or eBay feedback scores, or AVN forum posting rules, etc. One needn't restrict the reputation data to any one data species; you could use a composite of community feedback + usage statistics + genetic algorithms etc., and over time tweak the weight any category of data is given to account for its sample size, its expected margin of error, and its track record in terms of predictive power.
Sure, it's a time consuming undertaking & it'll take patience before we see results, but I don't see the real difficulty being in rigging up the system; I think the real difficulty will be in defining exactly what constitutes a quality article.
Now, take a minute to share a utopian dream with me: Imagine the day when registered Wikipedia users with good reputations will be able to make edits from a Tor connection.
Re: (Score:2)
If you have a change to my document. We need to have a way of saying 'i agree' and 'i defer to your judgment', etc.
I can give you the answer without even RTFA (Score:5, Insightful)
It's typical nerd hubris to believe that you can solve social problems through technological means.
It's been proven time and time again that you can't.
Re: (Score:2, Insightful)
Re:I can give you the answer without even RTFA (Score:5, Insightful)
Re: (Score:3, Insightful)
Re:I can give you the answer without even RTFA (Score:4, Funny)
Re: (Score:2, Interesting)
Re:I can give you the answer without even RTFA (Score:5, Insightful)
Slashdot's karma system is far from perfect, but at the end of the day it works. Can you game it? I don't really think so, at least not without a LOT of effort, which generally means contributing a lot of good content/ratings so that you can sneak in a very small amount of biased content or ratings.
Whether "ungameable" is possible or not I don't know, but I am quite sure that wikipedia's system could be improved upon massively.
Re: (Score:3, Insightful)
Re: (Score:3, Insightful)
If people using it today value neutrality, how is that going to change when a karma system is put in place?
Re:I can give you the answer without even RTFA (Score:4, Funny)
The wright brothers weren't trying to overcome social issues, they were overcoming a technical issue.
You can not solve social issues with technology. Social issue aren't always rational.
I have gamed slashdot karma many times.
In days of yor, I could intentional get a 50 karma in a week, and a -49 karma the next.
I can do it today.
Don't believe me? mod me up.
Re: (Score:2)
I say "puppies are to dogs as kittens are to cats".
You say "flawed analogy! Kittens are not canines!"
And I'm not convinced you really "gamed" slashdot in any significant way. Try to get a crappy (as seen by the community) comment prominently displayed for a long time, in order to push some agenda. Without balancing it out by posting a large amount of good (as seen by the community) content.
Re: (Score:2, Insightful)
I disagree. I set my preferences so anons don't have a -1, and people with karma bonuses aren't shown with +1.
If you have something good to say, it should stand on its own merit - and that applies to Wikipedia. We should accept information there because it is verifiable, and not because we trust the random person who wrote it.
Re: (Score:2)
Re: (Score:2, Insightful)
Re: (Score:2)
It's really the whole ratings system, of which karma is one aspect. Note also that people with good karma get to moderate more often (at least that is my understanding), so a spammer can't easily just create a bunch of accounts solely for the purpose of rating his own posts up. And meta moderation is also important.
The point is, a spammer would have his post viewed by very few people if he doesn't have any karma, and
Re: (Score:2)
Every dupe someone reposts stale comments.
Almost ever story someone whores and posts the article.
It takes a little effort, but you can boost your karma.
Recently, I found out that they may ban you from AC posts if you get enough down mods on past AC posts. I was posting asinine stuff AC, but it still hurt my account and I did not know that was an option...
Re: (Score:3, Insightful)
Re: (Score:3, Insightful)
There is a world of difference.
Re: (Score:3, Insightful)
That is not what nerds are trying, it is what society is doing: Trying to solve social problems through software for wetware (laws).
In the case of computer based communities, that laws are codified in programming languages, whereas in RL it is codified in legalese.
> It's been proven time and time again that you can't.
Yes... like flying.
I admit, my first statement seem to be more an argument against th
technology is the answer (Score:2, Insightful)
Re: (Score:2)
Maybe, but even if you managed to get it to provide meaningful insight into the reliability of the content, could you do it without turning the page into a rainbow striped eyesore?
Re: (Score:2)
surely these people who can beat the system are smart enough to make a useful contribution?
if they put all the effort into getting a good reputation, then won't they want to keep it?
rep based systems can work quite well - ebay for instance (although I can tell that a significant number of ebaybuyers don't really thi
Insularity is the key (Score:2)
What you CAN do is identify those thing for which there is no natur
The Slashdot moderation system proves.... (Score:2, Insightful)
Re: (Score:2)
Re: (Score:2)
Wikipedia's current system is "edit till the arguing stops". Ultimately, the more people sharing an opinion, the more the articles will bias that way. A good reputation system would not change this, just make it more efficient.
Re: (Score:2, Insightful)
But if what you say is true, then a karma system would only make this worse. Who would get the most karma? Those people who are spending all their time editing on Wikipedia, of course.
Re: (Score:2)
Re:The Slashdot moderation system proves.... (Score:5, Insightful)
Possitive modding is a little more shakey with "informative/interesting/insightful" all meaning pretty much the same thing in most people's mind, but that's not too much of a problem.
Group think can cause issues, but in reality there is such a wide range of modders it is often avoided (you can see some pro-MS or anti-Apple comments come through)... although the system isn't perfect I guess group think at least only makes content that most would want to see if they come here.
It is also interesting to note that most people do care about karma and do like to get modded +5, maybe the wiki system would work in a similar way - where people will care.
Re: (Score:2, Insightful)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Interesting)
Re: (Score:3, Informative)
Re: (Score:2)
Re: (Score:2)
So what you're saying is that because a bad, stupid, wrong moderation system doesn't work on slashdot, that some other moderation system wouldn't work on wikipedia?
I don't think you really completed that thought before you wrote your comment.
Re: (Score:2, Interesting)
Re: (Score:2)
Also I think it is odd to say that "because there is a case of something doesn't work, that will never work".
rep farming (Score:3, Insightful)
what we need are national ids and biometric logins.
i kid... i kid...
Re: (Score:2)
Bah. I'm already wasting several hours a week farming Cenarion Circle rep for my new axe, and now I've gotta farm wikipedia rep?
Re: (Score:2)
It's a form of 'payment'. Need to 'do good' before can 'do bad'. How wouldn't that help if vandals need to make (e.g.) 10 useful contributions for each vandalization instead of just vandalizing like it's possible now?
With the example above, for every 10 vandals, there only needs to be one more vandal still building up his reputation to undo the vandalism of all the others...
And that's without the people who don't vandalize.
Re: (Score:3)
* Yes, there's the karma-like concept, but that could easily be spammed by the opposition, just like it is on
Re: (Score:2)
Otherwise, I've found that most moderations are fine. Maybe I luck out while M2'ing?
solution in search of a problem (Score:4, Insightful)
Re: (Score:3)
I am progressively more and more disturbed by the wikipedophile focus on finding a solution to "vandalism". Liken it if you will to an right wing politicans campaign to rid the world of "terrorism".
I would concur that mindless destruction of someone's work is annoying and should be dealt with, however, I think it is important to understand that not all acts of destruction are in fact mindless - some are legitimate protests. Much as wikipedia likes to harp on abo
How about one for /.? (Score:4, Funny)
I mean, we could all moderate/evaluate the slashdot editors on their choice of stories and keep stats, like onna baseball card.
CmdrTaco
Dupes: 23
Veiled ads as news: 18
Old news: 17
Allowed Bad Grammar: 2,980
Allowed Bad Spelling: 9,874,376
Re: (Score:2)
Yes (Score:3, Funny)
YES - It works on
Re: (Score:3, Funny)
Re: (Score:2)
YES - It works on /.
I don't fully agree, I've never voluntarily trolled as much as since I have had a maxed out karma.
Re:Yes (Score:5, Insightful)
Re: (Score:2)
Re: (Score:2)
Is that like a petting zoo, but with personal ads?
Rep doesn't lead to reliability (Score:2)
somethign simular to yahoo answers (Score:2)
I've only edited a wiki once, and that was info on my home country. I have however been addicted to yahoo answers. thats what i doo all day ( why I dont know) one cool feature is you have to gain a certain rank before you can be allowed to either thumbs up or thumbs down and answer or question. I guess it's kind of a prove your worth sort of deal.
What about... (Score:5, Interesting)
Re: (Score:3, Funny)
Re: (Score:2)
when reading a controversial article, or in general any article one wants to be reasonably sure about, one could use this view and take into account mostly only the 'black/oldest' text. As much as it's easy to figure out obvious vandalism, it's not as easy, especially if one doesn't have domain knowledge, to identify subtle changes.
This would also be helpful for articles on my watchlist when I don't look at them for a while, since edits
Re: (Score:2)
Not a complete solution (Score:5, Insightful)
On the other hand, the pages regarding the fight between Hamas and IDF are as much a battleground as is the area around the Israeli/Lebanese border. I have been involved in Wikipedia for years and have just seen things deteriorate around these types of flame-wars. Wikipedia's leadership is not dealing with it well. Imagine Slashdot setting up a wiki where we had to determine which was better - Debian or Gentoo (or Ubuntu etc.), BSD or Linux, vi or emacs etc.
We are technical people, and there's the old thing about when you have a hammer everything looks like a nail. But I don't think a technical solution will help much in regards to this. I'm not even sure you really can have a neutral view about wars in the Middle East. And even if you could, Wikipedia's "cabal" is nowhere near able to deal with it, and I doubt they ever will be. Personally, I think most of the people in high positions at Wikipedia are jerks, all the flamewars and such seem to have driven most of the nice people off.
Things like Wikipediareview.com convince me that what will ultimately happen is alternatives to Wikipedia will pop up. Wikipedia is a new phenomenom, and it makes sense everyone edits on the same wiki, but why should that be? Why should pro-Hamas and pro-Israel people edit and battle on the same wiki? It makes little sense, and I'm sure in time, just as IRC went from one network to EFnet and Anet, and then split even more, I'm sure we'll see splits with Wikipedia. In the old days, the Encyclopaedia Britannica had one view of history and the Great Soviet Encyclopedia had another, why should the future be any different?
Re:Not a complete solution (Score:5, Funny)
Actually, every article on quantum mechanics exists in a state between vandalized and not vandalized. By viewing it you colapse the waveform and change it's value. Now, there is a good probability that it will turn out unvandalized, but as you have stated it occasionaly collapses into a vandalized article. After you leave the page, Wikipedia runs complex calculations in it's improbability engine and sends the article back into a quantum state.
P.S. This is presented as per my understanding of quantum mechanics which I learned entirely from Wikipedia. It may be wrong however as my viewing might have caused it to appear in a vandalized state.
P.P.S
Debian, Linux, emacs. That wasn't so hard. (Anyone who disagrees is a terrorist.)
Re: (Score:2)
Because splitting from Wikipedia to create the Palestinapedia would be an admission of defeat.
Complete solution not needed (Score:2)
But I think that discussing ways to improve Wikipedia is very valuable; only by proposing ideas and trying them out can things get better.
This is not a user reputation scheme; it simply colors text based on how many edi
Re: (Score:2, Interesting)
Because the harder you try, the closer you come to success.
In contrast to most articles on the Israeli-Palestinian situation where the writer is trying to advance a particular viewpoint, I've actually found the Wikipedia articles to be quite good. You actually have a situation where people on both sides of the issue are coming together to try to
Re: (Score:2)
This is bullshit... (Score:2)
Re: (Score:3, Interesting)
Sometimes it's obvious when a factual error has been made. Say, for example, somebody changes the article on Monarch butterflies to claim that they feed on th
Reputation a General Term (Score:2)
Consider Einstein's quote, "Marriage is nothing more than an attempt to make something lasting out of an incident." Obviously Einstein was a less than stellar social psychologist.
Vanadlism is not Wikipedia's main problem (Score:5, Insightful)
Larry Sanger has acutely commented on Wikipedia's anti-elitism and the way they have run experts off the system. Experts don't have the time or energy to debate fundamental points of well-understood scholarships with game-playing trolls. Further, even when they aren't teenagers, Wikipedia has become the home of everyone who wants history and scholarship to read the way they like it rather than representing some academic consensus. As a result we have politicians trying to rewrite their personal biographies (or those of their opponents), partisans on each side of the world's conflicts burnishing their allies and undermining their opponents (Israel/Palestine, Turkey/Armenia, US/everyone else), and devotees of everything from Microsoft Vista to Nintendo to PETA skillfully expunging objective truth from their deifications of the chosen object of worship.
So doling out karma to 100,000 teenage idiots is not going to solve Wikipedia's problem. In order to save Wikipedia, we need to destroy it -- it needs to be edited by more experts and fewer "normal people".
Re: (Score:2)
Re: (Score:2, Insightful)
Right now, Wikipedia strives to attribute all statements in its articles to a reliable source. This is something an idiot can do, once they are shown how.
Assuming that debates would somehow be fewer or settled faster and with universal agreement if there were more 'experts' shows no understand of human nature what-so-ever. The best WP can hope
Re: (Score:2)
One Idea from the artical that was worth while (Score:2, Informative)
1) People looking for reliable information would know that these parts of an artical have not been exposed to long term scrutney, and therefore may be inacurate.
2) The new, and therefore unverified parts would be mo
Wikipedia hubris (Score:2)
The topics on Wiki that are least useful are the obscure, non-controvers
Re: (Score:2)
Sounds fair, but there's that tricky part of obtaining the cooperation of everyone who cites a Wikipedia article as an authoritative source to go along. Judging from posts on Slashdot, for example, I'd suggest the "It's on the internet, therefore it's true." approach is, if not alive and well, valid enough to garner enough nods of
recognition for good edits (Score:2)
The link's web server does not seem to be responding, so I have no idea what they are proposing. I think a reputation system somewhat similar to slashdot's might be very useful if correctly implemented - users could establish a reputation for themselves over time, and edits by users with a bad reputation (or no reputation) might receive more scrutiny.
This might help with an unrelated problem: giving recognition to people who make good edits. I suspect a lot of people wouldn't post as often (or word thei
Here's my suggestion (Score:2)
I dunno... (Score:5, Informative)
views (Score:2)
Browsing a site might consist of choosing which views you want to see by default, and accessing other versions/site
Visual Cues (Score:2)
And I know just the way to implement this! New text starts with font colour #FFFFFF. At every edit, if the text survives, the colour value is decremented by 1. When it hits #000000 it is forever barred from being edited ever again.
It's useful:
Re: (Score:2)
And as far as the wise man goes, he is trivially proven a fool. Every page has the same color pattern just after its creation! Besides, certa
So now we rate people too... Bad move. (Score:5, Insightful)
The best parts of Wikipedia is a fast and easy way to edit information, no hassles, no extra effort required. You get out what you put in and that's it. You want to put in the work to be a vandal you're a vandal, but in the end you already know what you're doing. Type in a good sentance but someone replaces it with a better paragraph that's fine.
But instead of working on the core of the experience now we are going to spend time rating each others' facts, rating each other. Basically just killing time. The simple fact is we don't need it, this system is in place in a lot of other places and in effect it basically weeds out the bad apples at the inconvience of all the good users. "You'll have to do 5 discussion posts before you can edit an article" "you have to edit three more articles before you can add an article". This stuff doesn't help or appeal to anyone but "karma whore" types.
If I write a well written page about the new player on the Red Soxes, I should be able to go in to a page that links to it create that page, set up my links and go. I should be able to do this on the first day as well as the fifth year with the same ease. Adding in safe blocks and guards will only hurt wikipedia's overall goals, not help the ideas it promotes. The best thing to do is start handing out serious penalties for vandalism or obvious weasel words.
This doesn't even get into the idea of being able to do fast edits with out logging in, something that's helpful at times.
Web of credibility (Score:3, Interesting)
Having not yet RTFA, I'd just like to say that I agree, wholeheartedly, with the general notion... and I look forward to the day when our credibilities are incoporated into our digital signatures (that I hope we're also all using someday). - Joe
Credibility isn't like Trust (Score:2)
I know people that i think are absolutely credible sources about technology subjects, but who I wouldn't consider credible in discussing say politics (although i'm sure they feel they were qualified).
I see your point about having to assess how likely it was that someone would spout off something inaccurate, but i feel that most people do that at some point or another (see Slashdot)
Re: (Score:2)
Well then, if they often expounded about technology topics *and* political topics, then other people would be giving them a blend of high and low credibility scores... yielding a score of "so so". However, if they knew to keep their trap shut when the topic turned to politics, then they wouldn't get as man
PageRank, TrustRank and DMOZ (Score:2)
Sounds good in theory but in practice: (Score:2)
Xbox live (Score:2)
No (Score:2)
Short answer : No
Long answer : No..
I know of some wikipedia user who has a good reputation but vandalizes afrocentrism-related articles to push her personal agenda, so reputation is nothing actually. It's as if you believed in whatever scientists claim due to their reputation, as we know very well that even some reputed scientists make up some crap for non-scientifical publications from time to time for various reasons.
Re: (Score:2)
How about the minority of one? Given your example, only ONE is a TRUE expert on Barack Obama, that is Obama himself. Everyone else is less
Re: (Score:2)
The internet has proven that talking about something a lot, and 'correcting' people does not indicate expertise.