Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

How Can You Measure a Wiki's Worth?

timothy posted more than 6 years ago | from the weigh-the-electrons dept.

Businesses 78

moldar writes "I have been involved in a major project to migrate documentation from multiple sources to a wiki (Media Wiki, if you must know). Now, the PHBs are all asking questions about how organized the data is. I've already googled for various wiki metrics ideas, but mostly they focus on page counts, average page sizes, rates of edits, etc. Can anybody suggest better ways of measuring the quality of a wiki? Things like uncategorized pages, articles that are too small, etc? Any help or fresh ideas would be appreciated."

Sorry! There are no comments related to the filter you selected.

Turn it off. (5, Insightful)

zonky (1153039) | more than 6 years ago | (#24608837)

See how many phone calls/emails you get complaining.

Re:Turn it off. (1)

Fluffeh (1273756) | more than 6 years ago | (#24609441)

Or...

See how many edits there are on the "Who shot first, Han or Greedo?" entry.

Re:Turn it off. (1)

sm62704 (957197) | more than 6 years ago | (#24619355)

He should do some research [uncyclopedia.org]

Close! His problem is backwards. (1, Insightful)

Anonymous Coward | more than 6 years ago | (#24609597)

The problem is that he's trying to measure the wrong thing. He's trying to get metrics from the wiki itself. He should be getting them from the users.

So the simplest solution is to do a user survey about what's good and what needs improvement.

Here's some advice (4, Funny)

Red Flayer (890720) | more than 6 years ago | (#24608851)

I really don't know the best way to go about measuring a wiki, but I'd suggest not taunting it. It might get pissed and rip your arm off.

Re:Here's some advice (5, Funny)

thegrassyknowl (762218) | more than 6 years ago | (#24608905)

I suggest a new strategy, let the Wiki win!

Re:Here's some advice (2, Funny)

tomzyk (158497) | more than 6 years ago | (#24614015)

I thought it was Happy Fun Ball(TM) that you were not supposed to taunt?

Re:Here's some advice (1)

Red Flayer (890720) | more than 6 years ago | (#24615507)

My mistake. Of course it;s the Happy Fun ball we're not supposed to taunt; the Wookie, we're not supposed to Tauntaun.

Sell it? (1)

corsec67 (627446) | more than 6 years ago | (#24608873)

It would be worth whatever you can get someone to buy it for.

Nice editing on the part of whomever took the firehose article. Changing the title changed how anybody is going to read the article.

Getting a single metric for the quality of a wiki is going to be really hard, since the number would just about be made up.

If you want to say what percentage of pages are uncategorized, below a certain size, or orphaned, then do that.

Worth vs price; Value != value (1)

EmbeddedJanitor (597831) | more than 6 years ago | (#24610339)

How much would you pay for a gallon jug of air? You might value being able to breath but you'd probably be very reluctant to pay for air until your head wasunderwater?

People are very resistant to pay for something that they feel should be free. The same people that would pay a hundred dollars or more for MS Office would feel screwed if they paid $10 for OpenOffice.

Value to the economy is very different to the value marketeers attach to something. Wikipedia, Linux and many other free products are very valuable to those that use them, but taht does nopt mean people would willingly pay for them. Marketeers would see that these products have no value because you cannot make money out of them.

You measure like a pile of shit (-1, Troll)

Anonymous Coward | more than 6 years ago | (#24608917)

No matter how much you pile it on, it's still just a pile of shit.

In soviet russia... (0, Offtopic)

thegrassyknowl (762218) | more than 6 years ago | (#24608933)

your wiki measures you!

Metrics (4, Insightful)

Enderandrew (866215) | more than 6 years ago | (#24608945)

PHBs will always insist on metrics for things that can't properly be quantified.

What I would do is suggest that he task a team from another department to run a survey on satisfaction amongst wiki users. Has the wiki helped them? How often do they use it? What would they like to see changed.

This should distract the PHB for a while, while diverting your responsibility to come up with a metric.

Re:Metrics (1)

MinutiaeMan (681498) | more than 6 years ago | (#24609827)

That's a great idea for a delaying tactic, but it could easily backfire on you if the survey indicates that people aren't using the wiki correctly or at all...

Re:Metrics (3, Insightful)

Degrees (220395) | more than 6 years ago | (#24610377)

But if they are not using it, maybe it's a time-sink that needs to be dropped.

Re:Metrics (1)

TERdON (862570) | more than 6 years ago | (#24613161)

Of course, in the design of the survey, you should make sure that the result "people are not using it but they would if they knew about it" and "people are not using it because they tried and it sucked" aren't compounded into a single result. They are very different, but given the wrong set of questions, will be very hard to differ from each other!

Re:Metrics (3, Interesting)

corbettw (214229) | more than 6 years ago | (#24610051)

That's actually a pretty useful suggestion. Think about it, the real question the PHB is asking is, why should we continue to fund this thing? Are we getting our money's worth? The best way to answer that is to see how much, if at all, people are using it, and what they think of it. Your survey may just be the metric the bosses are looking for.

Re:Metrics (1)

Enderandrew (866215) | more than 6 years ago | (#24611051)

It is likely the closest "metric" that will answer the question.

Usage pattern statistics (4, Insightful)

MarkusQ (450076) | more than 6 years ago | (#24608949)

It depends a lot on what your wiki is for, but usage pattern statistics might be a good metric. For example, repeat visitors are people who are getting some value out of the wiki. Trampoline pages (pages people hit and then bounce elsewhere from a link in the page) might be another good thing to track. As well as abort-and-retry pages, where instead of following a link the user goes to the search box and tries something else.

You get the idea.

If the wiki is valuable to the people who are using it, you should be able to tell this from the logs--even if no one ever mentioned wikipedia in public, they could tell how much we care by noting that we keep coming back.

--MarkusQ

Re:Usage pattern statistics (0)

Anonymous Coward | more than 6 years ago | (#24611291)

...could tell how much we care by noting that we keep coming back.

--MarkusQ

By that metric, things like local TV news would rate extremely useful. I think you discount the tendency for the masses to continually use what they find easy.

 

Don't underestimate "easy" (1)

MarkusQ (450076) | more than 6 years ago | (#24616065)

I think you discount the tendency for the masses to continually use what they find easy.

That's a valid part of the metric. Easy is good. When you're figuring benefits / costs, "easy" reduces the denominator and makes the total score much higher.

When you just compare the accuracy of weather forecasting methods, "looking out the window" doesn't rate very high. But it's generally so easy that in practice it's the technique of first recourse and you'd be a fool not to use it. The great thing about "easy" is that, if you find you need more detail, more precision, or whatever, you haven't used up the alloted resources. And if you don't you are time / money ahead.

Anyone who could go to their bosses and say "our website is so easy to use that people just come back time and time again, and here are the statistics to prove it" is in great shape.

--MarkusQ

Re:Usage pattern statistics (1)

aero6dof (415422) | more than 6 years ago | (#24616643)

It depends a lot on what your wiki is for, but usage pattern statistics might be a good metric. For example, repeat visitors are people who are getting some value out of the wiki. Trampoline pages (pages people hit and then bounce elsewhere from a link in the page) might be another good thing to track. As well as abort-and-retry pages, where instead of following a link the user goes to the search box and tries something else.

One can transform these types stats into quantified estimates of savings vs non-wiki information references. e.g. what is the cost of 1000 5-sec wiki searches resulting in links to documents vs 1000 5 minute searches for documents on an internal network.

What is the time saved to edit a wiki vs time to update & distribute a published manual? (or if there is no manual, what is the time required for employees to re-develop the information noted in the wiki - this one is more difficult to estimate... but still relevant)

What is the time lag in noting a exception to a procedure in a wiki vs the time to disseminate the info in other ways?

You get the idea. The metrics aren't perfect, but they capture a general scale of savings in using the wiki. If the project isn't hitting breakeven on a regular basis yet, make sure to note the growth rate of some of the stats or maybe turn that into a training project to increase use to the point where it is giving regular self-sustaining benefits...

Impossible! (2, Interesting)

Gewalt (1200451) | more than 6 years ago | (#24608965)

You cannot "measure" anything until you have defined measurement criteria. So until you define the "worths" of a wiki, you cannot measure them.

Re:Impossible! (1)

SEWilco (27983) | more than 6 years ago | (#24609325)

Oh, help him out. The answers are 3, 12, 4.5, 42, 8, none, 7, 11, sometimes, and Delta Gamma Kappa.

Re:Impossible! (3, Funny)

corbettw (214229) | more than 6 years ago | (#24610067)

I don't know what the other questions are, but I guarantee the last one is "Where's the quickest place to get laid on a Friday night?"

Re:Impossible! (1)

SEWilco (27983) | more than 6 years ago | (#24611211)

You have to state the question in the form of a Wiki measurement. You lose the amount which you bid.

Re:Impossible! (1)

Gewalt (1200451) | more than 6 years ago | (#24616033)

Since this is /. the first question must be:
What is one plus one, for sufficiently large values of 1?

Not impossible! (0)

Anonymous Coward | more than 6 years ago | (#24609889)

Nonsense, a wiki is at least 2-3 LoC's, maybe 4 at the max..

How about this... (-1, Troll)

Anonymous Coward | more than 6 years ago | (#24609037)

Go tell your boss to shove six sigma up his ass. Not everything can be measured by a number. And just because it can't get a number assigned to it does not mean it isn't worth doing.

Re:How about this... (1)

Tablizer (95088) | more than 6 years ago | (#24610519)

Go tell your boss to shove six sigma up his ass. Not everything can be measured by a number. And just because it can't get a number assigned to it does not mean it isn't worth doing.

Unless you work for yourself, you have to *learn* to put up with PHB bullshit. That's just life, dude.

The PHB's at our pad wanted to give a group of people web cams so they could watch their faces in conference calls. The users for the most part don't really want them and don't want to mess with them. But if the big-wigs want it, they have to do it.

       

Re:How about this... (1)

xenn (148389) | more than 6 years ago | (#24610671)

The PHB's at our pad wanted to give a group of people web cams so they could watch their faces in conference calls. The users for the most part don't really want them and don't want to mess with them.

I can see why. That sounds kinda perverted. I'm guessing that it's not two way...

If it's not voyueristic, it's at least a bit extreme. Are they trying to fix a problem, or create one?

it must be a strange situation...

Organized? (2, Interesting)

tmarthal (998456) | more than 6 years ago | (#24609061)

The whole point of a wiki is that it doesn't have to be organized. It doesn't have a table of contents, it has a page list -- which are not organized in any sort of temporal or incremental way, but rather alphabetically.

You don't read a wiki start to finish like most PHB's think documentation should be read. Instead, try to make him understand that relevant links (internal link count metric) and search indexing (search count metrics) are what make a wiki organized.

Which is the whole point. You go from an organized list of things describing X and Y in word documents, to a wiki describing X with subpoints X{a,b} and Y with clarifications to Y{c,f,g,h}, etc. Some people don't care about Y{f}, but some people actually do. And since everything describing Y should be in the same wiki, then all of the users can go to their respective pages.

Its more of a culture change than anything. [Now, talking about wiki uptake... that should be something interesting.]

Re:Organized? (1)

Nefarious Wheel (628136) | more than 6 years ago | (#24610497)

The whole point of a wiki is that it doesn't have to be organized

Very good post!

At my company (one of the major international technology SI's, population >40k peeps) we've just moved our entire culture across to a global Wiki (Confluence) with a lot of MediaWiki imports from various wikis (our company's wikis, that is) around the world.

The existing knowledge base portal just didn't quite catch on, was too bureaucratically structured and just never achieved critical mass -- behind the scenes, a lot of individual Wikis had spread up to address the content shortfall (technical people need to store and share a **lot** of detail). Interestingly it's got full backing from our global CEO who somehow managed to read the grass roots movement and it all looks pretty good from this point. Basically he went by what he believed was the right thing to do, not from some pilot study built on guessumetrics. (It's amazing what a boost to morale this approach was, too. Leaders are supposed to Do The Right Thing.)

It's been interesting to see how the people involved wanted to echo the Wikipedia culture, but from inside the corporate firewall. So we have full attribution because it's integrated with our global AD, but from that point onward it's "everybody can edit". The reason why this openness was incorporated in the project charter is (a) It works pretty well for Wikipedia, and (b) the freedom to write and amend is going to refine the quality over time because we're a large group of professionals, so the standards will start high and raise higher, and (c) this is actually a peer review process akin to the peer review you would expect from a scholarly journal or academic institution (well yes, we do believe we're that good =). The mere fact that it's easy to use and you don't need to solicit permissions to get something written means takeup has been swift and the kb is growing daily and it's reached the point where it's useful to me in putting together a solution for a bid now. Metrics? Meh -- use disk space trends or some gross metric like that. It's useful if it's useful, and it's useful.

obvious (3, Funny)

Brain Damaged Bogan (1006835) | more than 6 years ago | (#24609145)

by using metrics of course

Re:obvious (1)

dunnius (1298159) | more than 6 years ago | (#24609483)

by using metrics of course

Or you can use imperial units if you want to torture yourself.

mod -1 (0)

Anonymous Coward | more than 6 years ago | (#24610219)

I mod you -1 for american bashing. Why? Because ... uh ... I'm american! And I dislike criticism!

Re:mod -1 (0)

Anonymous Coward | more than 6 years ago | (#24610757)

Merikans didn't invent the imperial system.

Re:obvious (0)

Anonymous Coward | more than 6 years ago | (#24611407)

I'm American you insensitive clod!

Re:obvious (1)

HTH NE1 (675604) | more than 6 years ago | (#24616319)

by using metrics of course

But imperialistics have been working so well.

"`What's the Wiki worth?' It exists and it's useful. That's enough! Off with his head!"

I've done this one a hundred times (5, Insightful)

CorporateSuit (1319461) | more than 6 years ago | (#24609269)

Many Moons [tripod.com] (I apologize for the hosting company in advance, the site is not mine) is a childrens' story everyone in IT, Sales, and Marketing should familiarize themselves with.

The truth of the matter is: You're asking the wrong people. You should be asking the suits what they would imagine in a report and what kinds of numbers they're looking for. If they're general and obtuse, tell them straight up they'll get a general report, or just make up numbers to keep them happy. Create a widget they can access from their desk that shows them numbers generated from the database that hardly matter. Does it matter what you deliver if they don't know what they want?

They're trying to imagine some great power-point presentation you'll be showing with pie charts and red/green/yellow/blue graphs popping out and wowing them? Make it so. They're trying to imagine a spreadsheet? Easier and more concise. They just want a status report at the end of the day on what percentage of the documentation has been migrated? You can probably get away with a few page count written on a napkin.

Remember the Alice's conversation with the Cheshire cat:
"Would you tell me, please, which way I ought to go from here?"
"That depends a good deal on where you want to get to," said the Cat.
"I don't much care where -" said Alice.
"Then it doesn't matter which way you go," said the Cat.

Re:I've done this one a hundred times (0)

Anonymous Coward | more than 6 years ago | (#24610551)

A teaser from the story!

"But the moon is out of the question. It is 35,000 miles away and it is bigger than the room the princess lies in. Furthermore, it is made of molten copper. I cannot get the moon for you. Blue poodles, yes; the moon, no."

Re:I've done this one a hundred times (1)

Architect_sasyr (938685) | more than 6 years ago | (#24611271)

Create a widget they can access from their desk that shows them numbers generated from the database that hardly matter.

I have an RRD graph titled "Processed Internet Packets" that just graphs /dev/random every 5 minutes. The Manager I built it for contentedly uses the graph for all sorts of things, and I can get back to my work.

Re:I've done this one a hundred times (1)

ais523 (1172701) | more than 6 years ago | (#24626141)

Use /dev/urandom instead, save entropy! Some day it's possible your system will be low on entropy for whatever reason and /dev/random will block.

Re:I've done this one a hundred times (1)

Architect_sasyr (938685) | more than 6 years ago | (#24653469)

/dev/random will block.

Only if you use a Linux system. FreeBSD /dev/random doesn't block.

Simple (2, Funny)

Dan East (318230) | more than 6 years ago | (#24609363)

eBay

good luck, you'll need it (0)

Anonymous Coward | more than 6 years ago | (#24609373)

The quality of the wiki is not the same as how organized it is, and organizational quality is human-measurable, but in the absence of good, labelled training data - or probably even with it - that's not computable. But you could apply metrics from graphing theory (pages as nodes, links as edges) and easily fool a PHB with jibberish.

ask the users to survey it ... (3, Interesting)

pbhj (607776) | more than 6 years ago | (#24609399)

Depending how much of a BOFH you want to be you could ask users to rate the wiki, either a rating for any page they use (hard to enforce): MS have something like this as too do IBM, IIRC, "rate this article on ...".

Alternatively you could do an exit survey, when a user leaves teh site you pop-up a survey to ask about their experience.

Perhaps you could simplify adding templates and get users to add class markings (I think that's what they call them) for things like "this is a lame article", "this article needs editing for brevity", etc. and markings of "this is a B grade article", etc., then run a test of how many of what grades are given and how many templates are used.

Note that templates describing failings will be more prevalent as the wiki gets more use (per user and overall user count) a PHB might not twig this and will think quality is going down.

You could also measure uptake versus the previous sources.

Re:ask the users to survey it ... (1)

Quantus347 (1220456) | more than 6 years ago | (#24615887)

It may just be me, but if I were one of your users, constant pop-ups asking me to rate the site would just result in low ratings of the site.

Re:ask the users to survey it ... (0)

Anonymous Coward | more than 6 years ago | (#24617883)

Alternatively you could do an exit survey, when a user leaves teh site you pop-up a survey to ask about their experience.

"Too many exit surveys."

Make it up (1)

4D6963 (933028) | more than 6 years ago | (#24609417)

1. Pretend to invent your own metric by lack of pre-existing satisfactory metric
2. Describe it using esoteric terms you know your audience won't grasp/buzzwords
3. Make up phony results
4. ???
5. Ask for a raise, errr, I mean, profit!

Couple ideas (1)

TejWC (758299) | more than 6 years ago | (#24609589)

I was on the same boat not that long ago; here are a few ideas:

1. Wiki documentation to non-wiki documentation ratio. Compare how much documentation there is between the wiki itself vs. other project documentation like your word documents and pdfs (assuming they are not just copy/paste stuff from the wiki). A simple word count as measure will probably please your PHBs.
2. Assuming that this is a programming project that you are in, you can also compare the % growth of the wiki vs. the % growth of the codebase. It somewhat implies how much people are contributing to the wiki vs. doing the actual project.
3. Assuming that people properly tag pages; you can try to see the average time it takes for a tagged article to become untagged. I don't know exactly how to do that in mediawiki but I am sure you can write a complicated SQL script in order to get such information (assuming you have direct access to the database).
4. Other people mentioned activity as a good measure. I would also include average number of edits per page (people are actually reading and editing pages instead of dumping things in) and also average time between edits (people are constantly updating pages).

Similar Situation (4, Informative)

SirLurksAlot (1169039) | more than 6 years ago | (#24609593)

I'm actually in a situation which is very similar to the OP's. The company I work for has a number of training/procedural documents which currently exist in .mht format (of all things), and we're in the process of migrating those documents into the wiki which comes built into MOSS (the latest version of SharePoint). I won't go into detail about how horribly crippled the MOSS wiki is, except to say it is. Anyway, I was asked to write a "Wiki Best Practices" document and I tried to include a few ideas on what makes a successful wiki because I knew the business users were pretty clueless about them (not that I claim to be an expert here). These were some of the points I tried to address:

  1. First and foremost are the users able to find what they are looking for? If they can't find it in the wiki they'll go somewhere else for it, and the wiki will fall on its face.

  2. Is the information accurate? If you're not meeting these two criteria your wiki is probably worthless, and your users will almost certainly resort to another source of information.

  3. Does your wiki exhibit signs of growth? By this I mean new pages/sections/categories being added when new information becomes available, and edits being performed to improve the quality of existing information. I think one of the biggest hurdles companies will have to overcome for this criteria is the fact that there tends to be a very top-down approach to the distribution of new information (This is definitely the case in my company). In order to allow the wiki to grow there must be enough users with create/edit privileges to keep it up to date. The more users there are who are actively involved in the wiki, the more likely it is to be successful. Of course, if management doesn't want to allow the user populace to be able to edit the wiki the task will probably fall to a select few, and they will inevitably become involved in their other tasks at which point the wiki will fall by the wayside and die a quick death. This is the battle I'm currently fighting, and I'm not entirely sure I'll win.

  4. Ask your users what they think. This one is pretty obvious, but easy to overlook if you're doing things like checking page hit counts and sizes. Put together a survey and ask them if they find the wiki to be easier/more useful than the previous method, if it saves them time (and if so how much), and what they think should be done to improve it. Ask them if the information they find is accurate or not. Finally, ask them what they did/didn't like about the previous method, and whether or not that issue has been addressed by the wiki.

Re:Similar Situation (1)

Conficio (832978) | more than 6 years ago | (#24612967)

Does your wiki exhibit signs of growth? In order to allow the wiki to grow there must be enough users with create/edit privileges to keep it up to date. The more users there are who are actively involved in the wiki, the more likely it is to be successful.

Does it not defy the purpose of a Wiki in the first place if not everybody has edit privileges?

After all the ability to see all edits in public and to see the edit history is deterrent enough in a corporate organization where everybody (theoretically) knows everybody.

If management is concerned about editing important documents, then they should encourage authors/owners of documents to keep tabs on edits and try to "investigate why the change has been made and if it might be actually a welcome change." (if you really need immutable documents, use attachments).

Using a Wiki as a publishing platform is missing the point of a Wiki, which is to communicate and share knowledge, not broadcast it.

Re:Similar Situation (2, Interesting)

SirLurksAlot (1169039) | more than 6 years ago | (#24613533)

I completely agree with you. This was one my biggest concerns when the idea of using a wiki was brought up at my company. We know that the real power of a wiki comes from the users, but management likely won't see it that way. You didn't quote the rest of that point where I indicate that I'm pushing for the user populace to have edit privileges ;-)

If management is concerned about editing important documents, then they should encourage authors/owners of documents to keep tabs on edits and try to "investigate why the change has been made and if it might be actually a welcome change." (if you really need immutable documents, use attachments).

Well, yes and no. The idea behind the wiki is two-fold. It's not just that users should be able to quickly and easily edit documents, but also that they still be accurate. Consider a hypothetical situation in which procedures may be changing semi-frequently in a customer-facing department, such as a call center. You want all procedures to be up-to-date, and it shouldn't take a lot effort to publish and re-publish the content, but you also don't want Joe McNewbie making changes when he may not completely understand what he is talking about. From the time he makes the (poor informed and/or possibly incorrect) change to the time at which someone spots the mistake and corrects it any number of other reps may have read the same material and used it during their calls. So now you have reps who are providing incorrect information, and possibly forcing the company to spend time/money/resources correcting those mistakes. User involvement will of course go a long way towards finding and correcting mistakes, but there is still the possibility that those mistakes will cause problems in the mean time. Of course, you could also argue that a wiki may not be the best solution in a case such as this.

Ok, so I realize I sound like I'm arguing against myself here, but what it really means is I can see both sides of the fence. Personally I'd rather give more power to the users, but I understand management's point of view as well. I suppose what it really boils down to is how much you trust your users to self-manage the wiki, and how much risk you're willing to take on by giving them the ability to make changes.

Re:Similar Situation (1)

vrmlguy (120854) | more than 6 years ago | (#24620129)

Consider a hypothetical situation in which procedures may be changing semi-frequently in a customer-facing department, such as a call center. You want all procedures to be up-to-date, and it shouldn't take a lot effort to publish and re-publish the content, but you also don't want Joe McNewbie making changes when he may not completely understand what he is talking about. From the time he makes the (poor informed and/or possibly incorrect) change to the time at which someone spots the mistake and corrects it any number of other reps may have read the same material and used it during their calls. So now you have reps who are providing incorrect information, and possibly forcing the company to spend time/money/resources correcting those mistakes.

I'd thinking out loud here, but do any wikis support a policy where anyone can edit, but a one set of users have approval rights on any edits, and a different set can only see approved pages? Or even anyone can approve, but you can't approve your own edits. Wikimedia already supports part of this, where users can watch pages for changes.

Re:Similar Situation (1)

SirLurksAlot (1169039) | more than 6 years ago | (#24620441)

Ah, good idea! Actually the MOSS wiki has this functionality; the "Workflow" feature allows you to do exactly that. I said the MOSS wiki is crippled (and I still stand by that), but that is one of the nice things about it. That would definitely open up the possibility of allowing all users to edit with an approval process in place.

Re:Similar Situation (1)

vrmlguy (120854) | more than 6 years ago | (#24625961)

Ah, good idea! Actually the MOSS wiki has this functionality; the "Workflow" feature allows you to do exactly that. I said the MOSS wiki is crippled (and I still stand by that), but that is one of the nice things about it. That would definitely open up the possibility of allowing all users to edit with an approval process in place.

It took me a minute to find (but I learned a lot about Moss, the plant, on various wikis), but I finally identified it as Sharepoint [microsoft.com] . Yeah, this is probably the way to go if (a) you're a business, (b) you've bought into the whole MS ecosystem, and (c) you want an otherwise crippled wiki that you can't improve.

I dug into Wikimedia's database schemas and it shouldn't be too hard to add an approval mechanism. I'd add just one table with pointers to a particular edit, the approving user, and the date approved; then you could treat it like any other piece of metadata. Beef up a few select statements and you're done. It looks like it could be a standard add-on.

Re:Similar Situation (1)

ais523 (1172701) | more than 6 years ago | (#24626193)

There's an extension for MediaWiki (FlaggedRevs) that does this, IIRC the German Wikipedia is using it at the moment. If you go to a random page at http://de.wikipedia.org/ [wikipedia.org] , you're likely to see "Gesichtete Versionen" on a bar at the top, showing that that version is "sighted" and approved to be shown to the public by someone with sighting privileges (a special group of users over there, but based on the way Wikimedia wikis normally work I imagine it's quite large). Anyone can view either the sighted version or (if it exists) the unsighted version (the sighters there seem quite good at their job, so most unsighted edits are quickly either sighted or reverted), but it's always clear which is which.

Connectedness (2, Interesting)

RockMFR (1022315) | more than 6 years ago | (#24609669)

The best measure of a wiki is connectedness, in my opinion. Do all pages have incoming links from relevant articles? Can all the pages be found by casual clicking, or are there some that only have links buried in the middle of paragraphs? Is everything categorized? You should have at least one well organized category hierarchy that contains everything, then add further categorizations from there.

Wrong target (1)

gmuslera (3436) | more than 6 years ago | (#24609773)

There are metrics that can be applicable to the wiki itself, but probably for what they want to know what they have to measure is the community, the people that will feed it, not the tool. Is like measuring the quality of a book by the quality of the paper or ink that was used on it.

You can use page size, amounts of edits, amount of pages, etc, but that will not imply that things are right or wrong. Sometimes a single word in the right context could say all you need. I remember that a page on wikipedia had a lot of edits (in the date of birth of someone?) because that info wasnt known for sure. A page that had a single author once is better or worse than one that had a hundred? All depends, but more important, is not there where is the answer they want.

Web stats probably best (1)

ShaunC (203807) | more than 6 years ago | (#24609843)

I'd say the best measure of a wiki's effectiveness is probably a thorough analysis of the access logs for the web server where the wiki lives. Run them through something like awStats [sourceforge.net] (personal favorite) and see what you get. Are people in your organization actually visiting the wiki? Which pages are most frequently accessed, and by whom? How many users are hitting the Edit or History portions of a page, as opposed to simply viewing static content? These are solid, quantitative numbers, and presented in a "dashboard" style format like you can get from awStats, that will probably satisfy the PHBs.

I try to avoid orphaned pages altogether. Every page in your wiki should fall under some umbrella or hierarchy, even if it's just a catch-all like "Miscellaneous," so that it can easily be found by people who didn't know it was there.

In my experience, people tend to use wikis one or both of two ways: either they do a title/text search to find a particular topic, or they just start exploring and reading what they have access to. The latter can be a great way to get new hires up to speed, or for people to gain knowledge about various business processes in their downtime. Orphaned pages are only accessible by search and IMO defeat half the purpose of having a wiki.

Why isn't this answer included in every Wiki? (1)

PurplePhase (240281) | more than 6 years ago | (#24610005)

I can understand why you'd be confused - trying to measure the quality and worth of information is a big problem (just look at Google and AI), let alone how it is organized.

There are a lot of problems with Wikis - even the benefits, that you can grow your base of information, result in problems: who is going to reorganize it so it is usable again? What are good lists and bullet points to divide gobs of text into into usable chunks of information?

Someone has already mentioned accurateness, but what that really means is correctness + point of view + timestamp of information, and these things are rarely if ever tracked normally, let alone in a wiki. Sure wiki's usually track all changes, but that doesn't mean the information is correct at the time of entry, nor that you necessarily know who's point of view is being described. And that's only if you take the time to go through all the logs! These details should be immediately apparent when viewing a page of valuable information: do you realize how many websites have articles and reviews without ever showing a timestamp of when they were written, submitted, or published?

In the ideal environment wikis would self-organize, ie. someone sees a page that's gotten too big and they take the time to divide and conquer. But that hasn't been my experience so far, and about half of the few times someone has rearranged them they end up using page titles that are awkward, and wiki's don't usually have a tools to rename a page or to list all links pointing to a page and selectively updating them (half to the new page, half to the old one with a new name...). Wiki management can be a real nightmare. Just like any large company's file repository.

Wiki SEO (especially within the wiki program itself) doesn't usually seem to be dealt with either - or if it is... I haven't seen it. I rarely have a wiki's search function turn up what I'm looking for even if I created the pages I need. So there's some other relationship to information here too that's missing some key features or concepts in our lives so far.

Anyone want to help?

Re:Why isn't this answer included in every Wiki? (1)

SirLurksAlot (1169039) | more than 6 years ago | (#24610175)

Someone has already mentioned accurateness, but what that really means is correctness + point of view + timestamp of information, and these things are rarely if ever tracked normally, let alone in a wiki. Sure wiki's usually track all changes, but that doesn't mean the information is correct at the time of entry, nor that you necessarily know who's point of view is being described.

I agree with most of your post, but you are basing this statement off of the assumption that the content of a wiki is open to interpretation. In a business setting (which the OP is apparently operating in) there is a high probability that the contents of a wiki will reflect policies, procedures, knowledge common to the company, and other information which the company has a stake in keeping everyone on the same page with. Opinion and interpretation aren't even factors in cases such as these. So the equation you mentioned really breaks down as correctness + timestamp.

Re:Why isn't this answer included in every Wiki? (1)

PurplePhase (240281) | more than 6 years ago | (#24616103)

Well, this is where we start crossing the fold.

Opinion and interpretation are factors in every case of communication. That's why EULAs and company policies have official rewrites every year, quarter, or incident. That's also why there is always an HR department whose significant responsibility (50% of time?) is to respond to questions from employee, customers, etc.

Granted some of that time is because people don't bother reading documentation or the search doesn't work, but one thing many, many documents don't include is a FAQ which (if worded clearly and properly annotated and linked) could clear up a lot of problems and frustration with communications.

It depends partly on where you're coming from. There are many kinds of business settings, really each of the types of relationships someone can have with a company:

1. Employee (many different types)
2. Owner
3. Management (many levels)
4. Policy communicator (HR, typically)
5. Policy manager
6. Customer
7. Provider (sells to company)

Each of these relationships has a different point of view and knowledge about any situation, whether it is business rules, labor laws, or when you can get in and out of the building.

It also depends a lot on definitions and what is assumed - the more you can assume, the more concise communication can be. But that also assumes we're all on the same page...

One thing not mentioned for this question is 'who is in control?' Typically wikis are open for everyone to read and modify, and if people modify the wiki instead of using a discussion page to ask questions... well it just confuses the matter. So yes, even the purpose of the communication needs some definition - hopefully explicitly.

There are two possibilities (4, Insightful)

RGRistroph (86936) | more than 6 years ago | (#24610375)

Look, the pointy haired boss doesn't really want or expect a numerical metric of organization. Immagine asking him if he has a function in Word that will tell him how organized an outline or document is. He doesn't, and if he never needed it for Word, he doesn't need it for a webpage.

Possibility 1: he wants to kill the project, and came up with this as a way to find an excuse. The best response to this is to get your resume out there on monster right now, and walk into his office tomorrow morning and say "do you want to kill the wiki, and is this metric a way to get an excuse ?" . Let that conversation go where it will.

Possibility 2: he read in USA Today that anyone can add stuff to wiki's and they are chaotic, so he's worried. He's never actually looked at any wiki including this one in his life. This is the most likely possibility. If you still have a job after asking the question in Possibility 2, go for this, or maybe start with this.

Here, you want to talk to him for about 5 minutes using the following words as much as possible, without actually lying: "review", "control", "process" and "catagorize". If you want to can reveiw everything he has ever said and come up with your own set of buzzwords, based on the words he himself likes to use, but these will do. Then you say things like "We have a PROCESS to CONTROL the QUALITY of employee submissions. I occasionally REVIEW the most visited pages in the wiki, and make sure they are ACCURATE and properly CATAGORIZED. I also REVIEW the least visited pages, and if they are not visited because they are improperly CATAGORIZED and linked, I fix that." Actually talk like that, and kind of shout the buzzwords at him. He's either stupid enough that he needs it, or smart enough to figure out what's going on and move on to something important.

All the geeks in this discussion who started actually talking about measuring connectedness of graphs and crap are totally off in the weeds.

Re:There are two possibilities (4, Interesting)

Strange Ranger (454494) | more than 6 years ago | (#24611641)

I was going to mod you insightful, but there's an easier and more positive way to apply those buzzwords...

A good PHB is mostly concerned about 3 things:

1. A decent ROI. ( From the business as a whole down to the paper-clip supplier and the birthday party commitee.)
2. Making the people above him/her look good.
3. Control. (So #1 and #2 above can be maintained over time and change. You have to stay organized when you delegate or it's impossible to manage anything)

If you concern yourself with the exact same things in the exact same order then it should be easy for you to figure out what to do and what buzzwords to say.
The doing might be hard, but the "what to do?" should become easily apparent.

In other words answer these questions as asked by your boss:
What's the ROI on your wiki project?
What's the weakest link to me (PHB) not worrying about this..meaning what person or machine or database do we need to protect and make redundant?
Who can I trust (to make me look good) when they say it's a success. I will keep asking dumb questions until I find a leader I can trust.
Who should I scold if the ROI on this project starts tanking?
In 2 or 3 short sentences, what does it do so I don't look stupid when people ask me. And when I ask "what does it do?" I do NOT mean how does it work. I mean what Return does it provide on what Investment?

Careful though, thinking like this will get you promoted FAST.

What category do you put CATAGORIZED in .. (1)

pbhj (607776) | more than 6 years ago | (#24613737)

English Language
  |_ American Spellings
        |_ Misspellings
              |_ C
                    |_ CATAGORIZED, replace CATEGORIZED, British CATEGORISED, not shouting "categorised"

If I spelt anything wrong I'm sure you guys have my back ...

Same way with any website... (1)

Blakey Rat (99501) | more than 6 years ago | (#24610941)

Install some analytics tags and look at the data. How many people are using it? How many pages are accessed every day/week/month? etc.

If it's public-facing, put ads on it and see how much money you make. :) Can't beat that for a "how much is this worth?" question.

So, in other words... (1)

Legion_SB (1300215) | more than 6 years ago | (#24611399)

... your bosses are demanding citations, and you're pissed about their lack of NPOV.

Slug it out in a good ol' fashioned edit war.

My experience... (1)

thephydes (727739) | more than 6 years ago | (#24611599)

As the person who started a wiki which has a small potential audience (10 million or so worldwide by some estimates) is that after a growth spurt at the start, it has now slowed down. Currently it gets about 100 hits per day - 3 or 4 of those are spammers adding junk pages and the same number is me deleting them, so around 90 hits per day on the front page. The metrics I use are : is the number of contributors increasing?, Are any pages being edited and how often? Are people returning? Is the number of pages growing? Whilst the accuracy of information is important, it is not in my opinion critical because the spirit of a wiki is that it can be made more accurate by another contributor.

Usege graphs (1)

Gori (526248) | more than 6 years ago | (#24612025)

I manage a large intranet wiki at a university, based on TWiki. We also do some research on its use. The main metrics I would suggest :

    * number of pages (cumulative graph over time, and per month/day)
    * number of edits, idem
    * number of views , idem
    * Number/size of attachments, idem
    * a histogram of edits/views per person
    * a histogram of people who only read and never contribute.

you can obviously anonymize the graphs if needed. A little straightforard programming-fu and access to the server logs goes a log way.

Before you show the histograms to the bosses, you need to tell them about the http://en.wikipedia.org/wiki/Power_law [wikipedia.org] , as you will be looking at one, guaranteed. 10 % contribute, 90% read. That is fine and natural, but they must understand that. You can demonstrate the fact that people read it a lot and that it is a resource they use (use the the views/views and no edits graph), and the (relatively few) people who actively contribute must be encouraged to do so, and maybe even given an explicit task to do so.

You can also plot a degree distribution histogram per page ( http://en.wikipedia.org/wiki/Degree_(graph_theory) [wikipedia.org] ), where the degree of a page is the number of outgoing links to pages + number of pages referencing back to it. If you get a power law again, you have a few important portal pages and lots of content. More uniform distribution suggests something in the lines of an encyclopedia/storage type of content. I would suggest that having a power law is better, as people find stuff more easily ( bar a good search engine...) You can also use any of the graphing tools (http://vlado.fmf.uni-lj.si/pub/networks/pajek/ or http://jung.sourceforge.net/ [sourceforge.net] ) to draw all pages out, and show different clusters of content. AGain, you can do this over time

Oh, and while you are at it, list the top/bottom 10 (20, 30) most used pages, and explain why. Observe how they change over time.

Im sure that there is enough suggestions here to get a nice progress metric for the boss. Good luck.

Gori

Sell it (0)

Anonymous Coward | more than 6 years ago | (#24612221)

Sell it. The money you get is the real world worth of that Wiki...

Irritating pop-up survey window. (1)

Bazman (4849) | more than 6 years ago | (#24612349)

When a user leaves your site, just pop up one of those irritating survey windows with a 'how useful was this wiki to you?' (You might have to say 'web site' if 'wiki' means nothing to them)..

Log some answers, or just plain old make them up, and put them in an Excel spreadsheet which you then print out and give to your PHB.

Don't use a wiki... (1)

blahplusplus (757119) | more than 6 years ago | (#24612493)

Use mind mapping software... they make professional mind mapping software for businesses.

http://www.thebrain.com/ [thebrain.com]

And it organizes information much better.

Support staffers (1)

liquiddark (719647) | more than 6 years ago | (#24612579)

If your organization is anything like mine, the most useful part of the wiki is how much simpler it makes life for the support team. Anytime a particular issue reaches a critical mass, a wiki article can go up and enter the revision cycle, and from that point forward all calls related to that issue can be answered with a single hyperlink.

A few essential metrics (1)

Conficio (832978) | more than 6 years ago | (#24613049)

I'd look at a few measures:

  • Number of pages (and their growth over time) as well as the pages/employee ratio
  • Number of WikiWords ==> links between pages
  • Ratio WikiWords/Page
  • Number of visits (and visits/employee ratio)
  • Number of bounces (and bounces/employee ratio)
  • List of top bounce pages and look if they are info pages, that stand on their own, such as company directory.

Also, institute some meetings where people go through the Wiki and try to eliminate "old news" that has only historic value. This keeps the info fresh and useful.

In addition allow for automatic discovery of new info. For example a subscription to a weekly query for chosen keywords that comes to your e-mail inbox. Set it up for the manager's names to the manager's inbox. It strokes their ego when they discover where they are mentioned (even if it is only the todo lists of last week's meetings :-)

~ot (1)

undantag (1345293) | more than 6 years ago | (#24615339)

See point #3 on the 10 Tips for a Successful Wiki [flickr.com] poster from gdc;

Invest in your wiki

Nothing is perfect off the shelf and your wiki will be no different. Most wiki packages are designed to be customized and extended, so take advantage of that by investing in a few days of developer time to modify it to suit your needs. Re-evaluate wiki usage at regular intervals to gather feedback on the most common gripes and dedicate resources to adjusting the wiki accordingly. The few hours or days spent tweaking it will come back tenfold in time saved by users.

IBM Research Tool (0)

Anonymous Coward | more than 6 years ago | (#24619175)

IBM History Flow is software that graphically represents the evolution of a live document or wiki.

http://www.research.ibm.com/visual/projects/history_flow/

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?