Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Ohloh Tracks Open Source Developers

kdawson posted more than 6 years ago | from the over-your-shoulder dept.

Privacy 115

eldavojohn writes "The startup company Ohloh has a database listing 70,000 developers working on 11,000 open source projects. Their aim is to 'rank' open source developers, which raises some interesting questions about exactly how useful this tracking company is. Questions like, 'Is there an accurate way beyond word of mouth to measure the importance and skill of a developer?' I found it slightly alarming that, to this site, the number of commits (with input from the number of kudos) tells how good a developer you are."

Sorry! There are no comments related to the filter you selected.

Accurate? Not for me (5, Interesting)

mrslacker (1122161) | more than 6 years ago | (#22505192)


I don't know how representative it is, or if it might improve over time, but I looked myself up.

I found mentions in 5 projects - _except_ they're all just versions of 2.6 kernel source with the same contribution for an obscure TV card cx88 variant I did. In practice, I'm sure I'm hardly alone in having contributions (mostly in small ways, but sometimes very considerably) to over 100 projects over the years. I guess I have to go through and add some of those projects.

Naw, CBA. At least I can make sure my resume is accurate.

So What Metrics Do You Suggest? (4, Interesting)

eldavojohn (898314) | more than 6 years ago | (#22505390)

I'm sure I'm hardly alone in having contributions (mostly in small ways, but sometimes very considerably) to over 100 projects over the years.
I also don't think you're alone in finding that metrics fail to measure good programmers. My boss constantly asks me for lines of code count from developers. No matter how many times I express this to him, this is not a measure of success or of how good a coder you are.

I tried to think of metrics to relay up the chain (a special thank you to the stat-scm goal in maven) but I come up with some pretty lame ones:
  • Code to comment ratio is desired at 1:1 (at least in the commercial world)
  • A class/method/function/procedure/module desired size should be defined and rated
  • # of Unit tests
As you can see these are the ones that I found could be automatically gathered. And even these have exceptions. Anything else I think of either takes too much time to gather or is subjective. This is tough, I would like to default to peer review but oftentimes I find teammates voicing their personal hatred for an individual or taking into account personal qualities when ranking a developer. Real Life Example: Teammate A is from MIT and teammate B thinks everyone from MIT is a god. Unfortunately Teammate A hasn't done anything but criticize everyone's code without any constructive comments to make it better.

I submitted this story hoping it would open dialog on measuring coding abilities in a semi-automated way.

Re:So What Metrics Do You Suggest? (4, Insightful)

JonTurner (178845) | more than 6 years ago | (#22505484)

# of penguin stuffed animals in your possession.
# of kernel builds
# of ICQ shouting matches ending in "Nazi!!!"
# of cans of Jolt consumed
# of steps from mom's basement to side door.

Or the old standby, lines of code (including comments, of course).

In short, there's no way to automatically judge the value of a programmer based on silly metrics. How would one score for "bugs not written" or "elegance of solution"/"nasty kludge avoided"?

FOSS is worth whatever the users and the coders say.

Re:So What Metrics Do You Suggest? (1)

infonography (566403) | more than 6 years ago | (#22505870)

Granted that those are excellent metrics, but you left out Vi/Emacs Flamewars and number of Microsoft Jokes. As to lines of code I once put Temporary Autonomous Zone by Hakim Bey [wikipedia.org] in my .plan The sysadmin for the system fingered me from a dialup and was rather pissed about having to disconnect.

But seriously, the problem with a lot of the FOSS communities is that there are a great deal of part timers. How do you measure the skills of someone who has a day job coding and the has a pet project or three? This will only hurt programmers who don't exclusively work on FOSS. The next time they apply for a gig at a FOSS company their closed source work will be overshadowed by some minor hobby work they did. It kind of ranks up there with embarrassing facebook and youtube videos.

No, no no! (2, Funny)

morgan_greywolf (835522) | more than 6 years ago | (#22505928)

of penguin stuffed animals in your possession.
# of kernel builds
# of ICQ shouting matches ending in "Nazi!!!"
# of cans of Jolt consumed
# of steps from mom's basement to side door.
No! You got it all wrong! Everybody knows that programmer productivity is inversely proportional to the number of Slashdot posts!

Re:So What Metrics Do You Suggest? (0, Redundant)

CarpetShark (865376) | more than 6 years ago | (#22506494)

In short, there's no way to automatically judge the value of a programmer based on silly metrics.


Personally, my vote is for the all-binary Microsoft-employee vs. Non-microsoft-employee model.

Re:So What Metrics Do You Suggest? (3, Informative)

ohlolo (837547) | more than 6 years ago | (#22506504)

Software development metrics are not worthless. They are, however, seriously misunderstood. This is partly why we built Ohloh to focus on Open Source: it's the world's largest testbed of available software development metrics.

One challenge to interpreting development metrics is having a clue about what is 'normal'. Just knowing your FOOBAZ count is X doesn't help much. Once you can compare your FOOBAZ count to 100k other developers, it may begin to give you some helpful perspective. Of course, relying on a single metric is myopic - which is why we offer comment ratio, language breakdown etc...

Btw, I agree that human opinion plays a vital role. That's why we also enable people hand out 'Kudos' to their peers - to acknowledge human judgement as well. The Kudo scores are then evaluated using a PageRank-inspired algorithm across all open source contributors.

Re:So What Metrics Do You Suggest? (4, Insightful)

The End Of Days (1243248) | more than 6 years ago | (#22505514)

What about some slightly deeper tracking into the repository? Figure out how much code a developer committed that had to be changed later.

Re:So What Metrics Do You Suggest? (5, Insightful)

morgan_greywolf (835522) | more than 6 years ago | (#22505608)

What about some slightly deeper tracking into the repository? Figure out how much code a developer committed that had to be changed later.
Not bad, but it depends a lot on the maturity of the project. Many young projects start out with several complete rewrites before they find a workable solution that performs well, is scalable, robust and reliable. And even on mature projects -- a lot of times and entire codebase is scrapped to accommodate new features: think Samba 3.x -> 4.x or GNOME 1.4 -> GNOME 2.0.

Re:So What Metrics Do You Suggest? (0)

Anonymous Coward | more than 6 years ago | (#22507348)

True. Then again, the GNOME 1.4 developers did kind of deserve death by firing squad.

But what's the point? (4, Insightful)

EmbeddedJanitor (597831) | more than 6 years ago | (#22506212)

Making OSS dev is hardly a competition sport, so why do this? Kudos is one thing, but that should come from the community, not from some database.

Some people will get a shiny glory and some will feel annoyed bbecause their projects/contributions have not been tracked.

Re:So What Metrics Do You Suggest? (2, Informative)

madmak (743091) | more than 6 years ago | (#22507930)

Exactly. There is much more to get out of the code repository. Check http://www.sourcekibitzer.org/ [sourcekibitzer.org] metrics. It is able to extract developers "know-how" score by crawling the repositories. Much better than simple Lines of Code.

Re:So What Metrics Do You Suggest? (1)

FatdogHaiku (978357) | more than 6 years ago | (#22510728)

Come on, no matter how you rank people, this just ends in a lawsuit when someone does not like their ranking, especially if the ranking does track errors and poor performance somehow.... It's sure not going to help build community or aid actual development... unless someone is developing a giant pissing contest.

Re:So What Metrics Do You Suggest? (1)

lymond01 (314120) | more than 6 years ago | (#22505636)

It's a little tricky because talent is only as good as what's seen. From a business perspective, they don't care so much about the getting there, as long as there's something done within deadline (but they want to know you're working to meet that deadline so they count your lines of code).

Nice thing about open source code is that it is reviewed by other developers as it goes. This site might be worthwhile if it has good input from thoughtful people. Like Slashdot people! The shiny...happy kind...! Right?

Re:So What Metrics Do You Suggest? (1)

samkass (174571) | more than 6 years ago | (#22505718)

The SEI and various other organizations focused on process have a lot of suggestions. There are some simple ones, such as "function points" * "complexity" as a measure of productivity. Or defects per function point as a measure of quality. Or some from the extreme programming world of user stories completed.

But nothing is really truly accurate, only guides humans. If it were, managers could be robots.

Re:So What Metrics Do You Suggest? (3, Insightful)

timewasting (1230064) | more than 6 years ago | (#22506284)

I always hated the lines of code metric because it was so useless. A really good architecture with a good coder leads to much LESS code that looks simple to the casual observer. Similarly, the number of comments might not be very useful as well. Spaghetti often has lots of extra comments, while good code is often self-commenting (at least in my commercial realm). Lots of unit tests are often unhelpful if there are no automated system tests. The best coders balance design, coding and testing in the scope of their deliverables and can effectively estimate how long it will take ahead of time. The art is coming up with a good architecture based on standard patterns and following up with concise, self-documenting code that is easy to maintain and enough tests to know when it works and more importantly when it doesn't. If you can do that in a reasonable amount of time, and less time than what is expected, you're good. If not, you need to be refactored.

Re:So What Metrics Do You Suggest? (1)

eh2o (471262) | more than 6 years ago | (#22507418)

Here are some more that can be derived automatically:

- Code cleanliness (e.g. consistent camelCase, whitespace, comment styles)
- Statistics of commits over the same section of code (may imply that it is buggy)
- Rate of API breakage -- how long does a function last before it's name/args/return value changes. This metric could be "weighted" by evaluating the prevalence of the function. A high rate implies an unstable design.

Some of the ones I don't like that Ohloh uses:

- "Mature" meaning that the project has a long history. That tells me nothing. Some of those projects had such pathetic early versions that it would be insane to use them today. On the other hand, there are probably millions of active Apache 1.3 installs. I would call a project mature when its currently maintained major version(s) have a long history.

- "Very Large Development Team". e.g., Drupal (contributions) with 575 developers. But does it really make it better? In my experience 95% of that project is junk. The quality of each contributing member needs to be evaluated.

Re:So What Metrics Do You Suggest? (1)

bishiraver (707931) | more than 6 years ago | (#22507726)

Number of unit tests doesn't necessarily denote good unit tests.

It would be a better metric to show what percentage of your codebase is actually covered by unit tests.

Using Test-Driven Development ensures that this percentage is notably high.

Re:So What Metrics Do You Suggest? (0)

Anonymous Coward | more than 6 years ago | (#22509164)

If you're not already enforcing this through dev tools, some code metrics can be gleaned from http://checkstyle.sourceforge.net/ [sourceforge.net]

Things like cyclometric complexity, Fan out, etc

If someone tends to write things with a very high cyclometric complexity I'd tend to say they can improve the legibility of the code they write. If they tend to have a high fan out, I'd say they could generally spend a bit more time modeling and designing.

This of course makes the assumption that you're aiming for OO software

Re:So What Metrics Do You Suggest? (1)

Pseudonym (62607) | more than 6 years ago | (#22510184)

My boss constantly asks me for lines of code count from developers.

I bet I know how you initialise a 10,000 element array, then.

The "Karma" Algorithm I use (1)

adamkennedy (121032) | more than 6 years ago | (#22511038)

After I gave up looking for good single metrics, years ago I invented a "karma" algorithm for use in CVS Monitor (google it).

The karma score is calculated for a single commit, and uses a combination of lines and files added/removed/changed (with some munging so that file moves don't get line scores), and then adds to that the size of the commit message in bytes, and then applies a maximum upper score per commit.

It's worked pretty well, since all the best ways of gaming your karma score also encourage good practices... for example.

* Write longer and more detailed commit messages

* Commit in several smaller pieces rather than large amounts at once

* You get (up to) double points if you implement on a branch, and then merge the branch to trunk. As a bonus, you can pick up major points if you are the guy that takes responsibility for merging all the brances to trunk.

Looking at several historical projects, the scoring system does seem to reflect the "importance" of each developer in a project to a reasonable degree.

Re:Accurate? Not for me (1)

morgan_greywolf (835522) | more than 6 years ago | (#22505470)

Ohloh doesn't even list me. On any projects I've contributed to, or even my own. *sniff*

Like I actually care. I don't do this stuff for recognition anyway. I do it because it's fun.

Re:Accurate? Not for me (0)

Anonymous Coward | more than 6 years ago | (#22505546)

Same here, missing about half my linux-2.6 commits (which is several hundred) even though they're correctly attributed by git.

Re:Accurate? Not for me (1)

Durin_Deathless (668544) | more than 6 years ago | (#22505748)

Commits are definitely a bad metric - for one of the projects I'm involved with it shows I have 24 out of something like 800 commits, which doesn't factor in the commits I made while it was still a private project in a repo with other things, and it doesn't count research and political work required to make the project happen. Also, lots of my contributions to things have been submitting a patch, which got me a mention in the commit log, but Ohloh doesn't pick up on that.

Re:Accurate? Not for me (1)

rgbecker (240211) | more than 6 years ago | (#22507844)

Well I'm not there, but neither was Guido van Rossum so I suppose you can call yourself an ubergeek :)

SLASHDOT SUX0RZ (-1, Offtopic)

Anonymous Coward | more than 6 years ago | (#22505362)

_0_
\''\
'=o='
.|!|
.| |
here is a goatse. please to enjoy. [goatse.ch]

I can't imagine... (0)

Anonymous Coward | more than 6 years ago | (#22505420)

I'm sure I'm not the only one who looked at that headline and wondered "How the hell did even a Slashdot editor misspell "Ohio" that badly? Even Taco could probably get within one letter of correct, if only because he's from Michigan.

sounds familiar (5, Informative)

Anonymous Coward | more than 6 years ago | (#22505450)

This reminds me of how academics are increasingly judged. It is more about how many papers and how many other people link to it rather than the quality of each paper's work or the note of the linking party. Accordingly, many authors inflate their 'impact' scores by splitting up papers and publishing nonadvancing science, no-one can blame them for this as many are trying to justify themselves to their departments or are still doing the postdoc merry-go-round looking for new jobs every 18months.

no thanks (2, Interesting)

dcskier (1039688) | more than 6 years ago | (#22505452)

You can't effectively rank developers. First there are just too many to rank. Even in college football, where thousands of people are paid everyday to monitor it, they don't try to rank all of the ~119 Div 1 teams, just the top 25. Secondly there isn't a simple metric to rank developers. It's about as smart as saying look I did the most work on this project because I wrote the most lines of code.

This could even have a negative effect if developers get concerned about their ranking and try to game the system instead of making quality contributions to projects.

Same here (5, Insightful)

jaaron (551839) | more than 6 years ago | (#22505454)

Most of my contributions were on website documentation, wikis, or mailing lists, which aren't included in these metrics. At the moment, a lot of my commits are done on repositories not directly available to the public. While I don't really need Ohloh to tell me if I've contributed to a project or not, it's still a little annoying.

And what about contributors who submitted patches that had to be committed by someone else? Or people who contribute by providing help on IRC channels, blogs, forums, or other mailing lists?

While ohloh metrics can be useful, they also need to be taken with a grain of salt, particularly the contributor metrics. They're a bit more useful on measuring a project as a whole (but they still miss a lot of activity).

Listed twice (2, Interesting)

solprovider (628033) | more than 6 years ago | (#22508366)

I am listed as two people with the same pseudonym; my real name is not found. I am listed for two related projects belonging to the same organization. Both of me have the same score albeit for different skills. Ohloh obviously only checks commits to the main branches; my commits and LOCs to an experimental branch of one project would drown my official commits. I won commit status due to my assistance on the mailing lists and a lengthy complicated patch for critical functionality; my name is in the credits, but the commit is credited to someone else. I cured a critical bug in a third project (a component affecting many FOSS projects), but my patch was included as text in the bug tracking system and committed by and credited to someone else. The system measures lines of code, but great programming often reduces LOCs (as happened in the patch to the third project.) I am about to post code in a bug-tracking system that will not be integrated into the project due to management's objections ("This function has been broken for 8 years and we are afraid to fix it"); this will not be credited. Contributions to official documentation are ignored, as is useful information on personal websites.

Another committer has a much higher score. He is involved in more products and has committed tons of code to the official branches. He has kudos from other developers. From researching several developers whose work I know, I agree with the scores (except mine. My fame is in the corporate world for completing critical, urgent IT projects. I am a very minor player in the FOSS world. I am surprised my score is so high based on the limited contributions considered.)

Some suggestions:
1. List one person as one person. The organization requires a unique pseudonym for all projects. Start with the page translating the pseudonyms to real names.
2. Look at mailing list activity and distinguish between questions and answers. The first post in a user thread is a question; additional posts are often requests for information; the final post not from the original poster is usually the solution. The first post in a dev thread is often a new idea; additional posts are clarifications so give credit for length to avoid "+1" posts but avoid crediting lengthy log listings.
3. Look at bug-tracking systems. Many official commits are patches provided by non-committers through the bug-tracking system.
4. Counting LOCs is usually a poor determination of the quality of contributions. One of my official commits added code. My commit was reverted and replaced with half the LOCs of the original code. While I would accept credit for the concept that was implemented, those LOCs should not contribute positively to my score. Did the committer writing the excellent replacement code lose points for reducing LOC?

Ohloh ignores many important types of contributions and poorly assigns credit for much work to the committer rather than the contributor. Ohloh does not distinguish between types of commits; many commits are correcting bugs introduced by the same committer. Assigning importance ratings to commits requires integration with bug-tracking systems; CVS and SVN do not have scoring mechanisms.

Ohloh cannot score personality. Much of my career has been accomplishing the "impossible" (whether functional or due to deadlines) as my focus is on business usefulness rather than technical limitations. One person (scored 9 by Ohloh) is an incredible programmer once someone proves something can and should be done, but is extremely resistant to new ideas. Creativity is not reflected in the scores.

Ohloh has a good idea. I like the kudos system (and hope merit wins over popularity.) The system still needs work to be accepted as a reliable source of useful information.

Re:Same here (1)

ubernostrum (219442) | more than 6 years ago | (#22509952)

While ohloh metrics can be useful, they also need to be taken with a grain of salt, particularly the contributor metric.

Particularly all of their metrics. Another example: one metric is the ratio of code to comments. This is debatable enough in and of itself (there are an awful lot of people who think code should never be commented, on grounds that code which requires comments must not be written clearly), but on top of that it takes a very narrow view of what constitutes a "comment". For example, a Python docstring doesn't count, which means that Python programmers who do the right thing will be penalized by the comment metrics.

business model (2, Interesting)

magarity (164372) | more than 6 years ago | (#22505502)

What the heck is their business model, or is this just a hobby site? About the only way I can think of to make some money is to take some under the table in exchange for a higher rating.

Re:business model (1)

darthflo (1095225) | more than 6 years ago | (#22506012)

TFA says classifieds, them digging through their data on request, subscriptions for them monitoring projects/devs.

I imagine they might make quite nice head hunter "equipment". Think company x wanting to incorporate SMB connectivity to their closed source product (and requiring a developer to do so). Instead of digging thru or placing classifieds, Ohloh would hook them up with the top 10 contributers to smbfs. This could turn out to be a great deal for both sides. Company X gets a dev really skilled in the task he's going to work on, Developer Y gets paid better for a gig in a field he's interested in.
If Ohloh should succeed, a few hundred commits in their database could be a great independent point in anyone's CV.

Their model is simple (1)

CarpetShark (865376) | more than 6 years ago | (#22506450)

It's a very simple model really, when you think about it. Let's examine their possible train of thought:

Sites can sell advertising when they get lots of frequent users. Sites need users to get users. Sites need some kind of user list to bootstrap. Where can you get a big list of users from? Why, isn't that opensource stuff based on lots of people communicating in the open, over the net? Oh, hey, let's use those suckers. Hmm. How can we make more suckers sign up after the first ones? Hmm... we need to make the ones who aren't in the DB feel like they should be. I know... rank them!

Hmm... do these people care about rankings? Will it actually be useful? Ahh... Who cares?

Re:business model (1, Interesting)

Anonymous Coward | more than 6 years ago | (#22507864)

Their website says they have independent investors. Maybe this is just a technology demonstration, or maybe they'll sell subscriptions when it's working well, or maybe they'll allow people to buy homepages to better present their skills, or allow companies to buy a customized search for programmers they want to hire. I can think of a dozen possible ways to make money off this.

Remember, a startup virtually never ends up doing what they start out doing. (The Apple guys started out making phone-hacking blue-boxes.) Even if they intend to make it big by tracking open-source developers, I doubt that will be their primary business in 5 years.

It's kind of sad that the an initial response to an interesting new company is 'they're not profitable today, so the only possible business model is bribery'.

I'm in the same boat: working at a startup with private investment but not taking money from users yet. We've been accused of being a front for various giant .coms (I wish!). Trust us, when we have something that's good enough that users will want to pay for, we'll have no problem flipping the switch and charging users for it. Until then, just chill.

Would this discourage contributers to open source? (3, Insightful)

softwaredoug (1075439) | more than 6 years ago | (#22505510)

Would this discourage contributers to open source projects? Now if I put on my resume that I've contributed to an open source project, somebody is going to want to look me up. I have to deal with all that baggage when I just wanted something to do in my spare time. Also, I really am not sure I feel comfortable being given an absolute rank. People always bring different skill/approaches to different jobs and I don't think you can arguably say one is better than another. I've worked in teams where everyone respects the different capabilities and limitations of each member. Its sort of like arguing there is an absolute thing known as "intelligence". Is there really such a thing or do we just all bring different skills/perspectives/approaches to the problems we solve? I'd prefer to think the latter, that everyone contributes what they can but has their own limitations. Talking about absolute "intelligence" or "value" seems condescending and elitist.

Re:Would this discourage contributers to open sour (1)

Bananenrepublik (49759) | more than 6 years ago | (#22506620)

Most open source projects' commits are already gathered on cia.navi.cx [cia.navi.cx] . I don't see what ohioh can add besides a link to the real name, which is easy enough to find out anyway.

Re:Would this discourage contributers to open sour (0)

Anonymous Coward | more than 6 years ago | (#22506860)

you don't have to use your real name on the interwebs, hoss

commits (0)

rucs_hack (784150) | more than 6 years ago | (#22505524)

It's as good a measure as any.

Few commits means either you're Donald Knuth, or you're not that actively developing your code.

In Open Source active development does tend to mean a reduction in crapness, software wise.

What else it could say I don't know, but since there are few, if any definitive means by which code quality can be measured (and don't give me that lines of code versus man hours rubbish, I heard enough of that nonsense at uni), it's probably a reasonable metric.

Nonsense (1)

Jane Q. Public (1010737) | more than 6 years ago | (#22505682)

It's not a measure at all. For example, on some projects a ajor contributor might not even have commit rights... the code would have to be committed by others. And so on.

They are attempting to measure something for which there is no consistent measure. As a consequence, there is no question that their "ratings" MUST be distortions.

Re:Nonsense (1)

softwaredoug (1075439) | more than 6 years ago | (#22506172)

Indeed, good point. Any metric can be used to the savvy developers advantage. Some people track how many bugs a piece of software has after a release... We just group similar bugs into one bug. When you know the rules, you can use them to our advantage.

I believe Fred Brooks points this out in the Mythical Man Month.

Re:commits (3, Interesting)

krog (25663) | more than 6 years ago | (#22505776)

It's as good a measure as any.

You might be right, but it still sucks. And in the case when all your solutions are crap, I think it's dishonest to present any of them as actual, workable solutions.

Re:commits (1)

kelnos (564113) | more than 6 years ago | (#22507518)

Agreed. And note that Oholoh appears to, by default, only track 'trunk' branches for subversion repositories. So if I spend 6 months and 500 commits working on something on a branch, which I later merge into trunk (one commit), that history isn't tracked. Sure, you can add the branches/ dir to the Ohloh 'enlistment' (what a stupid word), but it seems to be at least frowned upon.

And not all commits are code. A decent percentage of the commits in my projects are i18n/l10n-related. Those are even harder to track as 'contributions' because we have fewer committers than contributors -- many translators just send patches to our i18n mailing list, which those with commit access commit to the repository.

Also, for a while we had a translations maintainer who periodically (it might have been every day) ran 'make update-po' in the po/ directories of all our modules, which updates all the translation files to account for changes in the code, and then committed those changes. Probably the majority of those commits were just for trivial line-number comment changes in the .po files. Regardless, an automated script making daily commits would artificially inflate that committer's stats.

I know one developer who, when working on a new module, codes on it in private (possibly using a private svn repository, but maybe not) until he thinks it'ss ready (and he has very high standards), and then just imports the current state into the public repo as a single commit. So that's more history lost.

I guess this is also a question of how project/server maintainers are about preserving project history. Some people might not have cared so much in the past, and might have thrown away history when (for example) switching SCMs.

So yeah... going by commits in a restricted set of branches of public repositories is a bit questionable, and that's even ignoring the question of "does having more commits really mean you're a better or more valuable developer/contributor?" And I'd even say the answer to that question is "no" or at best "it depends," anyway.

Having said that, it's still a fun site to play with.

Re:commits (1)

epine (68316) | more than 6 years ago | (#22506808)

It's as good a measure as any.
Talk about damning with faint praise. That's how I aspire to be evaluated: lined up naked against a wall while my vital statistics are transcribed by a group of bonobo monkeys. Hey, it's as good a measure as any.

In fact, the monkey-measure is probably better than commit-count, because no matter how my spam box bulges, the monkey-measure is less likely to persuade me to exchange an effective work habit for an ineffective work habit in an effort to sway a useless statistic.

People who fail to vomit when linearity is projected on inherently non-linear systems soon become known as economists or middle managers.

There is a potential upside. If this new bit-pimp metric catches on, I'd love to be able to sell my commit-count history on eBay, or perhaps pull an exchange for a level 70 Druid.

Re:commits (1)

SanityInAnarchy (655584) | more than 6 years ago | (#22510032)

If you have commit access, maybe.

Consider the case where you don't have commit access, which means that not only is someone else committing for you, but they're probably committing it all in one big patch, rather than lots of little increments.

I know that at work, where I have commit access, I tend to commit all the time -- if I need to not break things for everyone else, I make a branch. But for open source projects, I might do a weekend's worth of coding before sending it in, and even then, someone else gets commit credit.

You're sort of right, though -- it's pretty impossible to find a good metric by which to measure productivity.

Number of commits? (4, Interesting)

Tarlus (1000874) | more than 6 years ago | (#22505532)

So in other words, I could commit some of my own code to a CVS repository, find some errors that I missed, fix them, commit it again, decide to add more comments, commit it again, find one more thing I probably could have done differently and then rewrite it, commit it again...

And I would be ranked highly as a great developer?

Re:Number of commits? (3, Insightful)

rucs_hack (784150) | more than 6 years ago | (#22505576)

Certainly as a diligent one who actually cares about code quality.

Anyone who thinks they can produce bug free code first try is an idiot. Anyone willing to accept that there are always going to be bugs, and actively looks for them is a good coder.

Re:Number of commits? (1)

Aladrin (926209) | more than 6 years ago | (#22505864)

Or was too stupid to code it right the first time.

I'm not calling him stupid... That was the example he was trying to get across. By this metric, stupid programmers that commit a lot because of mistakes are rated as highly as highly-motivated, caring programmers who commit a lot because they have a lot of additions to make.

Re:Number of commits? (1)

rucs_hack (784150) | more than 6 years ago | (#22506278)

What?

You can't be a very good coder yourself.

Making mistakes != stupid coder.

Making mistakes, finding them and fixing them makes a good coder, nothing else.

Sure the fix might not be great, it might be an ugly hack that needs to be refined, but you're only a bad coder if you don't fix your mistakes, not if you make them in the first place.

Re:Number of commits? (1)

kelnos (564113) | more than 6 years ago | (#22509116)

No, you're also a bad (or perhaps just "sloppy") coder if you constantly make mistakes. Sure, everyone makes mistakes. But some mistakes are "stupid" and some aren't. If you commit broken code that doesn't even compile (and you didn't even test it), and then make another commit to make it compile, then was that first commit a "stupid" mistake? I'd say yes.

This commit count thing is meaningless anyway. Some people commit more or less often, depends on personal preference. Just about the only things you can say are things like "this project has had no commits in 4 years, therefore it hasn't been very active in a while." You can't even make a good/bad value judgement from that statement.

Re:Number of commits? (1)

rucs_hack (784150) | more than 6 years ago | (#22510052)

Then I suggest you fail to understand the purpose of versioning systems. Their purpose is not to store perfect code. The purpose is to store the latest changes to code.

As a rule, the working, most tested versions of code appear in stable releases. These may themselves be in a versioning system, but as a rule they are marked as distinct.

Re:Number of commits? (0)

Linegod (9952) | more than 6 years ago | (#22505950)

If they cared about code quality, they would test it before they committed it.

Re:Number of commits? (1)

Tanuki64 (989726) | more than 6 years ago | (#22506604)

This is moderated 'insightful'? But definitely not by real coders. More by wannabes like a previous project manager of mine, who, whenever he found one of my bugs, complained I would not test my code. I just wonder how he knew that when he found one, that I previously did not remove 500, which I found myself through testing.

Re:Number of commits? (1)

Knara (9377) | more than 6 years ago | (#22507738)

The concern here was about people committing over and over again in a short period to fix bugs and thus being rated higher, not that you went "oops" and fixed a bug once or twice.

Re:Number of commits? (1)

Linegod (9952) | more than 6 years ago | (#22508716)

Exactly.

Re:Number of commits? (1)

linal (1116371) | more than 6 years ago | (#22505612)

if that was the case then i would be the best developer in the world

L337 development skills! (3, Funny)

hellfire (86129) | more than 6 years ago | (#22505700)

So in other words, I could commit some of my own code to a CVS repository, find some errors that I missed, fix them, commit it again, decide to add more comments, commit it again, find one more thing I probably could have done differently and then rewrite it, commit it again...

Your willingness to fix errors, add comments, and do code rewrites puts you in the pantheon of programming gods! The next thing you are going to tell me you actually write your own legible "how to" user guides in PDF!

Re:Number of commits? (1)

bdaehlie (537484) | more than 6 years ago | (#22505972)

I think each commit is weighted by the rating of the project you commit to. So no, that probably wouldn't get you very far.

A couple commits to Firefox, the #1 rated project, is probably worth many commits to a less highly rated project. I'll let you go to their site and figure out how they rate projects.

Re:Number of commits? (1, Informative)

Anonymous Coward | more than 6 years ago | (#22506318)

If you look around, eg javawoman not happy thread [ohloh.net] it is a bit worrying that an ex cat herder from M$ is not only behind the wheel, but clearly has the same marketing speak to shut up unhappy fodder for the soon to be commercialised service: ohloh_goes_open_source [ohloh.net]

Pedantic tip... (1)

Otter (3800) | more than 6 years ago | (#22505536)

"Kudos" is not plural, just a word that happens to end in "s", like "pathos". "Kudo", as used on that site, is as meaningless as "etho" or "mytho". The more frequent references to "many kudos" or other treatments of it as discontinuous are also incorrect, although much less jarring.

Re:Pedantic tip... (1)

kelnos (564113) | more than 6 years ago | (#22509264)

That was the first thing I thought of as well when I noticed their "kudos" system. Then I looked it up, and it turns out [reference.com] that kudos actually is the plural of kudo, and you can use them either way. I've never heard of it used in the singular form before, though.

Flawed, but interesting. (3, Insightful)

urcreepyneighbor (1171755) | more than 6 years ago | (#22505554)

Wouldn't such a system assume that everyone uses only one handle - or, their real name - all the time for every project? If so, then a lot of people - who contribute under multiple handles, nicks, whatever you want to call their identities - are going to missed or severely under-rated.

I would rather not have my real name attached to most of what I've contributed. One, because my code is so damn sloppy that it's embarrassing. Two, because I don't want the hassle of my real life - you know, offline - and my, uh, "digital lives" conflicting with each other. Three, if I was easy to find - online - I run the risk of being pestered with silly tech support questions.

UrCreepyNeighbor, while an accurate description of my personality, is one of many identities I have. Same could be said of almost everyone. I'm sure "HotChic17CA" doesn't use that username when she's talking with her grandmother, for example.

Re:Flawed, but interesting. (1)

tokul (682258) | more than 6 years ago | (#22506090)

Wouldn't such a system assume that everyone uses only one handle - or, their real name - all the time for every project?
It does not. Two persons with same nick handle can commit to different projects and they are not counted as one person. One person can have different nick handles and link them to own ohloh id in project properties.

Re:Flawed, but interesting. (2, Informative)

ohlolo (837547) | more than 6 years ago | (#22506194)

Disclaimer: I co-founded Ohloh. We let people 'claim' development contributions individually. So your Ohloh account can be related to as many (or as few ;-) nicknames on projects as you like.

Re:Flawed, but interesting. (1)

Knara (9377) | more than 6 years ago | (#22506542)

What stops me from claiming nicks which are not my own?

Re:Flawed, but interesting. (1)

Compholio (770966) | more than 6 years ago | (#22508290)

I just registered some of my own, the answer is "nothing".

Re:Flawed, but interesting. (1)

grumbel (592662) | more than 6 years ago | (#22507066)

Wouldn't such a system assume that everyone uses only one handle - or, their real name - all the time for every project?
You can register on the page and link all the different aliases back to together to refer to a single person.

I think its a much bigger issue that all those people sending patches will be ignored, since there isn't really a standard way in most version tracking systems to keep track of the patch submitter instead of those that actually commit it into the repository.

Re:Flawed, but interesting. (1)

kelnos (564113) | more than 6 years ago | (#22509304)

I think its a much bigger issue that all those people sending patches will be ignored, since there isn't really a standard way in most version tracking systems to keep track of the patch submitter instead of those that actually commit it into the repository.
That's one of the cool things about git (and maybe others). Each commit has both a "committer name" and "author name" field.

You can claim aliases (1)

Per Abrahamsen (1397) | more than 6 years ago | (#22507198)

There are facilities in ohloh to map aliases. It seems everyone can do it. I don't know if there is any conflict management, they seem to take wiki approach throughout their database.

I don't seem a problem in that free software contributions you want not to be associated with your legal name, won't be associated with your legal name.

Re:Flawed, but interesting. (1)

Kozz (7764) | more than 6 years ago | (#22508212)

I'm sure "HotChic17CA" doesn't use that username when she's talking with her grandmother, for example.

Definitely not. Instead, he probably goes with AgentJohnSmithFBI.

Re:Flawed, but interesting. (1)

mweather (1089505) | more than 6 years ago | (#22508328)

"I'm sure "HotChic17CA" doesn't use that username when she's talking with her grandmother, for example." Nor does granny use BonsageSlut69 when talking to the grandkids.

Names can be linked together. (0)

Anonymous Coward | more than 6 years ago | (#22508662)

If you could be bothered going to Ohloh and look at it, you will see that if you create an account you can then link to that all the different names you have used for different projects so that Ohloh knows they are all you.

Slashdot (1)

sm62704 (957197) | more than 6 years ago | (#22505578)

You only have to comment on slashdot to tell that this is a really bad idea. You have people modding a comment "troll" because they don't like a stated opinion, for example. You have people modding a first post as "redundant" and a spot-on comment as "offtopic". People suck, especially at judging other people.

And on a thing like that, you may have someone who knows absolutely nothing about code making judgements about coders.

It's a stupid idea. It actually sounds like some harebrained idea thought up by a PHB. This idea needs to die quickly and horribly.

-mcgrew

Re:Slashdot (1)

Tridus (79566) | more than 6 years ago | (#22505742)

If the first post is "FIRST!", Redundant isn't bad. I mean thats been done how many zillions of times already? At this point it is redundant, but in the context of the site as a whole rather then one individual article.

Of course if they gave me "-1 (Idiot)", I'd use that on the "FIRST!" people instead.

Re:Slashdot (1)

LMacG (118321) | more than 6 years ago | (#22505942)

I don't think he was referring to the canonical "first post", but to the first posting of an idea/thought/comment, which is subsequently repeated by somebody else. The somebody else should get the redundant mod (given due diligence in checking timestamps and allowing leeway as necessary).

And I'll agree with the GP that this Ohloh thing sounds like it came straight from a PHB.

Re:Slashdot (1)

sm62704 (957197) | more than 6 years ago | (#22506304)

I won't argue with that, but I've seen first posts that were actually commenting about the topic itself that were modded "redundant". In fact I try to not submit a comment until I see at least one other posted, because otherwise it will be modded to oblivion and nobody will see it, so what's the point of making the effort?

Fortunately it seems that whenever I happen to hit slashdto right when they've posted something new I'm invited to metamoderate, so by the time I'm done with thet there are at least a dozen posts already.

Re:Slashdot (0)

Anonymous Coward | more than 6 years ago | (#22505782)

You have people modding a first post as "redundant"

It is entirely sensible to mod a first post as "redundant". People don't come into the discussion with an empty mind (although some act as though they did), so it's redundant to point out things that everyone's likely to know already, or to make a joke that everyone's heard before. More generally, for a comment to be "redundant" means that it adds nothing of value to the discussion, without being actively negative like a flame or a troll.

What An Incredibly Inane Idea (1)

Jane Q. Public (1010737) | more than 6 years ago | (#22505638)

This is completely ridiculous. They are attempting to judge value when there simply is no objective measure for the kind of things they are trying to judge.

I think I would object to having my name listed on this site, even if the "rating" were high.

Re:What An Incredibly Inane Idea (2, Insightful)

Tanuki64 (989726) | more than 6 years ago | (#22506310)

They are attempting to judge value when there simply is no objective measure for the kind of things they are trying to judge.
This is only half true. You can judge the quality of code. How good does is comply to the OO principles? Makes the design sense? Does it look maintainable? Robust? Reusable? How is it documented? There are even some metrics, which could be measured by static code analysis programs. Nevertheless I doubt that for each with Ohloh registered software project a senior software engineer will spend time to analyze the code independently from its purpose and give an objective evaluation of the skills of the programmers.

Re:What An Incredibly Inane Idea (1)

BeanThere (28381) | more than 6 years ago | (#22510698)

They are attempting to judge value when there simply is no objective measure for the kind of things they are trying to judge.

I think you're confusing "no objective measure" with "no known algorithms to computate those measures". Just because we don't know how to implement meaningful metrics with a computer or with some formulas, doesn't mean they don't exist. The proof of this is that other humans who are very good coders *are* actually, on the whole, pretty good (in fact, the *best* system out of them *all*) at judging the quality of other coders - MUCH better than any known computational metric - that doesn't mean what they're doing is terribly subjective, there are effectively objective but poorly understood processes in the brain that analyse quality (based on amongst others aspects that Tanuki64 already mentioned) (we just don't know how to 'duplicate' those quality-analysing 'algorithms', but they must exist, because our brains can do them). The book Zen and the Art of Motorcycle Maintenance actually discusses this whole notion of 'quality' and the brain's analysis thereof.

In the "old days", instead of PHBs who like to think everything in business (including performance 'metrics') can be reduced to little formulas, we used to just promote to managerial positions the people who were just really good themselves, and *they* would usually judge who was good or bad, and that was the end of it. And there is nothing wrong with this, it *worked*, despite our contemporary egalitarian indoctrinations crying otherwise ("but, but, it could be unfair!", boo hoo cry me a river - the occasional incidences of poor evaluations would be more than compensated for by not only an overall improvement in analysing the quality of workers, AND consequently better *appreciation* of actual quality work done, which is important for motivation and morale) - plus, managers who actually understand what they are managing tend to be better. I'd like to see a return to that 'old' culture, but it sometimes almost seems like 'stupid' PHBs have managed to take over and now protect one another, keeping themselves enriched and in employ even as they destroy companies (but that's another topic). Those egalitarian notions where don't want to be *judged* (except by so-called "impartial", "fair" machines/formulas) also contribute to our own downfall.

Another approach might be to let the coders all evaluate and score one another via secret ballots; the aggregate scores could be (very very loosely) used as metrics - or rather, a 'rough guide' rather than absolute analysis - it's far from perfect, but it would still be much better than any automated metric, and better than letting a dumb PHB decide. It might suck for the 'overly arrogant and rude guy that nobody likes', but OTOH might make them be a bit nicer :)

Metrics like LOC make people motivated to add lots of pointless lines. Yay. Some of the the most quality commits I've ever done were deletes, and the most quality module rewrites I've ever done almost always allowed the system to do more with LESS lines of code. I constantly strive to streamline my code.

Sing for it! (1)

lymond01 (314120) | more than 6 years ago | (#22505732)

Like the bards of olde, OS devs don't code for money. They code for prestige and fame amongst their fellows! Surely this site will decide who is the greatest dev to walk the earth. And that dev will have his own code set in stone and copied for ages to come. That developer will be legend.

Unless, heaven forbid, the voting is more like the U.S.'s political system.

Advogato (1)

NaCh0 (6124) | more than 6 years ago | (#22506146)

It's been done before. It is called advogato [advogato.org] . This is a site where developers can join, blog, and rate each other based on a trust matrix.

Re:Advogato (0)

Anonymous Coward | more than 6 years ago | (#22508904)

That was the first thing I thought of as well, but Advogato doesn't grind through its data in depth. It just has simple relationships where one coder can anoint another as Apprentice, Journeyman, or Master. Now say you want to know "is Bob l33t3r than Jim?" when neither of them have a direct relationship to the other, and you are to lazy to ask their friends and research their code and make an informed opinion. You want the computer to answer the question.

A program could use Advogato's database to answer this question by giving everybody a ranking based on how many people consider them better or worse coders than them and comparing who is doing the considering. Of course, there is a problem with data input -- does "Master" mean "Better Than Me" (the easier for an algorithm) or "A Really Good Coder" (the likely human interpretation)? Also, what happens to the rankings if some jerk rates rms as an "Apprentice" and then calls someone like me "Master"?

Going back to the beginning, the reason this article was posted is because Advogato doesn't do this yet and Ohloh does. That's why it is nifty. It also tries to match developers to projects they work on and programming languages, although it is clearly buggy (Firefox is written in Javascript? No wonder it is so slow!)

Other frustrations (1)

djmitche (536135) | more than 6 years ago | (#22506256)

I work on Amanda, but the site misrepresents my contributions in two important ways, too: first, I commit a lot of other peoples' patches, so my name appears in the ChangeLog a lot less often than it appears in the commit log. Second, Amanda changed from CVS to Subversion a few years back, and Ohloh doesn't index the old CVS submissions. As a result, the project is marked as just a few years old (it was originally written in '92), and few of the many historical contributors are not listed. I would like to see some way to "correct the record," but I suppose that's pretty hard.

Re:Other frustrations (1)

r_cerq (650776) | more than 6 years ago | (#22510492)

I suppose that really depends on the SCM and how the commits are structured or processed.
I, for example, have commit access to one of the Linux kernel subsystems; about half the changes I commit, maybe a bit less, are contributed by users, and they appear properly attributed (to the original patch author) in the kernel git changelog and in ohloh.

As for SCM changes, care must be taken during the conversion... We started out with CVS, and are currently using mercurial; when the change was made, the whole history was imported to bootstrap the new repo, and the mercurial tree includes the full changelog from the CVS era; if ohloh started tracking the HG repository, it would get the whole history (it won't, since it looking at the kernel tree, but you get the idea)

Would not fly in Europe (0)

Anonymous Coward | more than 6 years ago | (#22506338)

Registering large amounts of personal - almost political - data from a huge number of people without asking their consent would be so much against the European privacy laws. Selling access to that data, or mining it for commercial gain even more so.

Bad measurement, but brilliant move (1)

fedtmule (614169) | more than 6 years ago | (#22506544)

This measurement is not particular good - but what software metrics are?

However it is still a brilliant move, as it will motivate a lot of developers to add projects to Ohloh's database. Developers will add just the projects they have contributed to, so that there ranking will go up.

right tool for A job ? (1)

goarilla (908067) | more than 6 years ago | (#22506690)

this could greatly facilitate finding the proper people to fund OSS development
especially if it's by the right companies (ibm,google,redhat,novell,...)

Re:right tool for A job ? (1)

goarilla (908067) | more than 6 years ago | (#22506734)

i mentthis could greatly facilitate finding the proper people to pay for OSS development

Quantitative metrics dont work on developers (1)

KillerCow (213458) | more than 6 years ago | (#22506752)

Quantitative metrics don't work on developers. As soon as a developer learns what it is, they are smart enough to game the system.

I [commit] can [commit] game [commit] any [commit] system [commit] based [commit] on [commit] commit [commit] counts[commit].[commit]

Numver off bugz fixd es eze 2 gaeme two.

Bug free code and low bug recidivism is easy. [have tester check code before checkin].

Number of projects? Sure. Every possible sub-component now has it's own source tree and project space.

Lines of code? Sure, I can write lots of code. It's one of my favorite things to do. On and on and on I can write code. It's like there doesn't have to be an end to what I say. Lots of productivity here. Oddoles and oodles of work product. Lots of code means lots of productivity, and man, I can sure be productive when I have to be. blah blah blah blah balh....

Open source playing cards to help funding!!! (1)

syousef (465911) | more than 6 years ago | (#22506922)

I can just see the tv ad:
"Wow! Collect yours today"
Then 2 kids in school uniforms
"I'll trade you my RMS for your Linus and Eric S Raymond!"

;fuck? (-1, Troll)

Anonymous Coward | more than 6 years ago | (#22507948)

Another looser... (1)

Yaa 101 (664725) | more than 6 years ago | (#22508038)

That tries to create scarcity of the abundant open source.

I like the dollar figure (1)

RAMMS+EIN (578166) | more than 6 years ago | (#22508992)

I like the dollar figure ohloh attaches to projects. "This is what it would have cost an enterprise to develop this software." It really gives you an appreciation for how much the open source community is giving to the world.

And yet again... (1)

Pseudonym (62607) | more than 6 years ago | (#22510200)

Once more, a coding-related site which lumps C and C++ together into one "C/C++" agglomeration. That's the point where I stopped looking.

And of course, they don't track mercurial (1)

Jonathan S. Shapiro (321593) | more than 6 years ago | (#22510742)

An increasing number of projects are using mercurial these days. Seems like a not-small oversight.

From the front page (1)

nog_lorp (896553) | more than 6 years ago | (#22510808)

Ohloh says

Mozilla Firefox 89 contributors, written in JavaScript.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?