Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

The Futility of Developer Productivity Metrics

samzenpus posted more than 2 years ago | from the measure-of-a-man dept.

Businesses 203

snydeq writes "Fatal Exception's Neil McAllister discusses why code analysis and similar metrics provide little insight into what really makes an effective software development team, in the wake of a new scorecard system employed at IBM. 'Code metrics are fine if all you care about is raw code production. But what happens to all that code once it's written? Do you just ship it and move on? Hardly — in fact, many developers spend far more of their time maintaining code than adding to it. Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities?' McAllister writes. 'Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?'"

cancel ×

203 comments

Sorry! There are no comments related to the filter you selected.

Refactor... (4, Insightful)

Tsingi (870990) | more than 2 years ago | (#38089360)

Refactor, refactor, refactor

KISS technology, nothing beats it.

Re:Refactor... (5, Informative)

ZeroExistenZ (721849) | more than 2 years ago | (#38089660)

Application design (have someone think about everything in broad lines, set out a main architecture-model in ADVANCE.)

Iterative design.

A good project manager prioritizing and communicating current development shielding programmer from clients (who during the "waiting time", are fantasizing changes).

In contrast, in debugging and QA, free way between the client and the developer (more efficient and gives more motivation to the dev as to slave for )

Realistic expectations: don't get your devs running around doing "quick" jobs left and right while they're trying to keep a tight schedule.

Keep everybody current: short standup meeting in the morning

Motivate communication; devs tend to get absorbed in their coding-problem (nd prioritize being "productive") but forget to be a team

The right personality on the right segment of a project: developers have their strengths and their weaknesses.

...

Re:Refactor... (1)

stanlyb (1839382) | more than 2 years ago | (#38089686)

I wish i had mod points man. Just, Keep It Simple Stupid.

Writing for reuse is lacking use case (4, Interesting)

rednip (186217) | more than 2 years ago | (#38089376)

If you don't have a use case for reuse, you shouldn't try to code for it. To many 'interfaces' are single use, see 'servlet' vs. 'http servlet'.

more reasons than just reuse. (5, Interesting)

RingDev (879105) | more than 2 years ago | (#38089756)

Writing for reuse can be excessive, but there are a number of reasons to move in that direction even if you don't intend to reuse that specific code block:

1) Unit Tests. If you abstract your functionality in a way that allows reuse, it also abstracts in for extremely easy unit testing. And unit testing will save you an incredible amount of effort in code maintenance.

2) Consistency. If you follow the same design pattern for all of your abstractions, all of your developers should be familiar with it. This makes it significantly easier for different developers to step into projects as the hopefully don't have to learn another person's style for abstraction.

3) Replacement and isolation. Need to implement a functional change? If your code is abstracted, like it would be for reuse, the functional change is limited to a single block, which is easily identifiable and if you're doing it right, unit-testable.

4) Just in case. Most of the time abstracted code doesn't get reused, and event when it does get reused it's usually a copy and paste job instead of a reference. Even so, if anyone ever does need the same functionality, it allows them to quickly rip off the exact piece they are looking for as opposed to trying to strip out your programs logic to get the tiny bit they want.

-Rick

Re:Writing for reuse is lacking use case (2)

Rob the Bold (788862) | more than 2 years ago | (#38090214)

If you don't have a use case for reuse, you shouldn't try to code for it. To many 'interfaces' are single use, see 'servlet' vs. 'http servlet'.

This is an especially good policy if you're a contractor paid by the hour.

Metrics fail every time (0)

Anonymous Coward | more than 2 years ago | (#38089398)

Holistic coding is the only way to go.

It's tricky (5, Insightful)

LunaticTippy (872397) | more than 2 years ago | (#38089404)

Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

Re:It's tricky (5, Insightful)

abroadwin (1273704) | more than 2 years ago | (#38089504)

Or you could just communicate with your developers, be aware of the work they're doing and judge their performance based on their effective productivity from your perspective. I've heard this called "management" before, but I know that word has been twisted to mean something more sinister as of late.

Re:It's tricky (4, Insightful)

Curunir_wolf (588405) | more than 2 years ago | (#38089672)

Or you could just communicate with your developers, be aware of the work they're doing and judge their performance based on their effective productivity from your perspective. I've heard this called "management" before, but I know that word has been twisted to mean something more sinister as of late.

That won't work, because we laid off all the middle managers years ago. Developers are all exactly the same - we just need to know which ones to slot into the critical path.

Re:It's tricky (1)

elsurexiste (1758620) | more than 2 years ago | (#38089846)

...and judge their performance based on their effective productivity from your perspective.

Your perspective will taint those assessments and assign different values to whoever you dis/like most. That's why we use metrics: they are quantitative and objective.

Re:It's tricky (1)

turbidostato (878842) | more than 2 years ago | (#38089952)

Yes, they do measure something, the problem being "...if interest".

Re:It's tricky (4, Insightful)

marcosdumay (620877) | more than 2 years ago | (#38090496)

Yes, they are objective and quantitative. Managers love that, because being objective excuses them from forming the kind of bias people used to call "leadership", and being quantitative they fit into nice formulas where they can convey no information at all to higher managers that are too insecure to ask what it means.

Just don't expect those quantitative and objective metrics to be correlated with the overall profit of your business.

Re:It's tricky (1)

MichaelKristopeit402 (1978292) | more than 2 years ago | (#38090570)

and how could those metric calculation systems determine if the actions taken by the subject were done to specifically game the metric calculation systems?

one time pads all the way down.

you're an idiot.

who is "we"?

you are NOTHING.

cower some more behind your chosen spanish rhetoric based pseudonym, feeb.

you're completely pathetic.

Re:It's tricky (5, Insightful)

nine-times (778537) | more than 2 years ago | (#38089928)

Yeah, I've come to realize that nobody understands what a manager is supposed to do anymore. Most people's concept is that managers exist to play the part of the PHB, but they don't do anything useful.

More and more, I think no one even understands the value of having a knowledgeable person make difficult decisions based on well thought-out judgments. People want procedures that they can just give to anyone and assume that the outcome will be the same, and then they want metrics that they can use to measure the outcome without knowing anything about the situation.

The proper role of a manager-- aside from particular responsibilities he/she might have on top of this-- is to understand the people working for him and understand his unit's role in the greater context of the company, and then to eliminate obstacles that prevent the people working for him from performing their roles. This might mean protecting the people that work for you from upper management, it might mean recognizing and rewarding good performance, or it might mean all sorts of other things. But to do it right, it takes a lot of work and it takes good judgment.

Re:It's tricky (1)

marcosdumay (620877) | more than 2 years ago | (#38090532)

"More and more, I think no one even understands the value of having a knowledgeable person make difficult decisions based on well thought-out judgments. People want procedures that they can just give to anyone and assume that the outcome will be the same..."

Don't you see several complaints around here that managers treat developers as gears in a box or stuff like that? Do you really think that is restricted to tecnicians?

Re:It's tricky (3, Insightful)

nine-times (778537) | more than 2 years ago | (#38090818)

Yeah, that's my point. As a manager, it's your job to learn about the people who work for you, to help them understand what they should be working on, to help motivate them, to judge their strengths and weaknesses, and to coordinate/arrange things so that their weaknesses don't keep them from doing their job well. Then you have to weed out people who aren't going to do a good job, and meanwhile reward, foster, and develop the employees who have potential and do good work.

That's a lot of work, but that's the work that a manager is supposed to be doing. Too often they shirk that work and instead treat their employees like "gears in a box". Sometimes it's laziness, but a big part of the problem is that our society/culture doesn't recognize the work/judgement of a good manager as valuable. We instead expect people to be interchangeable, which is a problem.

Re:It's tricky (1)

Bert64 (520050) | more than 2 years ago | (#38090342)

Yes that's the only effective way, but it doesn't scale to huge numbers and doesn't fit in with the corporate bean counters who like to see graphs and statistics...

Re:It's tricky (1)

abroadwin (1273704) | more than 2 years ago | (#38090456)

The problem there isn't the performance measurement strategy, it's over-extension of management. If your manager:managed ratio is such that it isn't an effective strategy then you have too few managers for the number of developers you have.

Re:It's tricky (1)

Kjella (173770) | more than 2 years ago | (#38090700)

Yes we've tried entirely subjective management, that's why we're trying to find objective criteria. Nothing like a manager that can set pretty much any performance rating he likes, that would surely not be gamed or abused. Combine that with most everybody thinking they're above average software developers and you got the scene set for feeling unjustly passed up. Not to mention the total lack of transparency, ability to compare with other teams or to measure the performance of the managers. Neither side is exactly total bliss.

Re:It's tricky (5, Insightful)

ackthpt (218170) | more than 2 years ago | (#38089512)

Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

In my book there are only three ways to measure code:

  • For speed
  • For size
  • For readability

Emphasis on any one and the others suffer. They're goal should be on bug-free code which meets a spec. Writing code is like practicing medicine, every patient is different and has its own demands.

Re:It's tricky (0)

Anonymous Coward | more than 2 years ago | (#38089876)

My metric is code separability into mostly independent parts and good interfaces between them. This way, it is easy to fix or change one part without affecting everything else. Speed, size and readability don't matter that much as long as your code is nicely modularised. You can have nice, fast, small and readable code which holds all other parts of your code hostage due to being dependent on too many or the wrong things.

Re:It's tricky (2)

stanlyb (1839382) | more than 2 years ago | (#38089950)

Actually, in my book there is only one feature that i wanna to see in every program/module/library. To.Be.Debugable. If i cannot debug it......sorry, but your code is crap.

Re:It's tricky (3, Insightful)

Rob the Bold (788862) | more than 2 years ago | (#38090096)

Actually, in my book there is only one feature that i wanna to see in every program/module/library. To.Be.Debugable. If i cannot debug it......sorry, but your code is crap.

Unfortunately, if you can't debug it, it's your source code that's crap. Because if you're debugging it, it's yours now . . .

Re:It's tricky (1)

stanlyb (1839382) | more than 2 years ago | (#38090708)

That's why when i try to debug your code (and making it by definition mine), i am using quantum mechanics methods, and the result is that i could tell you that there is a dead cat in your bag, without even opening your bag, and hence, by definition, it is is still your.dead.cat. (now how are you going to beat me....)

Re:It's tricky (0)

Anonymous Coward | more than 2 years ago | (#38090780)

Unfortunately, if you can't debug it, it's your source code that's crap. Because if you're debugging it, it's yours now . . .

Not if you're working in a team, the writer of the crap code is on holiday, and your boss asks you to debug it . . .

Re:It's tricky (1)

Hognoxious (631665) | more than 2 years ago | (#38089926)

In my book there are only three ways to measure code:

        * For speed
        * For size
        * For readability

Perhaps you've got a few pages missing. Usability, functionality, reliability...

Re:It's tricky (1)

ackthpt (218170) | more than 2 years ago | (#38090636)

In my book there are only three ways to measure code:

        * For speed

        * For size

        * For readability

Perhaps you've got a few pages missing. Usability, functionality, reliability...

That's the end product. This is about the actual writing of code. You can apply any or all of those in respect to the end product independently. A coder could write very neat code, fast code, or compact code, which has any kind of interface or performance.

Re:It's tricky (1)

DoofusOfDeath (636671) | more than 2 years ago | (#38090820)

I think you're missing a few:

Modifiability - This captures the extent to which a programmer correctly anticipates future changes that the code will need. It wouldn't be wise to judge a developer on this track record for just a single program, but with larger sample sizes, it could be useful.

Time-to-develop - If you ignore this metric, you're unable to distinguish between a program that gets written in a day vs. the very same program getting written in a year.

Now, you could argue that these two attributes are not purely about the code, but are actually about a code/programmer combination. But I'd say that also holds for your "readability" metric, which is subjective w.r.t. who's doing the reader.

Re:It's tricky (2)

jeffmeden (135043) | more than 2 years ago | (#38089524)

Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

A secret formula like, oh I dunno, customer satisfaction (maybe "how many bugs make it to the customer")? Every piece of software is written for a customer (of some sort) so maybe time could be spent actually thinking about how effective the code is AFTER it's used. Speaking as someone who doesn't write much code, but lives and breathes the consequences of good and bad code every day, I can say with confidence that quality would go up if the only coding "practice" was severe punishment of anyone who introduced/propagated/failed to eliminate bugs in the code they were responsible for.

Re:It's tricky (5, Insightful)

icebraining (1313345) | more than 2 years ago | (#38089616)

So, you want to reward Wally? He probably doesn't introduce much bugs...

Re:It's tricky (1)

ackthpt (218170) | more than 2 years ago | (#38089692)

So, you want to reward Wally? He probably doesn't introduce much bugs...

Neither does he produce. If the PHB weren't so inept he wouldn't be there.

Tell me you've never been in a workplace there didn't have a goldbricker like Wally.

Re:It's tricky (1)

jeffmeden (135043) | more than 2 years ago | (#38089702)

So, you want to reward Wally? He probably doesn't introduce much bugs...

If his projects are rated as satisfactory by management (this "metric" ALWAYS comes first, no matter what the development style is) then sure. Sometimes the zen of knowing what not to do is more effective than having the energy to code 10,000 lines of something that is buggy and ultimately unneeded.

Re:It's tricky (1)

turbidostato (878842) | more than 2 years ago | (#38090038)

Your policy would certainly render quality code... eventually. In the mean time you would be forgetting that nasty issue called "time to market".

Remember: if you are not ashamed of showing your code, you waited too much to promote it.

Re:It's tricky (2)

smelch (1988698) | more than 2 years ago | (#38090728)

It is odd how many code-sharing sessions start with "this is all ugly and I haven't had a chance to clean it up yet but...."

Re:It's tricky (1)

sjames (1099) | more than 2 years ago | (#38089936)

Coming up with a developer value metric is NP hard. The best you can do is use human intuition (AKA, a formula so secret even the user isn't sure what it is) to approximate a reasonable result and then look at metrics to see why the intuitive approximation is/is not valid. It cannot be proceduralized without hand waving.

Note to managers?: You don't WANT this proceduralized or YOUR manager will replace you with a spreadsheet. Watching your former employer tail spin into the water might provide some brief satisfaction, but it won't get you your job back.

Re:It's tricky (2)

c0lo (1497653) | more than 2 years ago | (#38090334)

Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code.

Two things:
1. Your point - which is "Tell me how you measure me and I'll tell you how I'll behave" - with the addition that it's so natural/visceral a reaction that one doesn't even need to intend to game the measurement, the behavior of the measured will alter
2. I'm firmly on the opinion that code is value-spent and is not value-created. From this point of view, seems very strange to me for one to use metrics of cost to measure productivity. Efficiency should be measured in how small the number of lines of code/bug fixes are needed for implementing/maintaining the desired functionality.

Re:It's tricky (1)

Rob the Bold (788862) | more than 2 years ago | (#38090380)

Whatever you measure will be gamed. Measure bugs fixed, and you will find people wasting time listing each tiny variation of a bug. Measure lines of code, you will get spaghetti code. It almost seems better to measure a bunch of things and use a secret formula to determine productivity.

The important thing to do before trying to figure out how to measure something is to decide what it is that you want.

Short answers (3, Insightful)

drb226 (1938360) | more than 2 years ago | (#38089466)

But what happens to all that code once it's written? Do you just ship it and move on? Do your metrics take into account time spent refactoring or documenting existing code? Is it even possible to devise metrics for these activities? Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers? How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly? Can any automated tool measure these kinds of best practices?

It bitrots. Yes. No. Maybe. Probably. Definitely. Possibly.

Re:Short answers (1)

ediron2 (246908) | more than 2 years ago | (#38089888)

Lets see...

  • But what happens to all that code once it's written?
    It Bitrots
  • Do you just ship it and move on?
    yes
  • Do your metrics take into account time spent refactoring or documenting existing code?
    no
  • Is it even possible to devise metrics for these activities?
    maybe
  • Are developers who take time to train and mentor other teams about the latest code changes considered less productive than ones who stay heads-down at their desks and never reach out to their peers?
    probably
  • How about teams that take time at the beginning of a project to coordinate with other teams for code reuse, versus those who charge ahead blindly?
    Definitely
  • Can any automated tool measure these kinds of best practices?
    possibly

Possibly?! More like 'Not Very Damn Likely'. Even peer-ratings metrics geared to get a sense of who on the team is most valued get gamed.

Metrics suck in any job (2)

GeneralTurgidson (2464452) | more than 2 years ago | (#38089470)

Unless it's your job to make up the metrics.

Re:Metrics suck in any job (1)

guybrush3pwood (1579937) | more than 2 years ago | (#38089576)

Bazzinga!

Re:Metrics suck in any job (1)

marcosdumay (620877) | more than 2 years ago | (#38090568)

I guess you were never assinged that kind of job.

Problem (5, Insightful)

bigsexyjoe (581721) | more than 2 years ago | (#38089478)

A lot of problems rating developer productivity. First, if a system is that good, then managers won't be able to game it to play favorites. Second, writing code for future use is always harder than writing code specific to the problem. Third, almost any metric is going to penalize a simpler solution. (Keep in mind that once you see a simple solution it seems obvious and everyone thinks they'd think of it. Fourth, evaluating developers well would require making the best coders managers, and that rarely happens for several reasons.

Re:Problem (2)

snowgirl (978879) | more than 2 years ago | (#38089782)

First, if a system is that good, then managers won't be able to game it to play favorites.

This is the most likely reason I see why no performance metric that works would actually be picked up. The ideal rating system for a corrupt manager is one where everyone is rated poorly, and then you selectively absolve bad performance metrics for anyone you don't want fired.

Of course, one also has to be careful in this situation and not have employees discuss their performance ratings, and especially don't have anyone else in management discuss performance rating absolutions. Think of the horrible legal shit storm you would get into if you rated one employee really poorly in one area, and then later that employee hears in a random meeting from your boss's boss that no one actually met that rating, and so if we held people to that rating, then no one would qualify as a good employee! Be sure once that event has actually happened though to stop recommending that every employee go to these diverse group weekly meetings...

Re:Problem (1)

c0lo (1497653) | more than 2 years ago | (#38090432)

First, if a system is that good, then managers won't be able to game it to play favorites.

This is the most likely reason I see why no performance metric that works would actually be picked up. The ideal rating system for a corrupt manager is one where everyone is rated poorly, and then ...

An ugly world (I know it from my past experience)... I mean the world in which a manager doesn't compete with other managers (on budgets, prestige, etc) based on successes, but is "chosen to be a manager because we need one and... let's see who's the most notable and less-covered-by-shit on the floor".
In the last case, the chosen person will be the most-full-of-shit-and-willing-to-spread it one... you can imagine how the managed team stinks.

Re:Problem (1)

fortapocalypse (1231686) | more than 2 years ago | (#38089862)

The first big problem though is lack of metrics and insight into what developers are doing. I worked on a team and helped implement big brother software on a large development team's computers. The problem was that there was too much data and the company that implemented the software then tried to provide inadequate visualization (e.g. 40% of time was spent on JSP/Java editing, 10% on a web browser, etc.) and just knowing *what* people are working on is not enough. You have to have metrics based on the output, so looking at source control has to play a part. The other metric that is important is customer satisfaction, number of customers, and possibly the effect of features/solutions on the amount of money made. That is the tough part to correlate to the work done. But without metrics, surveys, etc. you have nothing. And as Edie Brickell once said, "There's nothing I hate more than nothing."

Re:Problem (2)

MadKeithV (102058) | more than 2 years ago | (#38090088)

Second, writing code for future use is always harder than writing code specific to the problem. .

It's nearly always wrong, too. YAGNI. You Ain't Gonna Need It.

Re:Problem (1)

Smallpond (221300) | more than 2 years ago | (#38090552)

On a recent project there were two developerss fixing bugs. "A" would fix a bug in 4 hours rewriting 50 lines of code. "B" would fix the a bug in 9 hours rewriting 1 line of code. They let B go because A was more productive. Funny thing is they never ran out of bugs.

Project leader responsibilities (3, Insightful)

Synerg1y (2169962) | more than 2 years ago | (#38089490)

Wouldn't it be the project leader who monitors these on an individual basis? If a coder isn't pulling their weight its up to the project leader to address it up to the point of termination. Above that you have a suit who monitors the project leader's team performance and decides how well the project leader is doing. Of all the places layered management doesn't work, coding is not one of them. It's a challenge to hold a developer accountable because there are so many different approaches to the same problem in coding and a lot have definitive pros and cons.

Pfft (5, Insightful)

Allicorn (175921) | more than 2 years ago | (#38089492)

Can any automated tool measure these kinds of best practices?

No. The - for the sake of politeness, let's call them "people" - who invested time and effort into devising these schemes have actually built a complete chain of negative productivity by doing so. Remarkable.

Measure the objective not the code (4, Insightful)

Crashmarik (635988) | more than 2 years ago | (#38089494)

Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

Re:Measure the objective not the code (2)

RingDev (879105) | more than 2 years ago | (#38089916)

That's really the key to it. Focusing on objectives.

Are outputs matching estimates for time? Are programs being deployed/shipped on schedule? Are bug report rates with in acceptable ranges? etc...

And once you get good at that, start looking for ways to improve on the metrics. Reducing bug counts, getting more accurate estimates, and pushing for shorter (but still reasonable) estimates.

I knew a programmer who once solved a problem that was going to be addressed with a $100,000 inventory management system, with a piece of cardboard and a black magic marker. No code metric could show that performance. But outcome based metrics were all maxed out for that job.

-Rick

Re:Measure the objective not the code (2)

TheRealMindChild (743925) | more than 2 years ago | (#38090056)

It isn't that cut and clear. I can write a bug-free one line perl solution, but what happens when someone needs to maintain it? Objectives come from all directions. Your shop wants quick turn around and maximum profit. The customer wants quick turn around , solid code, AND a cheap price. Your goal is to be productive and get home before dinner. Notice the conflicting objectives.

Re:Measure the objective not the code (1)

Bucky24 (1943328) | more than 2 years ago | (#38090276)

I can write a bug-free one line perl solution, but what happens when someone needs to maintain it?

If it's in perl? You sacrifice to the voodoo gods whenever you need to maintain it, of course.

Re:Measure the objective not the code (1)

murdocj (543661) | more than 2 years ago | (#38090420)

Perl is write-only. You ask someone what the code is supposed to do and rewrite it.

Re:Measure the objective not the code (1)

MadKeithV (102058) | more than 2 years ago | (#38090172)

Are outputs matching estimates for time? Are programs being deployed/shipped on schedule? Are bug report rates with in acceptable ranges? etc...

There is one metric that matters: Is it making money? If you want to get fancy, add Could it be making more money? Everything else is window dressing. (Assuming of course that you're not working in a "cost center" development department, and taking into account regulatory affairs)

Re:Measure the objective not the code (5, Insightful)

MadKeithV (102058) | more than 2 years ago | (#38090140)

Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

I am also REALLY happy to have "that guy" that has absolutely shit productivity, but somehow manages to pick up on every time a "solution" is proposed by the rest of the team to a problem that doesn't exist or doesn't matter, and stops THEM from being really efficient at doing the wrong thing.
I'm also really happy to have "that girl" that doesn't seem to really be doing anything, but take her out of the team and everyone else starts floundering because she's actually constantly helping them be a lot more productive.
"Meeting the objective" is actually potentially just as bad as any other metric, because it depends on how you define the objective, and meeting it. What the customer asked? What the customer wanted? Or what the customer actually needed?

Re:Measure the objective not the code (3, Informative)

Crashmarik (635988) | more than 2 years ago | (#38090726)

Personally I am always happy with the guy who can get things done with one line of code instead of a hundred, but what I really care about is that objective is met and we don't have a host of bugs that require 10 times the cost of the development just to maintain. Its not hard stuff but it does require common sense and a hard nosed attitude both of which can be scarce commodities these days.

I am also REALLY happy to have "that guy" that has absolutely shit productivity, but somehow manages to pick up on every time a "solution" is proposed by the rest of the team to a problem that doesn't exist or doesn't matter, and stops THEM from being really efficient at doing the wrong thing.
  I'm also really happy to have "that girl" that doesn't seem to really be doing anything, but take her out of the team and everyone else starts floundering because she's actually constantly helping them be a lot more productive.

"Meeting the objective" is actually potentially just as bad as any other metric, because it depends on how you define the objective, and meeting it. What the customer asked? What the customer wanted? Or what the customer actually needed?

That guy and that girl are generally called project/team leaders. Don't fret, You raised an important issue. The guy you don't want on the team is the one that comes up with ridiculous edge cases and is needlessly obtuse. Like someone who invents coders that are doing the work of managers and senior managers then complains a measurement tool doesn't capture their contribution, or goes into a recursive loop trying to figure out what the objective should be.

Developers will cook the books anyway (3, Insightful)

Anonymous Coward | more than 2 years ago | (#38089506)

If they know their productivity is being measured, it becomes a contest to see who can cook the books the best anyway.

Complex things can be measured (1)

davidannis (939047) | more than 2 years ago | (#38089522)

The idea that complex things can not be measured is constantly thrown up by professionals who don't want to be measured unfairly or just don't want to be measured at all. However, doctors, teachers, and programmers can all have their output evaluated. I know that there is more to evaluating a doctor than survival rate and how often he remembers to wash hands between patients, but I know that hospitals that try to measure and improve doctors performance do a better job of helping patients. Reviews by peers and management are a good place to start. Yes, that can devolve into a popularity contest or a blame game but good management can guard against that. When I ran a software company we'd have meeting where we reviewed and discussed sections of code as a learning tool as a small part of our QA process. The end result was better code and a more educated, engaged programming staff. When you combine subjective measures like these with easily quantifiable measures you can get a good idea of how competent a programmer is.

Re:Complex things can be measured (2)

obsess5 (719497) | more than 2 years ago | (#38089592)

But that's the whole question, isn't it? What are the *relevant* "easily quantifiable measures"?

Re:Complex things can be measured (1)

marcosdumay (620877) | more than 2 years ago | (#38090680)

No doubt you can measure quality of code or the competency of a programmer. But can you stablish an objective metric with fast results (remember, corportations are "this quarter" oriented) that gets it right?

We need metric metrics (5, Funny)

naroom (1560139) | more than 2 years ago | (#38089544)

metrics provide little insight

If only we had some kind of.... metric metric.

Re:We need metric metrics (1)

Jah-Wren Ryel (80510) | more than 2 years ago | (#38089754)

metrics provide little insight

If only we had some kind of.... metric metric.

Sounds like ISO9000.

Re:We need metric metrics (0)

Anonymous Coward | more than 2 years ago | (#38090228)

If only we had some kind of.... metric metric.

I prefer my metrics to be ANSI metrics the way the supreme being intended, thank you very much.

Re:We need metric metrics (1)

Hognoxious (631665) | more than 2 years ago | (#38090452)

By analogy with metadata - data about data - "metametrics" was what sprang to my mind. And like most things that do after I've had a few jars, it a) already exists [mac.com] and b) is far too shit to deserve such a goshdarn clever name.

What about non-coding time? (2)

slapout (93640) | more than 2 years ago | (#38089548)

What about all the time you spend not coding: meetings, documentation, training users on how to use the system, working out the design before you start coding, answering emails, sending status reports, filling out time sheets, coordinating work with other developers, coordinating things with others in IT so your program will have a server to run from, being the go-between for the IT server team and the customer when the server goes down, creating database layouts and writing SQL?

Re:What about non-coding time? (1)

lwsimon (724555) | more than 2 years ago | (#38089758)

Surely writing SQL qualifies as coding.

Re:What about non-coding time? (2)

EricWright (16803) | more than 2 years ago | (#38090300)

Not according to some of the developers around here. Of course, these are the types that put "select * from table" into their Java code, then try to filter data in the application layer. Then they whine that their application is slow.

Seriously.

Re:What about non-coding time? (2)

lwsimon (724555) | more than 2 years ago | (#38090596)

Well, that's just ignorant, then.

I write SQL most of the day, and in my opinion, it's more difficult to do well than traditional programming. If I'm writing a webapp somewhere, I only have to keep the individual logic i'm working on in my head - with SQL, I may have to know how a single dataset is processed over a half dozen steps to get the result I want.

Re:What about non-coding time? (0)

Anonymous Coward | more than 2 years ago | (#38089778)

Funnily enough, I'm just introducing the ratio of "time spent on stuff that directly contributes to the customer solution:time spent on anything else" as a metric. Or, simpler: Customer Value Add:non Customer Add time

Some of the things you list belong in the first number; some in the second:
* meetings - Depends on the content, but presume non-CVA
* documentation - Assuming it's for a purpose and not just self-justifying: CVA
* training users on how to use the system - CVA
* working out the design before you start coding - CVA
* answering emails - Non CVA
* sending status reports - Non CVA
* filling out time sheets - Non CVA
* coordinating work with other developers - non CVA (ideal: being co-ordinated without spending effort on it)
* coordinating things with others in IT so your program will have a server to run from - non-CVA beyond a small amount
* being the go-between for the IT server team and the customer when the server goes down - non CVA. Better not have it go down.
* creating database layouts - CVA
* writing SQL - probably CVA unless involved in defect analysis & fix. Better to have no defects...

It's not a metric of the individual developers, but of the project they work in.

Re:What about non-coding time? (1)

Surt (22457) | more than 2 years ago | (#38090360)

Design is about as clearly non-CVA as it comes. Implementation is CVA. This is pretty much the whole philosophy behind agile.

Re:What about non-coding time? (0)

Anonymous Coward | more than 2 years ago | (#38090198)

Don't forget screwing around on Slashdot.

lazy management (5, Insightful)

J-1000 (869558) | more than 2 years ago | (#38089602)

This all comes down to lazy, gutless management. Why take the time to get to know Dave and monitor the quality of Dave's work when we can just look at a spreadsheet at the end of the month? Managers prefer to tinker with automatic analysis software rather than manage.

Which is more fun, getting a better handle on what Dave is doing, or researching fancy new software tools that might get you all sorts of praise from metric-craving executives?

Dave's job, which was once about creating a quality product, now shifts to merely satisfying the metrics.

Re:lazy management (0)

Anonymous Coward | more than 2 years ago | (#38089656)

>Dave's job, which was once about creating a quality product, now shifts to merely satisfying the metrics.

If Dave were the only engineer on it, then I'd agree.

If it's a complex embedded realtime system with 200 engineers working on it, then no.

Re:lazy management (1)

sjames (1099) | more than 2 years ago | (#38090260)

What applies to Dave applies to Bill, Jane, etc just as much. Now they're all more interested in looking good on the metric of the day than on producing a decent product. Any who make the mistake of focusing on the product will be eliminated systematically by the metric.

Re:lazy management (1)

asylumx (881307) | more than 2 years ago | (#38090244)

I hope it's different where you work, but where I work, the management is lucky if they are truly 100% management. But in reality, the managers of technical teams also have to play the role of project manager, technical lead, and sometimes architect. Yes, it sucks, and I wish it weren't that way, but trying to change that is like pulling teeth.

So even if Mr. Manager has good intentions, he really is prevented by other duties from doing his job to the fullest extent. It sucks, but I wouldn't feel right calling that person gutless for it and pretending they are a less than useful person.

Coding is an art (0)

Anonymous Coward | more than 2 years ago | (#38089684)

Coding is somewhere between art and engineering. You can hardly measure art.

Re:Coding is an art (4, Insightful)

stanlyb (1839382) | more than 2 years ago | (#38089824)

And every developer starts as an engineer, and ends as an artist. As simple as that. The engineer can do anything, but have not done it yet. The artist knows what to NOT DO, because he has already done it, and does not wanna to repeat all the teenage mistakes again. The artist just gives you the solution. End of story.

Re:Coding is an art (1)

HornWumpus (783565) | more than 2 years ago | (#38090266)

Engineering is the union of applied science, business and art.

Coding is just engineering. Most coders are bad engineers though. Still an immature field.

Repeatability (4, Insightful)

3count (1039602) | more than 2 years ago | (#38089700)

Metrics are valuable if you do the same thing repeatedly. If you build a new building that is like the previous one, you can collect metrics and compare your performance against history. If you write the same search algorithm again and again, you can collect metrics and compare to see how your performance changes over time. Of course, with software, you never repeat. Somewhere around the third time, you move it into some form of library, reuse it, and start on a fresh problem. Perhaps metrics are helpful in some situations, such if your team keeps repeating the same mistakes, you might find similarity in those mistakes (code smells.) There are plenty of people working on these problems and tools. But, from a management point of view, if you keep doing the same thing, you are doing it wrong, and code metrics are not going to help much.

Why are metrics so damn important (4, Insightful)

Riceballsan (816702) | more than 2 years ago | (#38089716)

I don't get the concept of everything needing to be quantified. Does the team accomplish what the goals of having the team are? Does it get developed in a fair timeframe? Is everyone on the team pulling their own weight, or are there complaints of someone slacking off? In the end if the product works then the team is doing well, if the product isn't there should be at least one hybrid manager/coder that actually works with the team members sees who is committing what and can tell off the bat if there is or isn't a weak link dragging the rest down. Actually putting a pen and paper number on a complex project is silly. Do authors get judged by the number of pages they write in a day, no they get paid by the success or failure of the book. You can't judge by the number of lines of code, bugs per line ratio or anything like that, because it is all subjective and has little to no bearing on the end product.

Re:Why are metrics so damn important (0)

Anonymous Coward | more than 2 years ago | (#38089882)

Management loves metrics because "if I can measure it, I can manage it." It gives them something to point to and say "fix this!"
It's also a CYA for management. If the metrics are wrong, it's a problem with the tool and not with the manager.

Re:Why are metrics so damn important (1)

thrich81 (1357561) | more than 2 years ago | (#38090306)

The most important reason to attempt to quantify the effort which goes into a project is so that you can predict the effort which the next project will require, and have solid numbers to back up your predictions rather than just someone's vague feelings or unquantified remembrances of previous projects. This is true of any engineering project. A second and related reason is to identify efforts in the project which don't seem to justify their costs by their results. This quantification doesn't help the current project as much as it helps the next one. I agree that the measures of "number of lines of code, bugs per line ratio or anything like that" don't work very well but something has to be tried or we are just reduced to art (authors, etc), not engineering, with no predictable product, end date, or cost.

Re:Why are metrics so damn important (1)

Maximum Prophet (716608) | more than 2 years ago | (#38090324)

... Do authors get judged by the number of pages they write in a day, no they get paid by the success or failure of the book. You can't judge by the number of lines of code, bugs per line ratio or anything like that, because it is all subjective and has little to no bearing on the end product.

Many successful novels, started out as magazine serials, which were paid by the word. Usually, though, they are edited before they are put into book form.

I've seen a bunch of movies, where I thought, "This would be a great movie, if they just cut half of it, then sped the result up by 1.5x"

I've often thought that coders should be paid by how *little* they write, as long as it does the job.

Hmm (2)

Hognoxious (631665) | more than 2 years ago | (#38089732)

Not everything that matters can be measured; not everything that can be measured matters.

-- Einstein (or maybe it was Franklin)
 

Re:Hmm (0)

Anonymous Coward | more than 2 years ago | (#38090270)

Measuring programming progress by lines of code is like measuring aircraft
building progress by weight.

-- Bill Gates

Do things right vs. doing the right things (0)

Anonymous Coward | more than 2 years ago | (#38089738)

While people can certainly game any system that measures how productive they are by analyzing the quality of their code; it certainly doesn't understand whether or not that piece of code actually needs to exist, or that feature, etc. For example, you may construct an office building which is 20 miles from anywhere does it really serve a purpose, should it have been built? The building meets all the criteria for a good building and is functional, has structural integrity, etc, but it's in the middle of nowhere. Software is elastic in that the original code may be great, but overtime the quality may degrade, or the additional code may have solid quality, but the cohesiveness of the code feel is lost. Indeed the soul of the coding becomes convoluted and becomes difficult to maintain given the various signatures that people leave in their wake. Show me software that can measure that.

This is BS (2)

fortapocalypse (1231686) | more than 2 years ago | (#38089752)

Measuring developer output/metrics effectively is a tough problem. Developers could solve it, but if they do, then they have to both change the way that they work and possibly work harder. Developers are smart enough to know that the metrics will be misused, even if the logic used to produce them is valid. Therefore, any solution will be ridiculed by the development community as insufficient, but the degree to which it is ridiculed will lessen as the solution improves. A solution though, is inevitable if development continues.

Re:This is BS (2)

DalDei (1032670) | more than 2 years ago | (#38089920)

BS

Re:This is BS (1)

fortapocalypse (1231686) | more than 2 years ago | (#38090084)

That's what I said, but you said it more concisely. If this were code, your solution would probably have rated higher than mine. But, after many (too many) years as a developer now, I can say that programming is about as much an art as carpentry. You can do beautiful things with both, but in the end it is about the artifacts and how they are used. They *are* measurable, and providing decent metrics for software development *is* possible. The biggest problem is that we don't want to admit that it is just a really difficult problem; instead we give up. And giving up is what is BS.

Example, please! (1)

Krachmanikov (670121) | more than 2 years ago | (#38089850)

Can somebody give an example of that IBM score card, please? Is it just another Balanced Score Card system, just adapted to software development? The article misses to give that information.

Re:Example, please! (1)

fortapocalypse (1231686) | more than 2 years ago | (#38090136)

Here's one [ifi.lmu.de] .

If you need metrics to know who your stars are ... (2)

Surt (22457) | more than 2 years ago | (#38090128)

... you are already doomed. You've gone so far down the wrong path there's no hope of recovery.

How much time is used on this tps report like BS? (1)

Joe_Dragon (2206452) | more than 2 years ago | (#38090382)

This sounds like some PHB how only knows how to be manger wanting numbers and having no idea about how developing works.

What if productivity were easily measured? (0)

Anonymous Coward | more than 2 years ago | (#38090458)

I suspect that if productivity measurement were so easy it could be automated, that programming would be "done" as a profession.

Take a look at jobs where productivity measurement is easily automated, and you get assembly line worker, fruit picker, banker. These are all jobs for people that aren't very smart. :)

Seriously (1)

luis_a_espinal (1810296) | more than 2 years ago | (#38090604)

Code metrics are fine if all you care about is raw code production. But what happens to all that code once it's written?

Seriously, does the author has to ask? Or anyone that does software development for that matter? Sadly, apparently they have to ask.

You never get to a point where you can say "code once it's written". Code changes after deployment, there are new releases, bug fixes, shit like that which crops up in real life.

So assuming you are gathering metrics the smart way and in a manner that your particular project and organization, then, you simply keep gathering metrics as you re-factor your code, add new features and fix new bugs.

At the very least once should be running McCabe, SLOC metrics (and LCOM metrics when doing OOP), # of open bugs per component and per release version and man-hours involved for a particular project/component/release. Identify and sort components by the # of open issues (fixed bugs, unresolved bugs, partially implemented features) and see where there is a relation between these issues and any or all of these metrics.

Project/department/solution stack specific metrics, when done intelligently serve as structural red flags. They help prioritize.

You can track which components have the highest McCabe metric, for example. Though you should always refactor, metrics like this can flag which part of the software needs more urgent refactoring, that's another example. You can keep track of the # of SLOC changes (additions, modifications and deletions) in a component from release to release (a measure of entropy), as well as the # of bugs detected (also from release to release).

Then you compute a ratio of entropy/bugs detected per release. And if you find that the ratio of your last release (named A) is considerably smaller to the previous ratio (named B), then you can take that as an indication that there might be B-A undetected bugs in this release. And if after a certain number of releases, the ratio is still small, then you can take that as an indication that your quality control has improved.

Things like that are what you do with code during development and during deployment and maintenance. It is never over with software. Until you completely decommission a software system, code is never over.

Why any coding metric is futile (1)

MarkH (8415) | more than 2 years ago | (#38090730)

I got first job doing COBOL back in 1995. Second code change I did I got called into managers office.

He asked how I felt about change. I stated it was easy being only 2 line change to 1 module.

That change saves us 2 million ukp a year he says.

Ever since tried to write as little as possible . Amazing what you can do when you understand the problem , understand the code and find the one change that does it.

Not sure my method of wandering through code, chatting to everyone and then submitting a 4 line commit would fit with IBM

Measure Effectiveness, not Productivity (3, Interesting)

TheWoozle (984500) | more than 2 years ago | (#38090806)

I think the idea of "productivity" is a hold-over of the Industrial Revolution that does not pertain to many of today's jobs; jobs where the unit of work is hard to define, and ultimately irrelevant. Are you telling me you pick your doctor by how many patients he can see in a day? Probably quite the opposite!

In terms of software development, I find that the *effectiveness* of a developer is more important, where effectiveness considers the following (not an exhaustive list):
      - Appropriateness of solution
      - Thoroughness of implementation (logging, exception handling, graceful failure, input validation, etc.)
      - Well-written, parsimonious code that is easy to read and descriptive of what it does
      - Works right the first time, no kickbacks from QA or end user

Give me someone who is effective but slow over someone who craps out junk quickly any day of the week and twice on Sunday! In the end, I don't care about productivity metrics, I care that the end users get a useful piece of software that does what they need with a minimum of headaches.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>