Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Researchers Test Developer Biometrics To Predict Buggy Code

Soulskill posted about 2 months ago | from the subject-was-asleep-when-this-code-was-checked-in dept.

Bug 89

rjmarvin writes: Microsoft Research is testing a new method for predicting errors and bugs while developers write code: biometrics. By measuring a developer's eye movements, physical and mental characteristics as they code, the researchers tracked alertness and stress levels to predict the difficulty of a given task with respect to the coder's abilities. In a paper entitled "Using Psycho-Physiological Measures to Assess Task Difficulty in Software Development," the researchers summarized how they strapped an eye tracker, an electrodermal sensor and an EEG sensor to 15 developers as they programmed for various tasks. Biometrics predicted task difficulty for a new developer 64.99% of the time. For a subsequent tasks with the same developer, the researchers found biometrics to be 84.38% accurate. They suggest using the information to mark places in code that developers find particularly difficult, and then reviewing or refactoring those sections later.

cancel ×

89 comments

Sorry! There are no comments related to the filter you selected.

Why is it always developers? (2)

i kan reed (749298) | about 2 months ago | (#47509659)

Every time I hear about a terrifyingly invasive means of "improving performance" its targeted at developers. Is it just selection bias, or does the world actually hate us?

Re:Why is it always developers? (3, Insightful)

dave562 (969951) | about 2 months ago | (#47509711)

The world hates putting up with buggy code.

Re:Why is it always developers? (1)

dave562 (969951) | about 2 months ago | (#47509765)

On a more serious note, a single developer mistake can potentially affect millions of end users (in the case of an application like Windows). Therefore it makes sense to focus on the developers. "With great power comes great responsibility" and all that.

Re:Why is it always developers? (4, Interesting)

K. S. Kyosuke (729550) | about 2 months ago | (#47509807)

And what about managers who steer the development effort in a direction highly likely to produce buggy code, those won't get measured?

Re:Why is it always developers? (1)

dave562 (969951) | about 2 months ago | (#47509915)

Of course they get measured. In the long term if they deliver too many screwed up projects, their superiors stop giving them projects.

Ultimately it is the developer's responsibility to push back against stupid managers and give them honest feedback about what can and cannot be done.

Re:Why is it always developers? (1)

Anonymous Coward | about 2 months ago | (#47510197)

I would like to know where the entrance is located to this magically meritocratic land you speak of, It is obviously not Earth.

Re:Why is it always developers? (2)

Medievalist (16032) | about 2 months ago | (#47510321)

And what about managers who steer the development effort in a direction highly likely to produce buggy code, those won't get measured?

Of course they get measured. In the long term if they deliver too many screwed up projects, their superiors stop giving them projects.

Ultimately it is the developer's responsibility to push back against stupid managers and give them honest feedback about what can and cannot be done.

I would like to know where the entrance is located to this magically meritocratic land you speak of, It is obviously not Earth.

You'll need an invisible hand to turn the invisible doorknob.

Re:Why is it always developers? (1)

gstoddart (321705) | about 2 months ago | (#47523175)

Of course they get measured. In the long term if they deliver too many screwed up projects, their superiors stop giving them projects.

Sadly, in my experience, they blame it on their team, and get promoted.

Bad managers are surprisingly good at making it look like someone else's fault.

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47509929)

The next step is to find commonality between stressed developers and identify the cause.
After that we will figure out why those managers put pressure on developers.
Eventually we will realize that share prices don't really matter and true happiness is achieved by living the life of an ascetic, pretty close to what developers already do.

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47513195)

Of course they get measured. For example, article about Stephen Elop [wikipedia.org] says that his time as CEO resulted in "Nokia profits fell 92% from 2.4 Billion Euros per year to 188 Million Euros per year." Nokia's "stock value dropped by 85% since Elop's takeover". So there, proof that it's tracked with hard numbers.

This man was famous for writing a particularly damaging "burning platform" memo that hurt Nokia tremendously. Nokia shed 11,000 jobs under his tenure.

This man is currently the Executive Vice President of Microsoft's Devices & Services.

The shocking results, which were absolutely impossible to predict that something like this might happen: Microsoft Just Laid Off Thousands of Employees With a Hilariously Bad Memo [nymag.com] .

So, yeah, the stuff gets measured. Major disasters that are reported on internationally may not be CEMs (career-ending moves), as this case proves, but you may sleep peacefully tonight knowing that yes, this stuff does get measured.

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47509713)

Flash--reason enough to hate developers.

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47509727)

It's because software sucks, and no one has any real idea what to do about it.
The beatings will continue until morale improves!

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47509871)

We know what to do about it, but PHBs decided it is too expensive. We can produce code, that is proofed (in the mathematical meaning of this word) to be correct, but it is certainly way more expensive than to blindly code and hope.

Re:Why is it always developers? (2)

bobbied (2522392) | about 2 months ago | (#47509877)

It's because software sucks, and no one has any real idea what to do about it.

You are more right than you know. Where writing software is a skill that most can develop, the really good developers are more a cross between engineers and artists. They are more like architects, where the form and function are both of high importance because having software that "works" (in that it does everything required [engineering]) and having software that is "workable" (in that it is easy to use [artist]) are worlds apart. Finding developers that do both engineering and art is rare.

It's not just the GUI interface, but ANY "interface" that needs to be usable, functional, simple to understand and complete. Designing an interface that works is easy, making it functional and simple to understand is much harder, and then making sure it is complete (does enough, but not too much to make it complex) is the real art.

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47511335)

Where writing software is a skill that most can develop

But a grand majority of people would write really poor software. Writing *a* piece of software is easy; anyone can learn by rote how to write a "hello world" program. But writing good software is 100% different. Even most current developers are absolute garbage, let alone if you were to bring in most of the population; that would be a nightmare! Most people simply aren't intelligent enough to do something that requires a lot of logic and critical thinking skills well, because they lack those things.

Developers as novelists (1)

handy_vandal (606174) | about 2 months ago | (#47515321)

... really good developers are more a cross between engineers and artists.

Agreed.

When talking with non-developers about developers, I use the simile that developers are like novelists, who work out stories in their heads, and commit those stories to paper.

A novel contains a set of symbols which, taken collectively, and written correctly, form an impressive body of knowledge that can change the world. (Tolstoy's "War and Peace" is my usual example.)

But if the symbols are faulty -- if the book is badly written, if the grammar and spelling are faulty -- then the book will fail to sell, fail to make its point, fail to change the world.

who the hell wrote this crap?!! (5, Funny)

Thud457 (234763) | about 2 months ago | (#47509987)

well duh, that vein on my forehead starts to throb and my eye starts to twitch when I read dumb code.

oh wait, that was my commit

Re:who the hell wrote this crap?!! (1)

DutchUncle (826473) | about 2 months ago | (#47510583)

Mod parent up, from personal experience. ;-(

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47509729)

Marketing pointed at the management, management pointed to the IT group, the IT group reminded management that they don't actually write the software that marketing was selling, so everyone decided to point at the developers.

Re:Why is it always developers? (5, Insightful)

netsavior (627338) | about 2 months ago | (#47509821)

because the average judge/jury/CEO/consumer/manager has no idea how to write code.

They can understand how a toilet is cleaned, how a sale is made, how a 1099 is filled out, how a fire drill works, how a sandwich is put together, how oil is changed, etc... but Coding might as well be a dark art.

Developers are part of a very narrow segment which has no reliable Key Performance Indicators. [wikipedia.org]
Part of that is developers are smart enough to game any system, because they can think in algorithms.

Want to track productivity on Lines of code? Fine, Developers can do NO WORK, and produce TONS of code
Want to track productivity on Number of defects introduced? Fine, doing NO WORK is the baseline for perfect.
Want to track productivity on Number of defects fixed? Fine, through the magic of hand wavery, defects can be found and fixed with no actual work happening

Compare that to well-defined Key Performance Indicators for sales... Bring in X dollars of sales, your performance is X.

CEOs HATE things they cannot measure... which means CEOs are a natural enemy of Developers.

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47510017)

It would be nice to replace code monkeys with code generator (programs), so that architects can just design the program and move on. KPIs are easier to measure here. We are already moving in this direction for many decades now (punchcards -> asm -> functional -> OO) and with Windows Workflow Foundation, etc. the goal is nigh.
Optimization specialists will take control from there and will have easy to measure KPIs as well.

Re:Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47510229)

"...and with Windows Workflow Foundation, etc. the goal is nigh."

AHAHAHAHHAHAHA, Sorry I usually don't respond with laughter, but that was something we'd hear from a "Microsoft Evangelist". It was dumb enough at least.

Re:Why is it always developers? (2)

david_thornley (598059) | about 2 months ago | (#47516683)

You do realize that we've done that over and over. We keep coming up with programs that do code generation. We introduced automatic programmers that represented machine codes with mnemonics and kept track of memory locations (these are now called assemblers). We introduced programs that would take scientific calculations (FORTRAN) and business logic (COBOL). We introduced "fourth generation languages" in the 1980s. All of these, and many more, were honest attempts to remove the need for programmers. They made life easier for programmers, but failed at removing the need for programmers. Windows Workflow Foundation isn't going to work any better.

There is a step in any programming process where somebody has to translate ambiguous specifications into some sort of formal, precise, notation. Once we have the formal, precise, notation, we can have programs take it the rest of the way. We need an actual human to do that translation, though, and that human is a programmer no matter what title or position. If architects can create formal specs, they're programming. If it takes code monkeys, they're programming.

Re:Why is it always developers? (1)

Anonymous Coward | about 2 months ago | (#47510585)

Your simplification of gaming the system can be applied to sales people too. It is not unheard of how some product manufacturers sell stuff to distribution channels and count it as sold, despite living in a warehouse or shop. Or how you can be an excellent talker and make people pay more for stuff they don't need (oh, this phone? no, you want the 5" screen!), but this can have detrimental effects, maybe the user pays more, feels cheated, and never comes to buy from the brand despite other products matching them perfectly.

Can you measure somehow lost future sales (haha, only if you are from the RIAA)? Well, that's how you measure future reduced bug defect rates due to say improvements to development testing quality and control. Which means: you don't. You put people who hopefully do the right thing and the right thing comes out. But it means trusting the people you put in charge. And you can't measure that either, or the effect they produce.

Unfortunately the world is ran by bean counters AT EVERY LEVEL. And that's why we can't have nice things, because they can only measure profit from simple tangible things. Now ask them how much a brand is worth and watch their head explode in fireworks.

Re:Why is it always developers? (1)

david_thornley (598059) | about 2 months ago | (#47516773)

I am thinking of a company* that sold customizable software to large customers who demanded the ability to customize. It also sold new features, which would have to be created by the development staff. At one point, the sales and development staff wound up in different cities, to the technical people weren't able to sit on the overenthusiastic sales people.

The sales people found that they could sell new features, and (a) it made it easier to close the sale, and (b) custom development cost money, so it raised the dollar amount of the sale. Therefore, by overselling custom features, a salescritter could increase not only the probability of a big commission but the size of the commission. The development staff had different ideas about getting flooded by requests for custom features, in that it destroyed the ability to move the main product forward in any coherent fashion, and resulted in large delays in actually shipping and (this was important) getting paid. This continued until top management stepped in and did something that actually made sense in the situation (imagine my surprise).

Smart people will know how to game the reward system, no matter in what field.

*I would like** to formally announce that this is not a company I worked at and signed an agreement not to talk about.

**For reasons I'm not discussing here.

Re:Why is it always developers? (1)

Zero__Kelvin (151819) | about 2 months ago | (#47511023)

You are assuming a mutual exclusion that doesn't exist,as well as a knowledge of the methodology used. If you apply all three, none of those workarounds work. Then, one could also do "blind metrics analysis", where different methods of analysis are used during different, and secret / unannounced windows of time.

You see, as smart as the typical developer thinks he is, it doesn't take much for someone with real skills to bury them ... deep.

Programming CAN be judged (2)

elgatozorbas (783538) | about 2 months ago | (#47511419)

They can understand how a toilet is cleaned, how a sale is made, how a 1099 is filled out, how a fire drill works, how a sandwich is put together, how oil is changed, etc... but Coding might as well be a dark art.

Disclaimer: I am in hardware myself and may completely miss the point here. However, our software/firmware folks do agile programming involving dividing programming problems into pieces which are assigned to programmers, followed-up on large whiteboards and being daily discussed in "scrum meetings" etc. (I may be confusing some concepts here but that is of less importance). The point being that your statement, that programming is some sort of unique dark-art-which-cannot-be-measured-by-managers, appears untrue to me and, honestly, rather pedantic. What these guys are doing is quite measurable. Maybe not by a silly measure like "lines of code", but by the measure of number of problems being solved, having a complexity that apparently everyone of them agrees on.

Indeed, the CEO doesn't know the exact details of how this works, but neither does he personally count the number of cleaned toilets.

Re: Why is it always developers? (0)

Anonymous Coward | about 2 months ago | (#47512989)

Measuring sales dollars is like measuring lines of code. You can sell projects that have no change to bring profit. You can sell stuff by cheating and lose a customer.

And how do you measure management? Imagine same manager with different teams and you get different results if the manager does nothing. Things become even more complex when manager does something.

Re:Why is it always developers? (2)

znrt (2424692) | about 2 months ago | (#47510013)

i hit 50 last sunday and been a developer since i can remember, and i still love my profession but the "guild" has changed an awful lot, from once being a peculiar bunch to the herd it mostly is today. most of my colleagues are much younger than me and ... what can i say ... they are often so brainwashed with corporate bullshit they break my heart almost daily. holy shit, they even blog about it! it's so depressing, it makes me wanna cry. :'(

sign'o'times, i guess. i can perfectly believe many of those sheep will cheerfully allow you to strap an eyetracker on them to check their nominal productivity.

http://www.picturesnew.com/med... [picturesnew.com]

Re:Why is it always developers? (1)

turp182 (1020263) | about 2 months ago | (#47512433)

I'm 40 now. I remember the late 1990s when I was young, as was everyone around me, and at a non-public facing reinsurance company, we had extra staff just doing pie-in-the-sky stuff no one was ever going to see. We got a lot done via inherent competence, I realize now that we were lucky, and we had budget.

In the early 2000s I led the design and development of a SOA rewrite of an existing VB6 app. We had an iDeisgn consultant come in for a week to get us started which was invaluable; but it was through luck, intuition, and a great team that we were successful.

Now I'm on an architecture/strategy path with good leadership (time for training, sandboxing, prototyping). I now realize that every project needs a seasoned lead for component/interface design, UX/unit testing, hardcore analysis, and general direction setting (as well as a solid QA team to find problems). Skimp on any of that, on a medium/large project, and luck is the deciding factor to success.

Oh, and learn the business. Send ideas to upper management. Really think about the business. It can be interesting, and good for the career.

Re:Why is it always developers? (2)

fuzzyfuzzyfungus (1223518) | about 2 months ago | (#47510065)

Every time I hear about a terrifyingly invasive means of "improving performance" its targeted at developers. Is it just selection bias, or does the world actually hate us?

Mostly because they are a newer profession and a trickier one to quantify.

Time and motion studies, along with 'scientific management' were already a serious hit in terrifyingly invasive performance enhancement for blue collar labor around the turn of the 20th century(Taylor and the Gilbreths being the poster children, with many successors). The workers who haven't been replaced by robots yet are likely still subject to a descendant of it. Though less amenable to automation, service sector jobs are also rationalized more or less as tightly as available technique allows.

Software development is still a work in progress because it only started existing comparatively recently and because it takes more technology to dismiss any "Oh, what we do here is unquantifiable skilled craftsmanship" positions.

It is selection bias, in that you apparently haven't heard of it happening to basically everyone it can reach; but the world does actually hate you, and is actively working on making software development absolutely as soul crushing as seems economically desirable.

Time for an for IT Union (0)

Anonymous Coward | about 2 months ago | (#47510485)

All of this metrics do not get to the real issues and some cases lead to people to gaming the system / people just doing the minimum not get fired or have there 6 bosses tell them about each time they mess up.

Also Scope creep, deadlines, and more lead to errors

Re:Time for an for IT Union (1)

i kan reed (749298) | about 2 months ago | (#47510541)

I'm truly sorry, but an IT union isn't happening, until at least my generational cohort is out of the system.

A. Too many libertarians.
B. Too many people convinced of their own prowess and respect
C. None of us are at much physical risk
D. We get quite a bit more than a living wage, in general

Those factors add up to an insurmountable barrier, even if I personally think the idea is wise.

This ... from the people who brought you Windows 8 (0)

Anonymous Coward | about 2 months ago | (#47510885)

All this talk about improving performance is from the people who brought you Windows 8, which is the worst step -backwards- in user productivity and interface in the history of microcomputers.

Let's see, it's a completely new way of interacting with PCs. Ah yes, Win8 is designed to only be used effectively with computers having primary interaction with users with a large touch screen. Then this touch-based OS is -forced- onto all new laptops: few if none of them having touch screen interface.

A completely new user-interface on a machine that only reason for existence is to improve its user's productivity. Where UI designers have been telling and teaching anyone who would/could listen that user productivity is maximized by allowing the user's to develop their own custom interface and to allow them to move this interface between platforms. As if anyone at Microsoft gives a shit about your productivity. As far as they're concerned, they are the only experts that matter. And they don't know shit about user interfacing.

For example, having a message box that pops up in the middle of the screen and then forces you to bend your wrist into a carpel-tunnel position in order to click on a tiny button to get rid of the message box. You can't just click on any button, with the mouse pointer anywhere to get rid of the message box. Which is the only thing any user wants to do immediately when they get a stupid fucking message box. This, after 25 years of focused research on interactive GUI operating systems.

Nobody should take anything that the Redmond Retards say seriously. Especially when they claim that it came from expert research. (i.e. blown out some idiot's ass).

Is there any company on earth that treats its customers with more contempt than Microsoft?

Re:This ... from the people who brought you Window (1)

s.petry (762400) | about 2 months ago | (#47512309)

Is there any company on earth that treats its customers with more contempt than Microsoft?

Comcast? AT&T? Anyone associated with the MPAA/RIAA?

Re:Why is it always developers? (1)

ranton (36917) | about 2 months ago | (#47513033)

Every time I hear about a terrifyingly invasive means of "improving performance" its targeted at developers. Is it just selection bias, or does the world actually hate us?

You really think developers are singled out here? There is an entire industry built upon business process improvement and operational management. I guess there probably are more invasive measures against professions like developers, but that is only because it has been hard to completely replace the profession with machines.

When you start having your bathroom breaks timed by your manager like some retail workers then you have it rough.

Double checking their work (0)

Anonymous Coward | about 2 months ago | (#47509663)

Thank goodness they are using this to double check the right parts of the code instead of trying to do something to make the developer's life a little more conducive to doing good work.

Hooked up to all the equiptment (2)

gunner_von_diamond (3461783) | about 2 months ago | (#47509685)

the researchers summarized how they strapped an eye tracker, an electrodermal sensor and an EEG sensor

I'm sure being hooked up to something similar to a polygraph doesn't make a developer more stressful at all. Was the fact that they had all this equiptment hooked up to them a factor in their statistics?

Re:Hooked up to all the equiptment (1)

i kan reed (749298) | about 2 months ago | (#47509735)

Checking the methodology section of the paper, they didn't feel it was necessary to include any sort of experimental control.

Now it can be hard to come up with controls for this sort of experiment, when you test the ability of an algorithm that tests for kind of nuanced data, like "where in this block code might there be bugs?", but it should've at least gotten a mention in the conclusion that it wasn't comparative to other methods.

Re:Hooked up to all the equiptment (0)

Anonymous Coward | about 2 months ago | (#47509769)

I'm sure being hooked up to something similar to a polygraph doesn't make a developer more stressful at all. Was the fact that they had all this equiptment hooked up to them a factor in their statistics?

insert picture of Malcom McDowell in Orange Clockwork here

Re:Hooked up to all the equiptment (1)

Jeremi (14640) | about 2 months ago | (#47510509)

Was the fact that they had all this equiptment hooked up to them a factor in their statistics?

Yes. When they compared their measurements against the measurements they gathered from the people they didn't measure, they took that bias into account.

:^P

Calibration (1)

Anonymous Coward | about 2 months ago | (#47509715)

I'm writing a networking multithreaded program in ASM that need to use very little memory, hence complicated registry magic and blatant violation of calling conventions.
The sensors are going to freak the fuck out.

Re:Calibration (1)

Zero__Kelvin (151819) | about 2 months ago | (#47511061)

You said "registry". That was enough to freak me right the fuck out, and I am a seasoned developer.

Wonderful....This won't be good... (1)

bobbied (2522392) | about 2 months ago | (#47509761)

Now my boss is going to be watching the developer's eye movements instead of testing code... This will not end well.

There is no magic bullet and where this might find the sections of code that your developer finds difficult to understand, it still isn't going to give you any idea about the quality of the code they produce. All you will know is how hard they concentrated when producing it.

I remember when we watched SLOC, but it was of marginal value. Then it was logical edges and complexity which was sometimes useful, but not always. Now they want to use biometrics to figure out how complex I find my code? It won't be any more helpful than complexity was.

Keeping code understandable and bug free has always been about naming identifiers, formatting, comments and using standard patterns and TESTING it as much as possible. All these golden bullets will only end up in your foot if you choose to use them....

Re: Wonderful....This won't be good... (0)

Anonymous Coward | about 2 months ago | (#47513025)

Actually code review has always been superior way to produce better code.

Problem is that code review lets you write only as good code as your team has the collective skills to do. But if you have even one "pro" the skills of every team member will skyrocket.

Here's a better idea (1)

gman003 (1693318) | about 2 months ago | (#47509763)

How about instead of spending a small fortune solving the handful of bugs caused by programmer typos, you spend that money on better requirements gathering, keeping specifications from changing constantly, and giving programmers time to actually unit-test and document their code?

If you want some fancy tech so you can write a paper on it, make an "electro-stimulus behavior moderation band", strap it onto clients/managers, and give them fifty thousand volts whenever they say or do something stupid.

Re:Here's a better idea (0)

Anonymous Coward | about 2 months ago | (#47509815)

Won't happen, as nobody would put up with the electricity bill.

you dont need biometrics for this at all. (5, Insightful)

nimbius (983462) | about 2 months ago | (#47509781)

When your developers cringe at a project, when they encounter a subroutine or callback that literally makes them groan, you've found exactly what needs to be refactored. if you find a python wrapper around a godforsaken class, or find explitives cursing a dead gods name in a forgotten universe, thats the code that needs your attention. Project managers, section leaders, whoever has direct line-of-sight communication with the dev pit needs to pay more attention.

the problem is 'refactoring' is a lie. as a DevOps (christ i hate that fucking word) engineer, I've been faced with rotting festering codebases for years in my career on a daily basis. the issue is business priorities interfering with good coding practices. I and 2 junior devs might want to go rip up a few thousand lines of horror-code to make everyone more productive, but we get denied. why?:

1. downtime is unacceptable for this application. this code controls so much, does so many things, and is so obscure (say it with me, payments processing subsystem) that to do ANYTHING to it is literally worse than pistol whipping the CEO's daughter.
2. New New NEW! we need to get in those swim lanes and stand up in those scrums nice and straight so we can deliver optimum ROI to our dear customers! who cares if the system crashes 5 times a month because this module is satans petrified asscrack, google just launched their new $app so our new $cloud_app_pro needs to go live NOW!.
3. we had the resources, but uber elite coders in our ranks were ganked to other projects months ago. they havent seen the code in 3 months, and we're sure they'll be along to help us again once they put in their 2 weeks and show up in flip flops for the knowledge transfer.
4. you were ganked from the refactor project and are now plugging away at an irritating new web 9.0 cash money matic piece of code that marketing wont stop skullfucking and your boss cant deliver fast enough. Catch this rabbit though and you'll be able to sit down and think through...wait....what was the refactoring project about again? oh christ is that CVS?

what this technology will get used for
efficiency sampling in your dev groups. eye tracking and biometrics will now subtly be included in SCRUM/ITIL/six sigma/devops/management wankfest.

Re:you dont need biometrics for this at all. (1)

drinkypoo (153816) | about 2 months ago | (#47511339)

1. downtime is unacceptable for this application. this code controls so much, does so many things, and is so obscure (say it with me, payments processing subsystem) that to do ANYTHING to it is literally worse than pistol whipping the CEO's daughter.

Then you can't afford not to have a backup server and a development server. This point needs expansion :p

This and more (1)

s.petry (762400) | about 2 months ago | (#47511373)

Once again we have some big sister/brother company/government claiming that they can do the impossible with biometric data. They don't address the primary source of the problems, which you lay out in detail.

Why was security skimped on in the code? Funding.
Why did funding get dropped? So that someone could get a bonus.
Who was the person that had the demo code for security? Canned to save budget.
Can't our Outsource code it? Not in their contract or business statement.

None of those issues are the coders fault, and this is the majority of our "shitty" code today. Piles and piles of shit so that someone in the management chain (or several someones) can get bonuses/raises/justify their existence in a company.

I'll give an alternate method of finding better targets for biometric scanning. Randomly sample executive and management emails. If you can win buzzword bingo in 2 or less random emails, you have a valid target. Build a "shifty eye" detector into power point, and there ya go!

64.99%, 84.38%, Really? (4, Interesting)

DrJimbo (594231) | about 2 months ago | (#47509793)

They tested 16 developers and gave statistics with four significant figures. I think you would need to test at least 100,000,000 developers to get such precise measurements. Who do they think they are? Dr. Spock on Star Trek?

Re:64.99%, 84.38%, Really? (2)

bobbied (2522392) | about 2 months ago | (#47509913)

They tested 16 developers and gave statistics with four significant figures. I think you would need to test at least 100,000,000 developers to get such precise measurements. Who do they think they are? Dr. Spock on Star Trek?

Naw, they just used a really accurate ruler, made each measurement 10 times and averaged their results...

You make an excellent point. There is no indication in the fine article about how accurate their results could be statistically, and given their really small sample size it doesn't seem likely 4 significant digits is justified.

Re:64.99%, 84.38%, Really? (0)

Anonymous Coward | about 2 months ago | (#47510151)

They tested 16 developers and gave statistics with four significant
figures. I think you would need to test at least 100,000,000
developers to get such precise measurements. Who do they think
they are? Dr. Spock on Star Trek?

Mr. Spock.. "but it couldn't possibly matter less". ;-)

Re:64.99%, 84.38%, Really? (4, Funny)

Anonymous Coward | about 2 months ago | (#47510269)

I think it the extra bogus precision has something to do with the conversion between Imperial and Metric developers.
(Oh, and something something dark side.)

Re:64.99%, 84.38%, Really? (1)

ortholattice (175065) | about 2 months ago | (#47510467)

While I agree the extra precision is misleading, the real problem is that they don't provide confidence intervals. Even a rounded 80% could be misleading if it could fall between 60% and 90% due to sparse data.

On the other hand, if in their report details they said "84.38% with a 99% confidence interval of 72.27% to 94.49%", then the extra precision is no longer necessarily misleading (it is just the calculational result of the model used) and, although it is a little pedantic and redundant, I would have no fundamental problem with it. It might even be argued that it is infinitesimally more precise, allowing the calculations to be confirmed by an independent researcher. However, for presentation in summary form, it would be much better to say "between 72% and 94% with 99% confidence".

The algorithm (0)

Anonymous Coward | about 2 months ago | (#47509833)

if(SLOC>0)
{
buggy = true;
}

Re:The algorithm (0)

Anonymous Coward | about 2 months ago | (#47510703)

If you code with that mindset, I hope I'll never work with you.

I strive to write "perfect" code if I have the time. It's not that hard, just double check and try to run the edge cases in your head whenever you do something complicated.
I'll still make mistakes, but only stupid mistakes and other easy to fix typos, not bugs and bizarre edge cases because I already tried to think of those.

It Works For Me

Re:The algorithm (1)

Zero__Kelvin (151819) | about 2 months ago | (#47511117)

"I strive to write "perfect" code if I have the time. It's not that hard, just double check and try to run the edge cases in your head whenever you do something complicated."

I highly doubt you can write a non-trivial C or C++ program without bugs, or really any language for that matter. I'm not talking about thousands of lines either. 100 or so should do it. The fact that you don't mention any kind of requirements spec, perhaps with the aid of some CASE tools, or at least a testing and feedback method, coupled with the fact that you think that you can do it "in your head" makes it clear that you have no idea how to develop code.

This is offensive (1)

J-1000 (869558) | about 2 months ago | (#47509853)

The core implication here is that developers are the source of mistakes, and those mistakes must be minimized. Never mind that developers are also a source of productivity and innovation, and that dehumanizing them decreases both.

Re:This is offensive (0)

Anonymous Coward | about 2 months ago | (#47510097)

Indeed, when devs are called "assets" or "resources", it's not a problem anymore to shove a probe up their asses to measure their stress level. Ever considered that "human resources" is the biggest oxymoron of this century?

Re:This is offensive (1)

gnoshi (314933) | about 2 months ago | (#47511449)

They are saying developers are the source of bugs (mistakes?), but not in the way you are suggesting. Developers are the source of bugs in that they write the code which includes the bug, and so it is not particularly surprising that you can read biometrics that indicate when developers are likely to produce code with bugs.
For example, if the developer hasn't slept in two days and so has saggy eyes and wildly drifting eye movement then that's a pretty good indicator that there will be some bugs, and indeed the developer is the source of the bugs because they are the source of the code.
Of course, the manager standing over their shoulder with an unreasonable product release deadline and the threat of job loss is probably responsible for the bugs in any reasonable sense.

Analogy time:
If someone is driving a car wearing a blindfold and crashes, they crashed the car. The person who put the blindfold on, held the gun to their head, and said 'drive' is probably responsible for the crash by any reasonable definition.

In summary: I don't necessarily think it is offensive to say that bugs are coded by developers, because they are. However, it is offensive to say that they are responsible for the bugs without taking into account the broader context in which they are working (and indeed, saying they are responsible for the bugs still doesn't necessarily mean that they are in some way wrong or deficient for entering a bug. People - even brilliant people - can and do make mistakes, and that is why review processes do (or should) exist.

Re:This is offensive (1)

J-1000 (869558) | about 2 months ago | (#47522539)

In summary: I don't necessarily think it is offensive to say that bugs are coded by developers, because they are. However, it is offensive to say that they are responsible for the bugs without taking into account the broader context in which they are working (and indeed, saying they are responsible for the bugs still doesn't necessarily mean that they are in some way wrong or deficient for entering a bug. People - even brilliant people - can and do make mistakes, and that is why review processes do (or should) exist.

Yes, that's a good way to put it. I wonder if they considered attaching biometric probes to managers to find out when they are most likely to come up with stupid ideas? :)

when developers talk to product managers? (1)

bityz (2011656) | about 2 months ago | (#47509883)

Should measure the interaction between developers and product managers before coding even starts - see how the developer responds to impossible requests, contradictory requirements, and meaningless buzzword filled descriptions...

Foreign workers (1)

jgotts (2785) | about 2 months ago | (#47509899)

Microsoft Research should also track how far the individual is working away from the main office of his company, because that has far more of an effect on bugs than any biometric reading. I recommend that they develop a special laser and a series of geostationary satellites and ground repeater stations. The total round trip time of the laser pulse will be a measure of how buggy the developer's code is.

1) Microsoft Research is wasting an awful lot of money to conclude that the reason why Microsoft's software is so terrible is that it's being written by outside companies in India.
2) Microsoft's well-paid American and European programmers are producing good or at least above average code, as would be expected.
3) Quality evaporates when they use foreign and H1B workers, who are educated in substandard universities, inexperienced in engineering, and/or do not have English superliteracy. [I've discussed elsewhere that basic language literacy is not enough to be a good programmer---you need to have enough language expertise to communicate without any ambiguity whatsoever to write good code because of the essential nature of interpersonal communication. By the same token, if you're writing software for Indians speaking Hindi your entire team should be Hindi-speaking Indians.]

It should write commit messages (0)

Anonymous Coward | about 2 months ago | (#47510027)

"This code was written while stoned"

Analyze difference in performance/productivity (1)

OutputLogic (1566511) | about 2 months ago | (#47510061)

Using the same methodology, I'd be interesting to analyze differences in performance/productivity of developers. I'd expect to see something like normal or log-normal curve.

uh huh (2)

Charliemopps (1157495) | about 2 months ago | (#47510093)

Example? [businessinsider.com]

That's where we're at?
I see this garbage all the time... and I never understand it. Growing up my father ran a factory and was damned good at it. His people showed up on time, did great work, had low scrap and were very productive. How did he manage this amazing feat? Minders that followed you everywhere in the factory? Discreet blood samples taken hourly? No...

They had stats. That's it. You produce X parts per week. Go way above that get a bonus, go way bellow that, you get fired. If everyone is getting bonuses the stats would rise... if everyone was getting low stats they'd first check for procedural problems that might be hindering peoples work and baring that they'd lower their expectations. It worked marvelously well... people would think of ways to go faster and bring them up... because it meant a bonus until everyone else caught on. Anything that made production harder was immediately reported because people wanted their bonus.

Damn near every successful factory in the country works this way. Do the same thing for code. What my father always said was "They could spend 7hrs in the bathroom every day... I don't care... if they can come out at hit 1000% efficiency that last hour to make up the difference it's fine with me. But they better keep in mind their peers are going to eventually figure out how they pulled it off and change the curve."

Re:uh huh (1)

Stormy Dragon (800799) | about 2 months ago | (#47510261)

The problem is that while widget A and being treated as fully equivalent to widget B, it's hard to compare one line of code to another line of code from a completely different program.

If good programmer just means "spits out more lines of code than anyone else", I can write all sorts of gobbledy gook that add lots more code without really participating in the actual purpose of the program.

Re:uh huh (1)

jcochran (309950) | about 2 months ago | (#47510267)

And what stats do you apply to code development?
Because quite frankly, that is the gist of the problem.

Re:uh huh (1)

mythosaz (572040) | about 2 months ago | (#47510337)

You simply use a longer timeline.

Nobody worth their skin judges a poker player by who won the last pot -- but over time I can measure quite well who the good ones are.

Re:uh huh (1)

jcochran (309950) | about 2 months ago | (#47510587)

And you've still avoided naming any metrics....

Number of lines of code? As mentioned earlier, one can easily inflate LOC with trash.
Also how do you evaluate a programmer who actually reduces the lines of code in a program? By the LOC metric, said programmer is counter productive. Then again you get the beautiful quote by Ken Thompson... "One of my most productive days was throwing away 1000 lines of code."

Code quality? Once again, how do you judge it?

Re:uh huh (1)

mythosaz (572040) | about 2 months ago | (#47510747)

I'm not avoid naming metrics.

More analogies. You can judge an entry level grade-school musician by notes missed. He gets credit for showing up to class and not tooting his horn (literally) during the rests. Advancement beyond that requires someone who knows what a good musician sounds like, the subtleties and nuances in his play and style.

You can judge entry-level programmers by lines of code, errors in their code, and other simple metrics. You take those numbers day by day, and like the poker player above, you know that sometimes good code brings bad results and gets discarded. You look at their performance over time, their ability to deliver expected workloads over weeks, months, quarters, and years. You look at their ability to provide complex solutions. Some of this might be, "I know it when I see it," but that's still how you measure people.

Art teachers can mostly agree on talented artists and musicians. Coding isn't that much of a mystery.

Get the right managers.

Re:uh huh (1)

complete loony (663508) | about 2 months ago | (#47512757)

Writing code is nothing like working in a factory producing identical components. It's more like designing a house, followed by an office building, a bridge, a power drill, a pacemaker, a roller coaster, a lunar lander, ...

If every developer is doing their job perfectly, they will literally *never* write the same code twice. Every single task they do will be different from all of their previous tasks. So how do you measure their output?

This misses two of the biggest developer problems: (3, Interesting)

jeffb (2.718) (1189693) | about 2 months ago | (#47510173)

1) Arrogance. You know that average developers have a hard time with some kinds of code, but you're a superprogrammer, and you don't have those problems. If someone decides later that there's something wrong with your code, well, they should've gotten their requirements straightened out before they told you to go and build it. The only time you lose your cool is when you have to deal with idiot managers, analysts, or users.

1) Complacency. You've been pounding on this code forever, and you just don't care any more. Yeah, there'll be bugs, people will yell, they'll get fixed. That's just the way development goes. Why get worked up about it?

My oh-so important opinion (0)

Anonymous Coward | about 2 months ago | (#47510339)

>Arrogance

The problem is that it's often justified. The level of skill can vary wildly between the beginners who struggle to learn the simplest things, spamming ever forum so much that you can't help but wonder if that's the norm, and people who come up with jaw dropping solutions on the spot like it's natural.
It's easy to spot "stupid" people, and there seems to be tons of them, but it's hard to notice when someone is smarter than you, because good code is the standard and the occasion to write excellent code isn't always there, therefore one would tend to think than he's smarter than the average.

To sum it up, I have no idea what I'm talking about and should really just stop posting.

Fixed It For You (1)

xdor (1218206) | about 2 months ago | (#47514455)

2) Bugs

I would stick to code reviews, test driven design (0)

Anonymous Coward | about 2 months ago | (#47510209)

Errors happen whether you are stressed and/or find a particular problem challenging, or when you are cruising and/or oblivious to the actual complexity of the problem you are facing.
Not to mention it will make everyone nervous just thinking that their vitals are taken while they work, since this has the potential of becoming a performance goal. Lead: "You need to relax, we end up reviewing every single line of code you write" Poor developer: "Could you point me to the nearest suicide machine?"

Codename (0)

Anonymous Coward | about 2 months ago | (#47510219)

Skynet

lol (1)

PC_THE_GREAT (893738) | about 2 months ago | (#47510425)

lol what the f*ck!
I quote from the Microsoft research paper:
"In this paper, we investigate a novel approach to classify the difficulty of code comprehension tasks using data from psycho-physiological sensors. We present the results of a study we conducted with 15 professional programmers to see how well an eye-tracker, an electrodermal activity sensor, and an electroencephalography sensor could be used to predict whether developers would find a task to be difficult."

15 developers is enough to reach a conclusion???
Note that the paper is about investigating the difficulty of code comprehension, not how bug are introduced, while it may be indirectly linked, it is not for a fact linked, it is just an assumption, there's no such direct relationship.
Also each developers codes differently and requires different working conditions, and feels different emotions while coding, how the heck can they assume that EEG results will show any similarity?

This is a bullshit research wasting time and money. Probably some kind of marketing feat that will be used to claim Microsoft no longer makes bugs in its code or just some sort of conspiracy to turn programmers into zombies hooked to machines.

Lol blame the programmer for the bug when bugs occurs for a lot more reasons than just one human doing the coding, building a program is more than just about a programmer.

Anyways, it is a research paper :p, everyone should bow down and say "amen you are right" :s. Stupidities

Development or crunching out code? (0)

Anonymous Coward | about 2 months ago | (#47510621)

Sure, it is possible to see which parts of the code are trivial, and which parts require actual thinking. You don't need fancy machinery for that, anyone can see the difference. But try to apply the fancy machinery to a good programmer actually trying to solve a real life problem! Suddenly it is not about what lines they stare most at, it is about what else they google, what they grep for in the library headers, where they go for similar problems, how they test various hypothesis about the problem...

And the really, really good programmers, they do the same kind of troubleshooting. But knowing their stuff that well, they think about those problems before writing the first line of code, and do not run into this kind of problems so often. Try to find an eye-tracker or any other magic to pinpoint the mistakes those guys just don't make!

Cause of stress (0)

Anonymous Coward | about 2 months ago | (#47510717)

They miss the cause of the stress: IT is terrible and incomplete specs. The errors are found when the specs are completed AFTER the coding and sometimes after the code MUST to be put into production.
Shove some electrodes into the Project Managers and Spec writers.

Just what we need (1)

Coditor (2849497) | about 2 months ago | (#47510963)

To be hooked up to some device while we work to measure how likely we are to wind up at the top of the stack rank. It could be completely automated, if it determines you just wrote sucky code it generates a pink slip email and a robot carries you out the door. Instant better code!

Interesting but misguided (1)

BlackHeron717 (3762173) | about 2 months ago | (#47511365)

While this information can be useful to software developers to help them optimize their own performance, it will likely prove detrimental to provide these metrics to managerial or production supervisors, as they typically only choose in intensifying the workload to improve efficiency. I would like to see more people tested and less emphasis on productivity, and more emphasis on how we can use this data to improve the experience of coding for the programmer. That would probably result in better code more readily than objectifying the software engineers through the use of data that might not speak to the experience of the vast majority of programmers.

My metric to predict buggy code... (1)

aeschinesthesocratic (1359449) | about 2 months ago | (#47511465)

..how many swear words are in the comments.

Blipverts (0)

Anonymous Coward | about 2 months ago | (#47512065)

If their skull explodes while stressing out over the code.. bloody 'X' marks the Spock.

More please (0)

Anonymous Coward | about 2 months ago | (#47514939)

The more the MS business units do of this stuff, the more of Bill Gates' heard stolen warchest they'll be eating into. Yes Microsoft, we want more of this sort of insane shit please! Next time, be sure to sample 1000 developers for at least a week each!

Let me get this straight... (1)

whitroth (9367) | about 2 months ago | (#47516399)

*Microsoft* is working on intrusive software to predict buggy code? I can do that without software - just point at the Microsoft campus, and any and all products....

                        mark

And now what? (1)

david_thornley (598059) | about 2 months ago | (#47516853)

So, Microsoft Research has developed a method to tell when a programmer is in a condition that tends to create bugs. That's nice. What happens with this?

I already know when I'm in a condition that tends to create bugs. It won't help there. It could be passed on to others, such as management.

Now, is management going to take action to reduce the amount of time I'm more vulnerable to causing bugs, by improving the office environment or discouraging overtime or making reasonable deadlines? Or is management going to find this a good evaluation tool and ding my performance reviews if I'm spending too much time in that condition? Is excessive stress going to become a thought crime?

The only way this would be useful is if management recognized that they needed to reduce buggy time and were rewarded partly on that basis. Anybody confident their employer will do that?

You don't need biometrics for that (1)

allquixotic (1659805) | about 2 months ago | (#47518049)

It's simple: Just install a program on your developers' computers that tracks how often (how many times in general, and for how long) the developer switches focus away from their IDE. If they're constantly googling, looking up reference docs or algorithms, etc., chances are they are doing something that's new, untested, uncharted territory for them. If they're just rattling off hundreds of SLOC at a time, while only needing IntelliSense as an aide, chances are most of it will work on the first attempt.

Programmers who use books made of real, physical paper foil this test and should be summarily fired.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>