Wiring Programmers To Prevent Buggy Code 116
mikejuk (1801200) writes "Microsoft Researcher Andrew Begel, together with academic and industry colleagues have been trying to detect when developers are struggling as they work, in order to prevent bugs before they are introduced into code. A paper presented at the 36th International Conference on Software Engineering, reports on a study conducted with 15 professional programmers to see how well an eye-tracker, an electrodermal activity (EDA) sensor, and an electroencephalography (EEG) sensor could be used to predict whether developers would find a task difficult. Difficult tasks are potential bug generators and finding a task difficult is the programming equivalent of going to sleep at the wheel. Going beyond this initial investigation researchers now need to decide how to support developers who are finding their work difficult. What isn't known yet is how developers will react if their actions are approaching bug-potential levels and an intervention is deemed necessary. Presumably the nature of the intervention also has to be worked out. So next time you sit down at your coding station consider that in the future they may be wanting to wire you up just to make sure you aren't a source of bugs. And what could possibly be the intervention?"
Can we wire Timothy to prevent duplicate posts? (Score:4, Informative)
Seems like we just covered this... [slashdot.org]
Re: (Score:3, Funny)
Admittedly this version sounded more exciting. I was imagining failed builds resulting in electric shocks.
Disappointed to find that wasn't the case.
Propaganda requires reinforcement. (Score:5, Insightful)
That they continue to try and blame programmers for problems that management creates should bother people. Yes, this is a rewording of the same article. The only way to sell this alchemy (I would not even call it pseudo science) is by continually bleating out the theme "programmers are baaaad and we can detect them being baaaaad".
Lets just say (though its impossible for them to do) that they can isolate every possible variable that would cause fluctuations in EEGs, heart rates, and everything else they want to track biometrically. How on earth do they plan to track and correlate that data without either knowing or directing everything you eat, drink, and touch (including shampoo, soap, and hand lotion).
Even better, how do they plan to either direct or track all of your personal interactions which can impact those same things. If they can't, then the numbers are skewed and we don't have any science, you only have biases.
Oh, I can see it now. "Monty. We know your dad died yesterday but you are performing sub par and your heart rate is elevated higher than we expect, so we are going to have to let you go. We can't have a repeat of your three days of sub par performance when your wife divorced you five years ago."
Now lets look at some of the great Microsoft Management decisions, and decide who's to blame: Programmer or Management. Lync can not copy/paste data. Do you really think a programmer didn't notice the lack of basic function and point it out? Windows 8, and in fact Microsoft's whole "One UI" strategy is management driven. Zune, was actually a decent device but management killed the program. Programmers said Win8 was not ready, Management released it (as they did with Vista, ME, countless back office products, etc..). Programmers have had fixes for security ready only to have them pulled by management for various reasons. "Can't fix that IIS back door because someone's code may break" is frigging hilarious, but how many times have we read just that from Microsoft management?
That list could get really really long, so here is the point. Why are they not hooking management up to these things and putting them under pressure instead of the programmers? Probably for the same reason I start with, which is that this is alchemy and not science.
Re: (Score:3)
That's not in any possible sense a bug. It is a strategic choice you disagree with. As for not releasing products when programmers say you should. I think Perl 6 provides a pretty good example of what happens when complex projects aren't disciplined with the "best version ships in 10 months".
Re: (Score:2)
Franchise destroying blunders are peer reviewed by the business press among others.
Alchemy (Score:2)
Alchemy? Not even that much science. Say Astrology.
What good does it do knowing how people produce buggy code when the #1 reason for the buggy code is underqualified people being pushed under unrealistic conditions?
I, too thought "wired" as in "we apply electric shocks when they screw up". I'm sure at least 1 Dilbert cartoon already anticipated this.
Re: (Score:1)
Oh so much this. Also, drastic changes to architecture toward the end of the project being driven by project managers who don't know how to say no to clients, and then the dramatic rush to complete these changes without taking the time to carefully rethink what these changes will affect.
Re: (Score:3)
100 volts for the first dupe, 200 for the second, etc.
Haven't decided on an arithmetic or geometric progresion yet
Re: (Score:3)
Fibonacci FTW.
Re: Can we wire Timothy to prevent duplicate posts (Score:1)
Take it a step further... (Score:3, Insightful)
...make sure you go back to the business guy who create the requirements being coded at the time, hook him up to an EEG, and see if he wrote a buggy spec :) ...for bonus points, put the EEG on the manager who just said they're going to measure productivity in LOC.
If we could detect stupid with an EEG, programmers would probably be the least useful people to put it on.
Not as novel as it sounds (Score:1)
The procedure is just a variant of letting the programmer write a piece of code and then hook him/her up to a polygraph and interrogate her/him if the code has any bugs, and if it does, in which source file and line number....
Re: (Score:2, Insightful)
The procedure is just a variant of letting the programmer write a piece of code and then hook him/her up to a polygraph and interrogate her/him if the code has any bugs, and if it does, in which source file and line number....
"Does this code have any bugs?"
"Yes, like all non-trivial programs, this program has bugs too."
"Where is the bug?"
"If I knew, I would have fixed the bug already. But that's not the point. There's always one more bug that haven't been found yet."
Re: Take it a step further... (Score:2)
Re: (Score:2)
"Difficult tasks are potential bug generators..."
So are easy tasks; it can be surprising how much of a bug generator is complacency.
Hi, it looks like you are writing difficult code! (Score:5, Insightful)
The last thing I need when I am focused of a difficult chunk of code is Clippy popping up and breaking my focus. Some tasks are just much harder than others. I think any decent coder knows when they are struggling and increases their focus or grabs someone on their team to help. Bad coders may not, but bad coders are always bad coders, and no tools will help someone who just can't get it or just doesn't care.
Re: (Score:1)
"finding a task difficult is the programming equivalent of going to sleep at the wheel"
Bullshit! To a programmer a difficult task is a challenge and sometime we generally crave amidst the drudgery that is routine, easy tasks. One day I am going to conduct a research study into drilling a hole in the skull of management and watching how their brains react when confronted with ethical dilemmas. Of course after the study finishes they'll be lobotomised for their own good.
Re: (Score:2)
Hear, hear. Of course that very craving may easily mean we horde the challenge to ourselves, shunning a second opinion at the very moment it would do us the most good.
As for management brains and ethical dilemmas, why would you assume there would be a reaction? Have you seen some sort of behavioral evidence that they even notice them?
Re: (Score:2)
Not only are the difficult problems more fun but how will you ever advance if you are only ever given easy problems.
Yes, someone might make more mistakes when they are doing a harder problem but that is how a person learns.
Taking away the hard problems isn't the answer but maybe requiring additional peer review for hard problems but
you don't need an invasive brain monitor to find this out. Just ask the programmer. As a programmer I can easily
tell you after it's done which parts were easy, which parts were
Re:Hi, it looks like you are writing difficult cod (Score:4, Informative)
Honestly, I find my buggiest code is the easiest crap that I'm just not paying attention to. The moment I have a solid challenge my interest peaks and getting it singing perfectly is the only acceptable end result. It may take a few tries but it's what I thrive on. I'm my least productive and most careless on the routine pointless tasks.
Re: (Score:1)
This is just what I thought when I saw this article. Do any of the study's authors actually *do* programming? Bugs can arise in any code: hard, easy, or middling. I think that someone here is exploiting the inherently buggy grant-writing system.
Re: (Score:3)
Same here. Usually the coding mistakes occur in the easiest code, and are usually the easiest to detect and fix. The hard and undetected bugs are the ones that are the result of multiple pieces of code interacting in unexpected ways, easy, medium or hard at the individual code chunk level doesn't really matter.
The other source I have found is leaving unspecified paths open to users. You think that you don't have to prevent a user from doing something because it should work. It is actually more effort to
Re: (Score:3)
Re: (Score:3)
Actually, as I recall the studies suggest that sustained work-weeks over the 35-40 hour threshold actually generate *negative* productivity - as in you get less total productivity out of a 60-hour week than a 40-hour week. A far worse situation than your phrasing suggests. I would actually suspect that in most situations hourly productivity starts to fall off much sooner, so that a 20-hour week would deliver more than half the productivity as a 40-hour one.
I can think of two important reasons a company (o
Re: (Score:3)
Or, rather than thinly disguised duckspeak, it could simply be that Management is still stuck in an 1890's industry mentaility where the longer you run the machines and the harder you run them, the more output you get and the sooner you get it.
Human creative endeavor, however, doesn't work like a hamburger grinder. It develops at its own rate, and not infrequently, any attempt to accelerate that rate will actually cause it to jam up. Rather like the old joke that if you pull on the paper as it comes out of
Re: (Score:3)
On the other hand, it might be useful to have an actual resident guru or two wandering around helping out anyone having trouble. I assume you believe you're considerably better than the average programmer, so imagine you're the guru who gets notified whenever one of the code-monkeys starts having problems. If you're as good as you think and have a decade or three of experience under your belt, then I suspect that more often than not you could make a couple key suggestions that would improve the rookie's c
Re: (Score:3)
Actually, having the guru ask for help from the rookie is a good way to engage the rookie. It's pretty motivating to the FNG to be able to say "hey, I really helped the smart guy, I must be doing something right!" Pair programming is one way to put them both in that situation.
Re: (Score:2)
That too, but I'd never suggest it to a large-egoed guru. Demand it perhaps, but a suggestion would almost certainly be a waste of air.
Re: (Score:1)
I did the guru thing at my last job. The problem with it is that you are helping everyone else who is struggling with things get their projects done, so they show visible 'progress'/'accomplishments' to management, while your manager wonders WTF you do all day (helping everyone else). Eventually you're probably laid off because you aren't adding any 'value to the organization' by having less overall projects/goals completed, even though you were integral to everyone else meeting theirs.
I remember my old b
Two gurus with egos in check = awesomeness. APIs (Score:3)
I've been coding, and studying to improve my skills, for decades. People come to me for help and advice. With one open source project I work on, all code must be approved by at least three people. My counterpart for one module is also very experienced, and a perfectionist. Sometimes our egos collide, but when we get past that we come up with something much better than either of us started with. That's particularly good when we're working on an architecture or API that otter people will have to code to
Re: (Score:2)
I have seen many books on software development that say that a significant part of a senior developer's job is supposed to be teaching, thereby increasing the overall team's productivity. Of course what an MBA would say is that the senior developer is not doing enough programming and direct the senior developer to stop helping others to the detriment of the team.
Re: (Score:1)
In my experience the majority of bugs are not simple logical mistakes that can be seen as wrong within a limited scope. They are locally-correct code that breaks a hidden dependency resulting in a failure of a completely different part of the system.
In theory, well-designed systems with good separation of concerns make the upstream and downstream dependencies clear, so this doesn't happen (and even when it does the automated regression tests catch it). In practice, deadlines prevent code from ever achievi
Re: (Score:2)
Yup, that's correct. The best way to combat bugs is to write modular systems comprised of parts that can be designed and tested independently. OO was a failed attempt at doing this. But instead of moving on to better systems (like systems that have actual algebraic type systems), programming culture is still pretty much stuck on OO and languages that use it (like Java or C++).
Re: (Score:2)
There is no such thing as an inherently 'bad coder', only someone who has not yet learned how to code properly. I really think there's a huge cultural problem in programming. People view things like functional languages as 'academic' and 'too hard', but if you consider the amount of effort saved in debugging and maintaining code, they are very much labor-saving devices. And I am by no means a functional programming fanatic; I program in both conventional and functional languages each day. But people need to
Re: (Score:3)
There are, in fact, inherently bad coders. Coding is just like anything else. Some people have the apitude, some do not.
I am an inherently bad bookkeeper, since I work by scanning the big picture and integrating it, and a meticulous line-by-line process will only result in my dropping lines.
I am an inherently bad sales person, because it's actually painful to me to approach people, and my persuasive skills are probably measured in negative numbers.
But I can execute prodigies in software coding without even
Re: (Score:2)
Take your pseudo-political bs elsewhere, I am not talking about being 'interchangeable cogs' at all. Sure, not everyone has the smarts to be a coder. That's painfully obvious. But if someone has the smarts to be able to write code that works - even if it is sloppy, horrible, and badly-designed (what you would typically describe as 'bad coding'), then they can be trained to write better code.
If someone sucks at the absolute basics and can't even write a loop without help, then you should obviously kick them
No silver bullet (Score:2)
Re: No silver bullet (Score:5, Insightful)
Difficulty act as a motivator, and helps developpers to focus. Most bugs I've seen (or caused) are on easy, boring tasks.
Re: (Score:3)
Re: (Score:3)
A good question would be what exactly do they mean by "difficulty" in the study? If they're simply detecting increased stress or focus then you may be right; however, if they actually found some neurological/physiological response that correlates well with "increased chance of introducing a bug" then they would be zeroing in on exactly the right kind of "difficulty" that would suggest intervention.
I would suggest that, even if they're detecting you working "legitimately difficult code", if the intervention
Re: (Score:2)
Re: (Score:2)
Can't be bothered to look up the obligatory XKCD but
Mission. Fucking. Accomplished.
If you're tasked with writing code to do X, and you find a way to do so without actually tackling any difficult problems head on, then where's the problem?
Breaking things is how we learn (Score:4, Insightful)
Doing hard or unfamiliar tasks is one way we improve and grow. Mistakes improve our understanding of a domain.
Flagging code for extra review based on "struggle detection" *might* be useful. Sadly, we all know that we'd end up with clueless management punishing or even firing good people because they were stretching to meet a goal.
Re: (Score:2)
Great, then go ahead and hire somebody at twice my salary who already knows how to to do all this $#@! instead. You've got to pay for quality my friend, and we both know you're far too cheap to do so.
Re: (Score:2)
Indeed. Detection of 'struggling' (or should we call it 'cognitive challenge' in this context) provides an excellent opportunity to have another developer head over and for them both to work on the problem, reducing the likelihood of bugs and design errors and potentially providing skills improvement for both people which the company then benefits from.
If someone is consistently struggling when working on basic tasks, it may be that the person isn't suitable for the role (and some people really aren't) but
Premise flawed? (Score:4, Interesting)
Re: (Score:2)
NOW, after my moderator points have expired, somebody posts something I would want to mod up!
From my own experience, when I had really difficult, gnarly problems, the code came out really clean at the end. The bugs came when I was least expecting them with stuff that should have been a piece of cake.
I think it might be a bit like what somebody once told me about private airplane pilots. Statistically, accidents didn't happen the most when people were novices, but after a certain number of flight hours.
Re: (Score:2)
Furthermore, good programmers often anticipate problems that lesser ones are oblivious to. On account of this, the former may show higher signs of stress (which is actually concentration) early, while the latter don't realize things are going wrong until they see tests failing in ways they don't understand, and only then will the stress levels reflect actual competence.
Two areas where this is particularly prevalent are concurrency and security - though often, in the case of security, the problems are not fo
Get rid of the distractions (Score:5, Insightful)
Bugs come when people lose track of the big picture. When they lose concentration and focus - just like when jugglers drop the ball.
If you want to reduce the incidence of avoidable bugs, get rid of the distractions. Ones such as other people interrupting, phones ringing, asynchronous non job-related alerts going off (fire alarms excepted) and the administrivia associated with the programming environment. Maybe even unplug them from their "entertainment", too.
There will always be non-avoidable bugs: ones where the programmer is simply making a mistake, isn't up to the task in hand or has been given a bad brief or wrong information.
Re: (Score:1)
This. Totally agree on this one. When you make context switches (sales/pm asking a 'quick question'), and you continue programming before the full context is restored. That's where bugs happen. That has been my experience in 35 years of coding. No distractions: less bugs. Mod up please.
Re: (Score:3)
There will always be non-avoidable bugs
That's what I was thinking. Some bugs are practically impossible to avoid, because it is almost impossible for a single programmer to have the entirety of a software system in his head, with all possible cases and side effects. Without complete knowledge of a system it's impossible to know the perfect solutions.
At some point it's up to testing and quality control to take responsibility and weight off the programmers shoulders.
Not all bugs are in difficult code (Score:4, Insightful)
Yes (Score:5, Insightful)
Bugs mostly come from when you're not paying attention, or you forgot something.
If you've doing difficult coding then writing code is the last thing you doing be doing. By the time you get to writing code it shouldn't be difficult anymore.
they think they know what they're doing. Security (Score:2)
Many bugs are of course a simple "oops"or "l forgot that. I knew, and it slipped my mind".
Another very large portion are cases where they think they know exactly what they are doing, but in fact they are in way over their head. Many, many times I've pointed out a bug and had the developer argue with me, even after I showed them the problem. I have to actually crash/exploit their application before they change it, while still arguing "an attacker would never think of that." Dude, the attacker doesn't ev
Re: (Score:2)
Not all bugs are in difficult code ...just in difficult tasks. Going to the management meetings ought to make their machine explode ;-)
So.... (Score:1)
Completely ignores bad specs... (Score:5, Insightful)
While completely ignoring that the specs handed down from the higher-ups are gibberish, contradictory, and physically impossible garbage. But, you know, that is not a possible source of bugs now is it?
Someone should first wire up management to zap them every time they get an idea for a "brilliant" addition.
Re:Completely ignores bad specs... (Score:4, Informative)
Mod parent up, please, this is spot on. You do this sort of "research" when you need to justify that the expensive toys you bought are actually used for something.
When I have seen the list of sensors they are sticking on the user, this has nothing to do with anything even remotely practical (have you seen a typical EEG sensor cap or eye tracker?). All the researchers are doing is running the test subject through a battery of experiments and classifying few measured values, based on some correlations - in an artificial setting.
This completely ignores the complexity of the problem - such as the biggest problem being constant interruptions from managers and colleagues, distractions in a noisy cubicle, bad specs, poor/inadequate tools, etc. What they are proposing is basically a Clippy on steroids with a ton of expensive sensors. Such papers are published a dime a dozen (google "assistive agents" for example), not sure why exactly this one got picked out as somehow interesting.
Re: (Score:3)
Someone should first wire up management to zap them every time they get an idea for a "brilliant" addition.
I had this at work today. Somebody arbitrarily decided to store 3 months worth of hourly MySQL backups. Never mind that they were being stored on the MySQL server itself (some backup!) but each backup was over 1 GiB - that is 30 GiB per day. 30 * 90 = 2700 GiB on a server with a 2 TiB hard drive that was already half full.
I enjoyed cleaning up that mess, but I as usual people with no technical knowledge continue to make technical decisions.
Re: (Score:3)
I agree.
The source of most bugs is pressure. Time and money. Which results in management making bad choices.
In early stages of software development, the specs and design documens are almost always too abstract and insufficiently detailed, but development is started anyway, due to time/money constraints. This is not nessecarily a bad thing, if you follow some iterative or agile product development process, but it means some big refactoring and probably a redesign will be needed on the way. Especially when us
Intervention? (Score:2, Interesting)
What isn’t known yet is how developers will react if their actions are approaching bug-potential levels and an intervention is deemed necessary.
Intervention - you get fired.
In software development/IT/software "engineering" asking questions is considered to be a sign of stupidity. So, you better search and search the internet. Spend all your time learning the code - and KEEP your deadline.
We need to get a job done. Spinning one's wheels for hours, days, or whatever over something that can be answered in 5 minutes is idiotic, but that's what you gotta do or you're an idiot. I know, I have been called an idiot because I asked questions. I try to be e
Re: (Score:2)
Tricky network q's are what is the 8th layer and MIDI part of the presentation layer or not
Wiring eh .... (Score:1)
A guy at work has been esposing hooking Jenkins up to an electric shock generator ... break the build ... BBBZZZZZTTTTT!!!!
False Premise (Score:3)
Difficult tasks are potential bug generators and finding a task difficult is the programming equivalent of going to sleep at the wheel....
( hmm. Easy tasks are potential bug generators and finding a task easy is the programming equivalent of going to sleep at the wheel... huh. that works too.)
^^
Like all the other sane people I stopped reading at that point.
The real reason for doing this is that someone has invested a lot of time and effort in the technology used for this (and has a lab) , and this idea was the best one he came up with.
Re: (Score:3)
Mod parent up, please, this is spot on. You do this sort of "research" when you need to justify that the expensive toys you bought are actually used for something.
When I have seen the list of sensors they are sticking on the user, this has nothing to do with anything even remotely practical (have you seen a typical EEG sensor cap or eye tracker?). All the researchers are doing is running the test subject through a battery of experiments and classifying few measured values, based on some correlations - in an
Re: (Score:1)
Exactly, on large complicated (as in many featured) codebases, bugs are more often introduced when the programmer is entirely unaware that her simple fix for the scenario she has been asked to correct actually breaks the code under other less common scenarios. ie happy ignorance is more dangerous than stressed difficulty - at least in that case a sensible programmer would typically ask for help.
Fundamentals (Score:2)
In the B2B space, a lot of code gets written to wire up databases to front-ends of some time, and most of the time it involves an RDBMS. Unfortunately, with all the reliance upon ORM frameworks, developers often can't write or diagnose decent SQL to save their lives. We have a good chunk of Oracle code written by a large integrator, and there are innumerable cursors, one after the other, where a simple SQL join would have done the job much more easily. In Microsoft land, people are leaning way too much o
Want better code? (Score:2)
.
The problem with Microsoft's approach in TFA, is akin to "when all you have is a hammer, everything looks like a nail".
It is actually a lot better to solve the actual root problem, than trying to find and treat symptoms.
Good premise, bad reporting? (Score:4, Interesting)
Moronic (Score:2)
Difficult tasks are potential bug generators and finding a task difficult is the programming equivalent of going to sleep at the wheel.
This is moronic, at best. It's the easy things that are the equivalent of sleeping at the wheel. The hard stuff makes us stop and think. I always spend much more effort analyzing hard problems, and breaking them down into manageable chunks, than I do when I'm on cruise control implementing the simple stuff.
Who thinks up this crap?
Re: (Score:2)
Perhaps we should fix it:
Making ungrounded assumptions about the causes of errors in someone else's field of expertise is the intellectual equivalent of going to sleep at the wheel
Wha? (Score:2)
finding a task difficult is the programming equivalent of going to sleep at the wheel.
Say what?
Finding a task difficult is a chance to learn something new. Finding a task difficult is a chance to flex your mental muscles.
Re: (Score:2)
Good luck with that my friend (Score:2)
wrong target (Score:2)
I believe that a far more productive idea would be to apply strong electroshock therapy to managers that insist on unreasonable due dates since that is the single greatest cause of software bugs (cutting corners to meet an arbitrary date).
Not quite right (Score:2)
There is no evidence to suggest most bugs come from tasks developers find difficulty. Which is implying that "shotgun troubleshooting" is causing bugs ---- this is a real potential phenomena, but I believe it is one primarily suffered by novice programs.
Competent programmers are likely to seek help and be extra careful about it when they find something difficult, which suggests more testing and more careful coding and quicker discovery of bugs with the thing they are finding difficult..
What's more lik
Wrong audience, do this for *executives* (Score:2)
Easy Peasy (Score:1)
How about this. (Score:4, Insightful)
Re: (Score:2)
Knowing some of the managers I have met, they would call you up on the phone, multiple times a day and if you'd answer, "HOW DARE YOU ANSWER THE PHONE!? DON'T YOU KNOW THAT THAT CAUSES BUGS!?" However, if you ignore the calls, you will get a face to face meeting demanding why you are avoiding them.
Managers, can't work with them, can't get paid without them!
test it on (Score:2)
Wiring? (Score:2)
Should we tell whoever wrote the headline what 'wiring' actually means?
For me it's reversed (Score:2)
Years ago I discovered that I used to slow down and struggle with simple things, soon after making a mistake (a bug) that I obviously missed. So I learned to go back and review my most recent code whenever I sensed that I began struggling. It appears that in those cases I subconsciously registered the mistakes. Of course you can also introduce bugs while happily coding along, but whenever you falter for no apparent reason, just review your most recent code for mistakes.
Think defensively (Score:2)
Missing the point (Score:2)
Personally I didn't find a correlation between how I struggle writing code and the number of bugs I produce. I'm not sure about why but it's probably because I'm more careful with hard problems.
Anyways, I think the worst bug-generator is code that is hard to test, or not tested enough because of time constraints or poor management. This can explain the correlation : complex code (lots of inputs, lots of steps) tends to make programmers struggle and is hard to test. Also, programmers having a hard time tends
90% of My Bugs... (Score:1)