Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

NHTSA Has No Software Engineers To Analyze Toyota

kdawson posted more than 4 years ago | from the analog-agency-in-a-digital-world dept.

Government 459

thecarchik writes "An official from the National Highway Traffic Safety Administration told investigators that the agency doesn't employ any electrical engineers or software engineers, leaving them woefully unable to investigate correctly what caused the most recent Toyota recall. A modern luxury car has something close to 100 million lines of software code in it, running on 70 to 100 microprocessors. And according to consultant Frost & Sullivan, that number will rise to 200 to 300 million lines within a few years. And the software that controls the 'drive-by-wire' accelerators of Toyota and Lexus vehicles is one potential culprit in the tangled collection of issues, allegations, and recalls of many of those vehicles for so-called 'sudden acceleration' problems."

cancel ×

459 comments

Sorry! There are no comments related to the filter you selected.

With all the recent US layoffs ... (4, Informative)

WrongSizeGlass (838941) | more than 4 years ago | (#31250506)

... there is plenty of talent out there for them to hire - even if only on a project by project basis.

Re:With all the recent US layoffs ... (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31250524)

Well I don't know if "talent" is the right word. The people who get laid off are the worst 10%. Usually the real slackers.

Re:With all the recent US layoffs ... (0)

Anonymous Coward | more than 4 years ago | (#31250668)

[citation needed]

Re:With all the recent US layoffs ... (1, Funny)

bigredradio (631970) | more than 4 years ago | (#31250696)

So you are saying they would be perfect for a Government job.

List of software powered cars (1, Insightful)

Anonymous Coward | more than 4 years ago | (#31251018)

Slashdot, please provide a list of software-powered cars so I know which cars to avoid like the plauge.

Seriously, most software out there is so poor quality I don't want to run it outside of a VM. I really do not want my life to depend on software...

Re:List of software powered cars (4, Informative)

Chyeld (713439) | more than 4 years ago | (#31251226)

Anything street legal without a needing a special waiver for emissions.

Re:With all the recent US layoffs ... (-1, Troll)

cxx (1749498) | more than 4 years ago | (#31251072)

Finally, a way for Obama to actually create jobs!

Re:With all the recent US layoffs ... (2, Insightful)

Marxist Hacker 42 (638312) | more than 4 years ago | (#31250738)

I know plenty who are laid off for other reasons- such as their C-level executives being slackers and the whole bloody company going under.

Re:With all the recent US layoffs ... (1, Funny)

WrongSizeGlass (838941) | more than 4 years ago | (#31250792)

Simple AI: If a question ends in a vowel, the answer is no. Otherwise the answer is yes.

Question: Is your signature true?

Re:With all the recent US layoffs ... (0)

Anonymous Coward | more than 4 years ago | (#31250914)

Yes.

Am I lying?

Re:With all the recent US layoffs ... (3, Funny)

Marxist Hacker 42 (638312) | more than 4 years ago | (#31251118)

No

Re:With all the recent US layoffs ... (2, Funny)

ottothecow (600101) | more than 4 years ago | (#31251206)

Are you a liar?

Re:With all the recent US layoffs ... (0, Troll)

FatAlb3rt (533682) | more than 4 years ago | (#31251262)

Do you have an odd physical attraction to goats?

Re:With all the recent US layoffs ... (2, Insightful)

ryanvm (247662) | more than 4 years ago | (#31250918)

Not in the auto industry - it's mostly union. There is no correlation between ability and likelihood of employment.

Real talk. (-1, Troll)

Singularity42 (1658297) | more than 4 years ago | (#31250714)

Hate to break it to you, but you got "laid off" because you were probably useless even if you worked for free. You got fired. Try nursing home aide.

It's a lot easier to blame the economy than admit your own shortcomings, isn't it?

Huh! (1)

oldhack (1037484) | more than 4 years ago | (#31250508)

I think I met couple EEs at NHTSA back in the 90s...

Re:Huh! (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31250522)

That was back when that philandering commie Clinton was in the White House. Since then we've learned that the government that governs least governs best. Just look at Wall Street!

Re:Huh! (0)

Anonymous Coward | more than 4 years ago | (#31250750)

Couldn't agree more if wall street knew it wouldn't be bailed out from the bad investments by the government they wouldn't make those investments.

Re:Huh! (0)

Anonymous Coward | more than 4 years ago | (#31250856)

how'd that work out for Bear Stearns?

Re:Huh! (4, Interesting)

megamerican (1073936) | more than 4 years ago | (#31251054)

If the NHTSA didn't exist Toyota would have had to spend money to fix the problem instead of paying ex-regulators [dailyfinance.com] to quash multiple investigations.

Toyota (TM) hired ex-government regulators to kill at least four investigations into problems with its cars in the U.S. That's the conclusion of an investigation by Bloomberg. The news service reports that, "Christopher Tinto, vice president of regulatory affairs in Toyota's Washington office, and Christopher Santucci, who works for Tinto, helped persuade the National Highway Traffic Safety Administration to end probes including those of 2002-2003 Toyota Camrys and Solaras, court documents show. Both men joined Toyota directly from NHTSA, Tinto in 1994 and Santucci in 2003. "

The same goes for Wall Street. Most of the financial regulators are former high level executives from Goldman Sachs or strong ties to them and other financial institutions.

I don't understand why we need so many useless regulators who are usually wolves being put in charge of the hen house when the courts could easily handle this. It's going to end up being prosecuted in a court of law anyway and not solved by some magic regulation hand-waving.

Here come the shackles. (4, Interesting)

HungWeiLo (250320) | more than 4 years ago | (#31250518)

Here comes DO-178B for cars.

I wonder what the cost is per line of code?

consultants (3, Insightful)

N7DR (536428) | more than 4 years ago | (#31250532)

Surely it would be a serious inefficiency for NHTSA to maintain on staff a large number of specialists to handle this kind of problem? Isn't that exactly what (properly qualified) consultants are for?

Re:consultants (4, Informative)

Hatta (162192) | more than 4 years ago | (#31250808)

Given how much of our vehicles are run by computer, I don't think there should ever be a lack of demand for software engineers at the NHTSA.

Re:consultants (2, Insightful)

TubeSteak (669689) | more than 4 years ago | (#31250888)

Surely it would be a serious inefficiency for NHTSA to maintain on staff a large number of specialists to handle this kind of problem? Isn't that exactly what (properly qualified) consultants are for?

I agree that it'd be inefficienct to have a large number of EEs & SEs on staff, but they have no one to do even a simple sanity check on the hardware and software that is being certified for public roads. And that strikes me as a failure of their organizational mission.

Re:consultants (3, Insightful)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#31251200)

If 100,000,000 LoC is common(albeit probably concentrated in more or less irrelevant things like the fancy display console, rather than the ECU) there is no such thing as a simple sanity check... And new cars and models are coming out all the time, from a variety of manufacturers, who are presumably constantly tweaking.

Under the circumstances, you pretty much have two options. The radical, future-looking one is to say "Ok, clearly complex software is the future. We are going to do whatever it takes, build up a serious software engineering team, impose standards that would make medical device makers cry, sponsor research in automated verification, whatever. Yeah, it sucks that we have do deal with that complexity; but so it goes." The traditional conservative(and, much more likely to fit within your budget and not ruffle feathers) option is to throw up your hands and treat the software as a black box. Have your existing test engineers use their existing techniques, or limited variants, to run the vehicles through test conditions, hoping that, if the test conditions effectively model the real world, any real world critical bugs will appear in testing, at which point you can kick it back to the people who wrote the code and tell them to fix it.

It seems pretty clear that the NHTSA has pretty much gone with option two. And, frankly, it is hard to blame them under the circumstances. Even at the best of times, technical regulation is a pretty unsexy legislative priority, and tends to be funded accordingly. It wouldn't take an actively antiregulatory corporatist to raise an eyebrow at a request for the sort of resources that you'd need to seriously audit the code in each new car coming off the line. And, if you don't have the resources to properly evaluate code from a CS or formal verification perspective, empirical black-box testing under real world-ish conditions is about the best you can do.

Re:consultants (4, Insightful)

sjames (1099) | more than 4 years ago | (#31250966)

Given that there isn't a car made today whose safety can be properly evaluated without the skills of EE and software engineers, why would it be inefficient for the agency responsible for that evaluation to have people with those skills on staff? It's not like next years cars won't have even more of the same complete with modified firmware to examine.

Given that the safety evaluation will involve interactions between mechanical, electrical and software systems, you'd want a cohesive multi-disciplinary team, not a revolving door.

Re:consultants (4, Interesting)

rainmayun (842754) | more than 4 years ago | (#31251128)

I can promise you have independent verification and validation contracts are bread & butter in the federal contracting world. The federal government has made huge strides in the direction of outsourcing almost all technical expertise, and quite a bit of management expertise (google "federal PMO contracts" for lots of random examples). The few civil servants left in many agencies are a kind of sheepherders, managing vast groups of contractors.

Rise of the machines (1)

Senes (928228) | more than 4 years ago | (#31250538)

Al Bundy: what do you mean I can't get out?
Clerk: I'm sorry, sir, the computer controls the doors too.

Welp (3, Interesting)

Pojut (1027544) | more than 4 years ago | (#31250558)

Such is the cost of more complicated technology. Although, I will admit, this problem seems awfully widespread for Toyota to have not caught this at some point in their QC/QA process.

I'm reminded of the "recall" speech in Fight Club...

Re:Welp (4, Funny)

Monkeedude1212 (1560403) | more than 4 years ago | (#31250636)

Which car company do work for?

A major one.

Re:Welp (0, Redundant)

mcgrew (92797) | more than 4 years ago | (#31250806)

Must not be General Motors or General electric then, it's been a long time since they were promoted from major.

Re:Welp (1)

jadin (65295) | more than 4 years ago | (#31251036)

Which is exactly why they are under investigation - to find out when they knew about the problem. If they waited until the cost justified the recall, they could be in trouble.

Heads better roll (4, Funny)

dave562 (969951) | more than 4 years ago | (#31250576)

If the statement in the article is true then this country is in even worse shape than I thought. It seems like rarely a handful of months can go by without the realization that yet another Federal department is completely incompetent. How in the hell does the NHTSA even do their job?! They are supposed to ensure that vehicles are safe but they don't even have the staff to do that.

What the hell is wrong with our country?

Re:Heads better roll (0, Flamebait)

mcgrew (92797) | more than 4 years ago | (#31250694)

What the hell is wrong with our country?

Eight years of Bush and a year of Obama. The heads of these agencies report to the President, so if those agencies fail, the fault lies with him. Four years of Ryan and six years of Blago has pretty much ruined Illinois. It's hard to say if we'll ever recover.

Re:Heads better roll (2, Insightful)

dave562 (969951) | more than 4 years ago | (#31250770)

It has to be deeper than just the President. The NHTSA lacking EE's and SE's is institutionalized fail. They don't even have the talent to meet their mandate. It required a full blown Congressional investigation into dozens of fatalities for someone to stand up and basically say, "By the way, we can't do our job."

Re:Heads better roll (1)

kalirion (728907) | more than 4 years ago | (#31250698)

I never even know NHTSA existed.

Re:Heads better roll (1)

AndersOSU (873247) | more than 4 years ago | (#31250998)

ever heard of a "4 star" or "5 star" crash rating? Then you've heard of the NHTSA.

Re:Heads better roll (2, Insightful)

happy_place (632005) | more than 4 years ago | (#31250746)

Q. What's wrong with our country? A. The price to make you perfectly safe, six times over, is prohibitively expensive. This seems like a stupid approach to the issue. I mean, just how many engineers need to be hired to make you feel safe? And exactly how do they test all 200 million lines of code? If Toyota's engineers missed something like this, do you honestly think that the government is going to magically find it? It's not like Toyota engineers did this sort of thing on purpose. They made a mistake. It's now costing lives. That's killing Toyota too.

Re:Heads better roll (4, Insightful)

dr2chase (653338) | more than 4 years ago | (#31251010)

Big picture, it's not costing that many lives. Bad drivers are much deadlier, and simply sitting on your butt in the car and not getting enough exercise is deadlier yet.

Re:Heads better roll (4, Insightful)

Areyoukiddingme (1289470) | more than 4 years ago | (#31251024)

I mean, just how many engineers need to be hired to make you feel safe?

One would be a good start. Oh hell, let's get wild and crazy and say.. 2.

Certainly more than zero.

Re:Heads better roll (1)

cxx (1749498) | more than 4 years ago | (#31251150)

I'd even be happy with a lone consultant, just to advise them. It's a start, at least.

Re:Heads better roll (3, Insightful)

rainmayun (842754) | more than 4 years ago | (#31251270)

The government's job in this case is not to duplicate the testing done by Toyota engineers, but rather to provide oversight and verify that Toyota's engineers ARE doing it, to a degree of completeness and correctness that satisfies statutes and regulations. Clearly that task requires substantial technical expertise, but it's not the same task.

Re:Heads better roll (5, Insightful)

tonywong (96839) | more than 4 years ago | (#31250754)

What is wrong is that everyone started believing the mantra that smaller government is better government. This isn't just limited to the United States.

In Canada, the province where I live (Alberta), derives a major part of its revenues from oil and gas. In the same conservative government 35 years ago, we had 2 independent arms of the government who could determine how much royalties were owed to the government from the oil and gas producers.

Today, we have no one in our government who is able to determine how much we should be collecting and therefore have to rely upon the oil and gas companies to tell use how much they are supposed to remit. Our own government auditor believes we have been bilked out of billions yet somehow we have a leaner and, ahem, more efficient government.

Just remember that the only thing to stand up to a big business nowadays is big government, and the goal of any big business is to convince everyone that a small government can watch over big business just like a big government can.

Re:Heads better roll (2, Insightful)

The End Of Days (1243248) | more than 4 years ago | (#31251052)

The problem, of course, is that nothing can stand up to big government. That's a tiny problem though, it's not like the government would ever abuse its power to grab control of the citizenry, right?

Re:Heads better roll (2, Insightful)

Chris Mattern (191822) | more than 4 years ago | (#31251116)

Just remember that the only thing to stand up to a big business nowadays is big government

I'm sorry, but if you think the antidote to big business is big government, you're delusional. Big government is big business's *partner*. It's always been that way, and it'll always be that way. Handing government more power means that there will be plenty of regulations. You *do* know that a regulation-heavy environment favors big business, not small business, right? Small business can't afford the compliance department you need.

Re:Heads better roll (1)

jeffmeden (135043) | more than 4 years ago | (#31250786)

Are you kidding? NHTSA sanctions the testing, develops *some* of the test protocols and performs *some* investigative work to identify problems. Their best strategy to create/keep cars safe in the US is to make sure the manufacturers go through the right processes in creating them. Does that mean having code auditors at the NHTSA looking over the shoulders of programmers at all the car manufacturers? I don't think it does. Does it mean the NHTSA should mandate auto makers to do rigorous code audits of their code, possibly with third party consultants? That sounds a lot more practical. The NHTSA (along with most of the government) should be working *smarter*, not harder or bigger (read: more expensively).

Re:Heads better roll (1)

dave562 (969951) | more than 4 years ago | (#31250882)

How are they developing effective tests without engineering talent to guide the creation of those tests? How are they validating simulated tests if they don't even have the theoretical and practical knowledge that engineers would give them? It isn't like the NHTSA should be doing all of the testing or code audits for the auto makers. However they should have some talent on hand so that when Toyota says, "It isn't the electronics.", someone at the NHTSA can begin to verify it.

Re:Heads better roll (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#31250794)

If the statement in the article is true then this country is in even worse shape than I thought. It seems like rarely a handful of months can go by without the realization that yet another Federal department is completely incompetent. How in the hell does the NHTSA even do their job?! They are supposed to ensure that vehicles are safe but they don't even have the staff to do that.

What the hell is wrong with our country?

I can only assume that a bunch of fucking retarded Eurotrash modded that funny.

Re:Heads better roll (5, Interesting)

je ne sais quoi (987177) | more than 4 years ago | (#31251002)

I don't why I even respond because I'm sure to get a troll mod but I'd just like to point out that one of the major political parties solution to bad government is no government at all. This poorly functioning government is a direct result of the dual conservative mantras: 1) deregulation of markets is necessary for them to perform well and 2) less government is better. We saw how well #1 worked in the banking industry, this is more of the same. #2 results in chronically understaffed government agencies, or government agencies not able to do what they're supposed to do (e.g. the Republican senators holding up Obama's appointees right now).

My parents both worked for the FDA and if the NHTSA operates in any similar way to the FDA, it's a shadow of itself in the 1970s. For the FDA that means that there are less food inspectors and no surprise, there is a rise in food poisoning incidents. I wouldn't be surprised if NHTSA is also chronically understaffed. Additionally, even if individual government workers wanted to do their jobs, they are often prevented by doing so because that is not perceived as "business friendly". The political appointees who run the show are in the thrall of private industry, in fact, they are often people taken directly from private industry (e.g. big pharma lobbyists often run the FDA). This "government capture" is the fault of the democrats just as much as the republicans, e.g. Obama lied about hiring lobbyists in his campaign. Basically, we have a non-functioning government and one party's answer to this is the get rid of the thing all together. That is one solution but that wouldn't prevent things like this incident with Toyota.

I'm sure Toyota will do the right thing though, because that would be in its interests as a good corporate citizen. *snicker*

Re:Heads better roll (0, Offtopic)

dave562 (969951) | more than 4 years ago | (#31251164)

Completely off topic, but have you ever spoken to your parents about the way the FDA handles vitamins and supplements and their seeming propensity to force anything off of the market that threatens pharmaceutical interests?

The following article discusses the FDA's handling of L-Tryptophan because it produced similar clinically observed effects as Prozac and other SSRIs.

http://www.qhi.co.uk/features/feat_002.asp [qhi.co.uk]

Re:Heads better roll (4, Insightful)

wisnoskij (1206448) | more than 4 years ago | (#31251106)

"How in the hell does the NHTSA even do their job?"
Like every other safety certification organization. The car companies pay for a certificate, NHTSA takes some of the blame when something happens, and the general population feels safe knowing their is an entire organization dedicated to protecting them.

Re:Heads better roll (1)

SilentStaid (1474575) | more than 4 years ago | (#31251218)

I don't have any points, but I'd just like to go on record and say that this should be modded Insightful. Not funny.

Re:Heads better roll (4, Informative)

eh2o (471262) | more than 4 years ago | (#31251266)

Years of deregulation and resource starvation have strangulated our regulatory agencies to the point where they are unable to act.

Much of this based on Greenspan-style Libertarian philosophies that market forces can correct any problem including fraud and crime, a position which he himself has now renounced and we as a people have yet to heed.

Since the late 80s we have been riding on a giant ponzi scheme and its all coming crashing down right now. And yet, nothing. I expect things to get much worse.

Computer Engineers needed (4, Insightful)

HalWasRight (857007) | more than 4 years ago | (#31250588)

They don't need Electrial Engineers or Software Engineers. They need Computer Engineers [wikipedia.org] , people who are trained to understand both sides of the hardware/software boundary.

Re:Computer Engineers needed (2, Funny)

Deltaspectre (796409) | more than 4 years ago | (#31250640)

Even better, this one [mattel.com] only costs $12.99!

Re:Computer Engineers needed (1)

oldhack (1037484) | more than 4 years ago | (#31250682)

Dude, shut up. The one after you will call out for embedded systems engineer. The one after that will call out for ECM/TCM engineer.

Re:Computer Engineers needed (2, Insightful)

Anonymous Coward | more than 4 years ago | (#31250732)

Speaking as someone with a CMPE degree, employers see me as under-qualified to do EE work and over qualified to do programming work. What they need is either EEs with heavy embedded programming experience or software engineers with (guess what) embedded programming experience. The title isn't that important.

Re:Computer Engineers needed (0)

Anonymous Coward | more than 4 years ago | (#31250934)

In my experience, a spork [wikipedia.org] is not superior to a fork or a spoon. I've found the same tends to hold true across EE, SE, and CEs.

Re:Computer Engineers needed (0)

Anonymous Coward | more than 4 years ago | (#31251080)

But it is better than fork at being a spoon and better than a spoon at being a fork.

Re:Computer Engineers needed (1)

WrongSizeGlass (838941) | more than 4 years ago | (#31250956)

Based on a recent news article I believe Woz would be able to help them out with at least one of the Toyota problems.

Re:Computer Engineers needed (0)

Anonymous Coward | more than 4 years ago | (#31250990)

yeah, great job EDS did for GM embedded.....

Re:Computer Engineers needed (0)

Anonymous Coward | more than 4 years ago | (#31251066)

They don't need Electrial Engineers or Software Engineers. They need Computer Engineers [wikipedia.org] , people who are trained to understand both sides of the hardware/software boundary.

They don't need any engineers. I have it on good confidence that the entire accelerator problem is related to the floor mats.

Check please!

100 million lines of code?? (2, Insightful)

Anonymous Coward | more than 4 years ago | (#31250628)

I find that extremely hard to believe. Jurassic Park ran on just two million lines of code. I doubt all the lifetime output of all the readers of this thread, combined, equals 100 million. I further doubt that such complexity is remotely necessary to run a car, and that it is remotely possible to debug that much complexity to the standards of, say, the airline industry. And that NHTSA could audit that code in any respectable amount of time. I hope beyond hope the number is wrong.

Re:100 million lines of code?? (4, Informative)

quantumplacet (1195335) | more than 4 years ago | (#31250712)

Re:100 million lines of code?? (1)

hansamurai (907719) | more than 4 years ago | (#31250968)

What a revealing article:

The F-22 Raptor has 1.7 million lines, the F-35 about 5.7 million, and a 787 has 6.5 million lines, but somehow a consumer automobile needs 100 million?

I'm honestly surprised this is the first major incident.

Re:100 million lines of code?? (1)

WrongSizeGlass (838941) | more than 4 years ago | (#31250752)

I doubt all the lifetime output of all the readers of this thread, combined, equals 100 million.

Surely you jest ... or you've been favorably sheltered from our endless verbosity, pedantic ramblings and self-serving diatribes.

Dr Zoidberg: Loot at me, I'm helping!

software TOYOTA? (1)

fregare (923563) | more than 4 years ago | (#31250632)

What is this thing called software and what is a TOYOTA?

100 million lines? Sure, we will get right on it (4, Insightful)

jeffmeden (135043) | more than 4 years ago | (#31250674)

What exactly would the NHTSA do with a set of engineers? Audit all 100 million lines of code for each and every car they suspect has a safety issue with the computer system? Yeah, that sounds like a worthwhile endeavor. How about they do it the old fashioned way; collect the reports, identify the risk, and sanction the manufacturer to find/fix the problem. Thinking that an NHTSA coder (or a hundred) would have gotten to the bottom of this Toyota issue in any reasonable amount of time is a joke!

Re:100 million lines? Sure, we will get right on i (1)

jimbolauski (882977) | more than 4 years ago | (#31250832)

Just look at any large software company they have people looking through the code and bugs are still found, if the bug was easy to find TOYOTA would have found it. The last thing we need in NHTSA injecting itself into the coding process.

Re:100 million lines? Sure, we will get right on i (1)

rotide (1015173) | more than 4 years ago | (#31250854)

Well, if you don't know what you're asking for, how would you ever know if the answer they give you is even close to reality?

"Hey, I need you to investigate x, I have no idea how to even analyze x, but I trust you will investigate it exhaustively!"

"Sure, we fully investigated x and it's fine."

"Oh, ok, we'll take your word for it, thanks!"

You have to at least be able to understand what's going on to a certain degree before you can tell someone to fully investigate it _and_ then trust their results.

So yes, they should have a set of engineers who can read code well enough to know what is doing what and ask a company to exhaustively test it.

Finally, 100 million lines of code sounds like an awful lot of code for a throttle and/or braking system. I have a feeling that number is bloated to include things like when to pop on the low fuel light or seatbelt warning sounds. Pretty sure you can whittle that 100 million down at least 50 if not 95% and figure out what code actually controls the systems being reported as an issue.

In short, yes, if you're going to be educated in the field of vehicle safety, you can't claim ignorance to the _whole_ command and control system that lies in the computers that have existed in cars for more than a decade.

Re:100 million lines? Sure, we will get right on i (1)

jeffmeden (135043) | more than 4 years ago | (#31251102)

Having worked (and been) a coder, I can tell you the last thing that would be productive is for the phone to ring at Toyota and for an NHTSA software engineer to go "hey guys, check out line 213343, I think you forgot to call the destruct method on that instantiation before the function closed, I bet that's why your cars are crashing!"

One more (or a hundred more) sets of eyes isn't the solution, the solution is better coding *practices* along with better testing. In short, the NHTSA needs QA and Project Management types to sort through the steps that led up to the bug being introduced. No one seems to want to comment on how many of those they have (or what they are busy doing). There may well be an understaffing (or improper-staffing) at the NHTSA, but saying "oh god theres no coders get them some coders!!!" is *not going to help*.

Re:100 million lines? Sure, we will get right on i (0)

Anonymous Coward | more than 4 years ago | (#31250900)

Amen!

Re:100 million lines? Sure, we will get right on i (1)

keithjr (1091829) | more than 4 years ago | (#31250954)

Who's going to identify the write said reports, and identify the risks? Are you trusting Toyota to do this in-house? Because the article shows the NHTSA has zero qualifications do any diligence on its own.

A line-by-line audit is silly, and nobody is suggesting this. However, I can't see why the department that oversees embedded systems (automobiles) has no electrical engineering talent on hand.

Re:100 million lines? Sure, we will get right on i (1)

scourfish (573542) | more than 4 years ago | (#31251028)

Most of that code is probably autogenerated from some control scheme in a Simulink-type toolchain. There are other ways to audit than looking straight at the microcontroller code, to that regard.

Re:100 million lines? Sure, we will get right on i (1)

mswhippingboy (754599) | more than 4 years ago | (#31251256)

100 Million??? Really? What the hell is it written in, Intel 4004 assembler code?

Re:100 million lines? Sure, we will get right on i (1)

cxx (1749498) | more than 4 years ago | (#31251260)

Exactly -- that would be worthless. Rather, they need an advisory panel that can examine the QA practices and such.

I'm used to examining million LOC codebases -- give me or anyone else here on ./ a few days to look at their procedures, bug database, unit tests, etc., and we'll be able to tell whether this sort of problem could occur again and what was done to solve it. But I wouldn't/couldn't do the testing myself, ever.

Re:100 million lines? Sure, we will get right on i (0)

Anonymous Coward | more than 4 years ago | (#31251272)

Or, instead of what you said, we could just ask NHTSA to get with DOD and find out how they manage software (hint: IEEE 12207 or, if you're old school, MIL-STD-498).

Re:100 million lines? Sure, we will get right on i (1)

SeattleGameboy (641456) | more than 4 years ago | (#31251280)

Last time I checked, there were something like 1500 or so complaints about sudden acceleartion filed. They recalled 8 million cars, but if you include every model with the complaint, you are probably looking at 4X or 5X of that number. Even if you stick with 8 million, 1500 out if 8 million is 0.019%. Good luck trying to reproduce a problem that has a reproducible rate of 0.019%.

How many microprocessors was that again? (2, Insightful)

jdgoulden (1575977) | more than 4 years ago | (#31250718)

70 to 100 microprocessors? I imagine that this is true only if you employ a fairly broad definition of "microprocessor" and note that the vast majority are single-purpose devices in self-contained systems. I doubt that the "microprocessors" and "lines of code" that run the stereo or the climate-control system - or even the airbags - have any connection with the driveline.

Re:How many microprocessors was that again? (1)

sanosuke001 (640243) | more than 4 years ago | (#31250834)

Your code doesn't share global temp variables between packages?

Re:How many microprocessors was that again? (1, Interesting)

Anonymous Coward | more than 4 years ago | (#31250922)

Certainly not a direct connection but the automobile industry does use CAN (Controller Area Network) to link many systems in the car, it is a shared bus to so say that a bug in one system couldn't effect another may not be entirely accurate.

One would hope mission critical system have separate buses but with a safety administration with no ability to check, who knows?

Re:How many microprocessors was that again? (0)

Anonymous Coward | more than 4 years ago | (#31250950)

Perhaps the definition is a little broad, but it is not incorrect.
I have a radio I am working with on the desk next to me and I count 5 microprocessors, one of them a dual core (OMAP).
This isn't even a navigation system. It is just a radio. And I'm not counting the microprocessors that certainly exist in the XM module and CD mech (probably 2 or 3).

These micros are mostly single-purpose as you say and relatively low power (some of them are low-power ARM micros, others run at a sub-100 MHz range). This doesn't mean they aren't microprocessors. Any one of them has more power and memory than desktop computer from 20 years ago. The entire radio probably has more processing power than was used to send a man to the moon in the 60's.

Microsoft Hotline (2, Funny)

imscarr (246204) | more than 4 years ago | (#31250724)

Can't they just call Microsoft's toll-free number and ask someone over there to look at it?

100 million lines of code? (1)

understress (85878) | more than 4 years ago | (#31250802)

I didn't RTFA, but I've seen the comment about a modern car having something like 100 million lines of code in articles before. Now, I am not in any way qualified to say that number is to large or to small. But as an embedded systems software developer, that seems like an INSANE amount of code. I'm the manager of the engineering department at my employer (small manufacturer in US) and I have very strict requirements for comments in code. Even if you count the lines of comments in our code (probably around 50% of the file content), our largest project to date is around 35,000 lines of C code. Now I realize that since we are targeting smaller 8 bit MCU's with limited resources, this limits what we can do.

But still, 100 MILLION lines of code? Does anyone have any input on whether or not this is accurate? Or do automotive software engineers like to comment their code more than anyone else?

Re:100 million lines of code? (1)

Sir_Lewk (967686) | more than 4 years ago | (#31250920)

Previously I would have suspected that absurd bloat like that would have been the result of bureaucratic NHTSA regulations, but obviously that's not the case...

Re:100 million lines of code? (1)

Logical Zebra (1423045) | more than 4 years ago | (#31251144)

Maybe that 100 million figure includes things such as multimedia, navigation, etc. Even then, it seems large to me.

Re:100 million lines of code? (1)

vlm (69642) | more than 4 years ago | (#31251188)

But still, 100 MILLION lines of code? Does anyone have any input on whether or not this is accurate? Or do automotive software engineers like to comment their code more than anyone else?

You're thinking of "the car" as the engine computer, transmission comp, and the ABS comp.

They're journalists, and they're counting the rear set DVD player, the GPS display, the onboard cellphone/big brother tracking device that unlocks the doors...

Re:100 million lines of code? (0)

Anonymous Coward | more than 4 years ago | (#31251234)

This is bull. There is no way a luxury car has 100 million lines of code in it, i don't even believe it executes 100 million lines of code for the entire life of the car. I'm a software developer and have been writing code for quite some time now and the metric my company uses is 4 lines of code per hour. Which seem like a snails pace to write code, but that metric takes into account, meeting, sick days, code reviews and every other distraction that keep you from coding. So (and check my math) if you've got a team of 500 developers (which would be the biggest team I've ever heard of for one development effort), it would take 24 years to write 100 million lines of code. Feel free to correct me if I'm wrong.

Re:100 million lines of code? (2, Interesting)

mcgrew (92797) | more than 4 years ago | (#31251258)

I've seen the comment about a modern car having something like 100 million lines of code in articles before. Now, I am not in any way qualified to say that number is to large or to small. But as an embedded systems software developer, that seems like an INSANE amount of code.

Someone posted a link to this article [ieee.org] that confirms it. I can't find the comment with the link; someone must have modded him down past my threshhold. But the article linked itself confirms that it is indeed an insane amount of code, insanely implimented.

The avionics system in the F-22 Raptor, the current U.S. Air Force frontline jet fighter, consists of about 1.7 million lines of software code. The F-35 Joint Strike Fighter, scheduled to become operational in 2010, will require about 5.7 million lines of code to operate its onboard systems. And Boeing's new 787 Dreamliner, scheduled to be delivered to customers in 2010, requires about 6.5 million lines of software code to operate its avionics and onboard support systems.

These are impressive amounts of software, yet if you bought a premium-class automobile recently, "it probably contains close to 100 million lines of software code," says Manfred Broy, a professor of informatics at Technical University, Munich, and a leading expert on software in cars. All that software executes on 70 to 100 microprocessor-based electronic control units (ECUs) networked throughout the body of your car.

It gets worse.

And unlike most commercial aircraft, which have strict firewalls between critical avionic systems and the in-flight entertainment systems, there is more commingling of information between the electronic systems used to operate the car and those for entertaining the driver and passengers. According to a Wharton Business School article entitled "Car Trouble: Should We Recall the U.S. Auto Industry?," a few years ago, some Mercedes drivers found that their seats moved if they pushed a certain button; the problem was that the button was supposed to operate the navigation system.

100 microprocessors? (0)

Anonymous Coward | more than 4 years ago | (#31250810)

I don't believe it. In WHAT?

I can't come up with a list of 100 things in a car that it makes sense to have a microprocessor for.

Are they counting stuff like the radio, the gps, the dvd players in the seat backs? None of that stuff has to do keeping the engine running, and doesn't need to be considered for safety purpouses.

Why would you need more than one computer to control the car anyway? I guess you might want a seperate one to control the airbags in case the crash is caused by the main one failing, but other than that I don't see why you need more than one CPU to control the engine, check the brake fluid, tire pressure, etc.

Re:100 microprocessors? (1)

X0563511 (793323) | more than 4 years ago | (#31251156)

Because it isn't a "computer" that controlls the parts of the cars. Each microcontroller has jobs it does. This is more efficient and "safe" than a single monolithic computer.

Note that ECU actually stands for Engine Control Unit. Looks similar to CPU, but it ain't.

Sounds like the government needs to be updated (1)

erroneus (253617) | more than 4 years ago | (#31250866)

Seriously. How did they not see this coming. They have been hearing cases about secret codes and OBD standards and the like for quite some time now. The fact that cars are running with the added use and assistance of digital computational systems is well known. If they are not equipped to do testing for safety purposes, they are simply not equipped to do their jobs. And I'm afraid to ask about air vehicle safety testing now...

Why would they? (1)

Gadgetfreak (97865) | more than 4 years ago | (#31250940)

They respond to problems, they don't reverse engineer things. Does the FDA or the Surgeon General's office have engineers to paw through the lines of code in MRI machines or CT scanners, or anesthesia machines, or respirators, or any other number of computerized medical machines? No... they get tested emperically, just like cars do. It's very difficult to prove that some of these flaws exist.... remember the Audi "sudden acceleration" problems in the late '80s that almost killed the brand? That was pre-computerized throttle and transmission, and STILL was impossible to prove. Audi made pedal spacing changes, but largely to avoid the inevitable suicide of doing 'nothing.'

Engineers or not, it's going to be quite difficult to prove that there's an actual "flaw" in the design, let alone negligence,when there are so many millions of vehicles without issue.

So What? (1)

PingPongBoy (303994) | more than 4 years ago | (#31250996)

Safety related functionality should have a redundant overriding mechanism that isn't subject to the vagaries of software failure. For example, if the engine computer suddenly wants to run an explode subroutine, the fuel valve should limit the outcome to chitty chitty bang bang.

Then you don't have to check every line of code, you just have to check the overrides.

This all story starts to look like swine flu (1)

stoev (103408) | more than 4 years ago | (#31251008)

Sure there are bugs in the code. Any code has bugs. ANY car has bugs. I have the feeling that somebody is making a black PR campaign to create panic to humble Toyota.
Same was with swine flu - somebody wanted a panic to sell more medicines. There was also SARS several years before that.

How many people died or were injured because of the claimed Toyota software bugs? Give me a number.

Re:This all story starts to look like swine flu (4, Interesting)

binarylarry (1338699) | more than 4 years ago | (#31251062)

http://abcnews.go.com/Blotter/RunawayToyotas/toyota-acceleration-problems-new-evidence-imprisoned-minnesota-toyota-camry-owner/story?id=9903455 [go.com]

This guy apparently killed a few people and got put in jail for it. Now it looks like he was telling the truth when he said the car wouldn't stop.

Re:This all story starts to look like swine flu (1)

Sicily1918 (912141) | more than 4 years ago | (#31251176)

The deaths are something like 13 or so -- not an incredibly high amount, but there's evidence that suggests (and that's not strong enough a word) that Toyota's been aware of this since about 2002 and has actively tried to stop any and all probes, lobbied against safety changes, and made bogus recalls (e.g., the floor mats) in order to positively affect their bottom line.

100 million lines??? (0, Redundant)

Tomfrh (719891) | more than 4 years ago | (#31251038)

IANASE, but 100 million lines of code sounds a little over the top. Can someone verify this?

Re:100 million lines??? (0)

Anonymous Coward | more than 4 years ago | (#31251142)

I'm will to get that the vast majority of this code resides in the in dash navigation/stereo/do it all systems that most luxary cars have and thus isn't really "embedded code" in the mission critical microcontroller handling 1 or 2 functions sense.

This is the government, not an engineering firm (2, Insightful)

rm999 (775449) | more than 4 years ago | (#31251170)

I totally disagree: the NHTSA shouldn't hire engineers. NHTSA should not do the job of Toyota's engineers and testers; they were created to set policy and propose safety laws. The NHTSA should hire economists, policy makers, and maybe some scientists. But the job of ensuring the nuts and bolts of a car are safe should fall on the car-maker, with strict repercussions if they fail.

My biggest problem with all this is what people on Slashdot should already know: looking through and understanding millions of lines of code would take an engineer a few lifetimes - how many engineers are we proposing NHTSA hires? They could learn Toyota's software system, but then what about Ford cars? Or BMW? All for a government organization with 600 employees...

In cases like this, NHTSA should force Toyota to hire a third party (objective) consultant to create a technical report. Maybe a small team of engineers could remain on staff to read and understand those reports.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>