×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Carmack's Armadillo Aerospace Rocket Crashes and Burns

ScuttleMonkey posted more than 6 years ago | from the doomed-rockets dept.

Space 353

mcgrew (sm62704) writes "New Scientist is reporting that John Carmack's 'Armadillo Aerospace' has suffered a large setback in the Northrop Grumman Lunar Lander Challenge after one of its two main rockets crashed and burned. 'During the test, Texel lifted off and hovered without incident, then descended again and touched the ground. But it then rose again unexpectedly and began accelerating upward. "Crap, it's going to fly into the crane, I need to kill it," Carmack recalls thinking. He hit the manual shutdown switch, turning off the vehicle's engine in mid-flight. Texel was about 6 metres above the ground and fell like a stone. One of its fuel tanks broke open when it hit the ground, spewing fuel that ignited and engulfed the vehicle in flames. "It made a fireball that would make any Hollywood movie proud," Carmack says.' No one was hurt in the crash, but the vehicle was destroyed."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

353 comments

to boldly go.... (1, Flamebait)

downix (84795) | more than 6 years ago | (#20322111)

And to think, they want us all to ride in these things commercially....

Re:to boldly go.... (5, Insightful)

morgan_greywolf (835522) | more than 6 years ago | (#20322147)

And to think, they want us all to ride in these things commercially....
You wouldn't have wanted to fly in an airplane commercially had you been around in the days of the Wright Brothers and Kitty Hawk.

Bad comparison (5, Interesting)

EmbeddedJanitor (597831) | more than 6 years ago | (#20322551)

Significant rocketry started in the 1940s and space travel in the 1950s. That's over 50 years to get its shit together. Yet, in approx 120 launches the space shuttle program has lost two vehicles/crews in huge fireballs. If planes crashed that often LAX would have a crash before breakfast every morning.

Or, put another way... within 20 years of the Wright Brothers the airplane industry had far better safety records than the space industry does after 50 years.

Re:Bad comparison (5, Insightful)

veganboyjosh (896761) | more than 6 years ago | (#20322605)

The Wright Brothers also had lots of things to pull from that already flew. Birds, insects, etc. Hell, even those seeds that fall like little helicopter blades have natural wing shaped leaves to help them slow down/disperse away from the tree from which they could have gotten data.

There aren't any naturally occurring animals or phenomena from which to figure out space travel/launch/re-rentry. I'm not saying the safety record is stellar (yukyuk), but getting off the ground is a little less complex than getting off the planet (and back).

Re:Bad comparison (5, Insightful)

savuporo (658486) | more than 6 years ago | (#20323003)

Right, the difference is, aircraft followed the natural evolutionary approach to safe and economical transportation.

Space launchers have never done that. They have always tried to leapfrog to a "complete solution". Most of the launchers active today have their heritage in ICBMs. Apollo program got started by replacing the warheads with men in tin can. Thats not how you build a reliable and safe transportation device.

Or take shuttle. It was designed on paper, and the very first hardware iteration was declared operational configuration. Thats just nuts. You try to take and build worlds first ever reusable space transport, and you try to do it in one hardware iteration ? Try more like something between ten and hundred to get it right.

The trouble is, space industry has always been run by governments across the globe, due to certain historical circumstances. It never undertook the normal evolution of hardware and technologies that has happened with other, commercial transportation markets.

And thats exactly what Armadillo and their kin are trying to do now. Build stuff from the ground up, fly a bit, crash a few times, build it better and so on. Enter the competitive pressure of marketplace, and you will get the right incentives to build affordable, safe and reliable space transportation.

We dont know what these will turn out to be, whether its VTOL rockets like Armadillo and Masten are building, or XCOR HTOL approach, or something else entirely. This evolutionary path is yet to be walked down.

Re:Bad comparison (1)

moderatorrater (1095745) | more than 6 years ago | (#20323053)

If you compare amount of time, then yes. If you compare number of flights, than the space industry is far behind where the airplane industry was. Not only that, but the initial investment is much higher to get into space than it is to get into the air.

Re:to boldly go.... (2, Funny)

Anonymous Coward | more than 6 years ago | (#20322149)

I think it was Doomed from the start.

Re:to boldly go.... (4, Insightful)

Reality Master 101 (179095) | more than 6 years ago | (#20322225)

And to think, they want us all to ride in these things commercially....

Actually, this is exactly why John and company will be successful. The biggest problem with modern aerospace is "paralysis by analysis". They're so afraid of crashing anything that they have to produce (sometimes literally) millions of pages of documentation before they actually put something into the air.

Armadillo learns by *doing*, not just by creating paper studies. When they're ready to put humans in space, you can bet that their ships will have had hundreds of test flights.

Re:to boldly go.... (2, Insightful)

Anonymous Coward | more than 6 years ago | (#20322433)

Had Carmack's rocket killed someone (or many people), he would would have been stopped by "paralysis by lawsuit-ysis". Ignoring the huge dangers of rocketry by cutting corners during design may be cheaper in the short run, but as soon as real human lives are lost because of it, you can bet your ass they are going to have to spend more time and money testing their designs "on paper".

Re:to boldly go.... (3, Insightful)

Reality Master 101 (179095) | more than 6 years ago | (#20322509)

Had Carmack's rocket killed someone (or many people), he would would have been stopped by "paralysis by lawsuit-ysis". Ignoring the huge dangers of rocketry by cutting corners during design may be cheaper in the short run, but as soon as real human lives are lost because of it, you can bet your ass they are going to have to spend more time and money testing their designs "on paper".

The point isn't "cutting corners", the point is learning by testing and learning with actual hardware, rather than testing with paper. No one was in any danger at any point during this test. You would have a point if you could claim they were cutting corners in *safety culture*, but they're not. They're not strapping people into test vehicles. There is no human risk here at all.

Re:to boldly go.... (1)

XenoPhage (242134) | more than 6 years ago | (#20322611)

Had Carmack's rocket killed someone (or many people), he would would have been stopped by "paralysis by lawsuit-ysis". Ignoring the huge dangers of rocketry by cutting corners during design may be cheaper in the short run, but as soon as real human lives are lost because of it, you can bet your ass they are going to have to spend more time and money testing their designs "on paper".



The point isn't "cutting corners", the point is learning by testing and learning with actual hardware, rather than testing with paper. No one was in any danger at any point during this test. You would have a point if you could claim they were cutting corners in *safety culture*, but they're not. They're not strapping people into test vehicles. There is no human risk here at all.

If I may, doesn't anyone remember the recent explosion [slashdot.org] at Mojave that claimed 3 lives? While rocketry related, it wasn't a flight test. It was also a team with vastly more resources. Testing new technology has it's dangers, especially when highly combustible materials are involved.

John and his team have an excellent track record thus far, and have continued to make safety a main issue. I'm sure that this experience will teach them even more, helping to make the next flight even safer.

Re:to boldly go.... (5, Funny)

Tim Browse (9263) | more than 6 years ago | (#20322893)

I'm sure that this experience will teach them even more, helping to make the next flight even safer.

You mean even safer than a huge orange fireball?

I don't know, that's a pretty high bar.

Re:to boldly go.... (1)

Bluesman (104513) | more than 6 years ago | (#20322491)

If they're ready. Learning by doing has a tremendous cost associated with it when it means you're blowing up rockets in order to learn that your GPS guidance system has some bugs in it you didn't account for.

Frankly, it surprises the hell out of me that they're using GPS as the primary guidance system. Any amount of radio interference and you'll end up with problems. It might be ok once you get up in the air enough to correct for temporary loss of signal, but near the ground, yer screwed.

Re:to boldly go.... (1)

asc99c (938635) | more than 6 years ago | (#20322593)

The cost of blowing up a rocket is much lower if you've not had to produce thousands of pages of documents though. It obviously is quite a lot of hardware, but I don't really believe the actual hardware costs the tens of millions commonly quoted even for small rockets.

It is not for real (2, Interesting)

WindBourne (631190) | more than 6 years ago | (#20322635)

It is more for the competition. After that, you can bet that it will be changed. Though, I have to say, I have been think about the GPS units. It strikes me that if we are going to explore the moon and mars, we should be developing cheap GPS sats. to send there.

Re:to boldly go.... (1)

xenocide2 (231786) | more than 6 years ago | (#20322855)

The bigger part to me is where they replaced a part without checking whether it was a suitable replacement. 10G != 4G.

Re:to boldly go.... (4, Informative)

brainlessbob (973044) | more than 6 years ago | (#20322655)

This was how soviet was testing their rockets, by trial and error. They launched a prototype and then they looked how it flew and why it blew up if it did. Saw a documentary about soviet rocket engineering and in it some nasa guys said it was one of the mayor reasons why soviet was greatly ahead of USA in rockets.

Re:to boldly go.... (1)

peacefinder (469349) | more than 6 years ago | (#20322881)

Although it seems like they think a little more analysis and procedure documentation might have saved them a vehicle here.

I lurk on an amateur rocketry mailing list where a few AA personnel participate, and it's interesting to read about their opinion of lessons learned. Between this and the Scaled accident I think they'll be a bit more cautious and deliberate for a while.

It's a learning process (5, Insightful)

evanbd (210358) | more than 6 years ago | (#20322285)

So they're not there yet. Big deal. Armadillo's attitude to safety is that it's ok to risk the vehicle in testing, as long as people aren't at risk. They do a *very* fast development cycle, and they don't pretend to be able to find every problem through analysis -- which means some of them get found the hard way. That's a *good* thing for safety, not a bad thing. You *can't* find every problem through analysis, even if your budget is 5 orders of magnitude larger than Carmack's and you try.

Carmack's approach is to treat the vehicle as a developmental test platform, and that involves a certain level of risk to the vehicle and acceptance of that risk. The result, however, is that he learns things a *lot* faster than he otherwise might, and as a result the entire development program is faster and cheaper, counting the cost of the lost vehicles.

When Carmack shifts the vehicle from developmental status to operational testing status and then to operational status, I'd be happy to trust him when he says it's safe. It's unfair to criticize him for being unsafe now -- crashing the vehicle wasn't a safety risk!

Re:It's a learning process (0)

Anonymous Coward | more than 6 years ago | (#20322719)

So they're not there yet. Big deal. Armadillo's attitude to safety is that it's ok to risk the vehicle in testing, as long as people aren't at risk.

Could they have deployed a parachute or something to prevent such an obvious destruction of said vehicle? The worst part, they might not be able to analyze the cause of the failure if the machine is toasted.

Re:It's a learning process (1)

XenoPhage (242134) | more than 6 years ago | (#20322773)

Could they have deployed a parachute or something to prevent such an obvious destruction of said vehicle? The worst part, they might not be able to analyze the cause of the failure if the machine is toasted.

I don't believe the vehicle was high enough in the air for a parachute to have made any difference. By the time it would have deployed, the vehicle was on the ground.

Re:It's a learning process (2, Informative)

peacefinder (469349) | more than 6 years ago | (#20322907)

The vehicle fell from only about 20 feet. Much too low for a parachute. As the repost above [slashdot.org] says, their failure analysis is already pretty much complete. They know what went wrong to cause the condition that led to the fall.

There's an old saying that kernel engineers use (0)

Anonymous Coward | more than 6 years ago | (#20322799)

You are absolutely spot-on.

There's an old saying for these kinds of situations when developing a kernel. It's:

        Opps

Re:to boldly go.... (1)

MSFanBoi2 (930319) | more than 6 years ago | (#20322371)

Care to remember all the issues the space program had with its first rockets? This is still far from an exact science, as thanks to NASA, most of the civilian rocket crews for the large stuff is still playing guessing games.

They will get it... and when they do its going to be a whole new world. Literally.

Re:to boldly go.... (3, Funny)

Anonymous Coward | more than 6 years ago | (#20322409)

I'm with you. I'm with you 110%. This is EXACTLY why I have long opposed private spaceflight. Long story short, profit = cut corners = death. We saw it at the composites factory, and we'll see more of it. Private interests just do not have the long term perspective necessary to take the appropriate caution to prevent deaths. This is why space colonization should always be a government function.

Would NASA cut corners like this and end up killing someone? Hell no.

Re:to boldly go.... (2, Interesting)

tj2 (54604) | more than 6 years ago | (#20322623)

Would NASA cut corners like this and end up killing someone? Hell no.

Let's see.....

NASA death toll = 10 (3 Gemini astronauts, plus one space shuttle full)

Armadillo death toll = 0

You, sir, are a buffoon.

Re:to boldly go.... (1)

dan828 (753380) | more than 6 years ago | (#20323027)

NASA has lost two shuttles, so the toll is 17, not 10, but it's a faulty comparison anyways. Armadillo has barely gotten off the ground where NASA has been putting people in to space for 50 years. More time + more missions = greater chance for accident. Let's talk when Armadillo has a few manned missions under their belts.

Re:to boldly go.... (1)

XenoPhage (242134) | more than 6 years ago | (#20322753)

I'm with you. I'm with you 110%. This is EXACTLY why I have long opposed private spaceflight. Long story short, profit = cut corners = death. We saw it at the composites factory, and we'll see more of it.
Scaled has *NOT* released a detailed report on what happened. It could have been anything from human error to a bad part that caused that explosion. NASA has had plenty of human error problems, and I believe it was faulty parts (damaged o-rings) that caused Challenger to explode back in 1986? The Columbia accident seems to be a combination of both, but that's also debatable.

Private interests just do not have the long term perspective necessary to take the appropriate caution to prevent deaths. This is why space colonization should always be a government function.

Would NASA cut corners like this and end up killing someone? Hell no.
How do we know that NASA isn't cutting corners? It's not like they publish information like this...

Re:to boldly go.... (0)

Anonymous Coward | more than 6 years ago | (#20322515)

For a moment it read like "Hardware: Carmack's Armadildo Aerospace Rocket Crashes and Burns"

How funny (4, Interesting)

WindBourne (631190) | more than 6 years ago | (#20322555)

When I was a kid, I remeber the coming of the 747 (I paid attention since my father was a commercial pilot). Many ppl swore up one side and down the other, that this was a NIGHTMARE in the making. They said that they would never go because it would crash all the time killing more ppl than were in my town (small town). Their were so many cowards and small thinkers. Fortunately, Boeing pushed it, built it, and now, it is the major largest craft going.

Another group thought that we had no business going to the moon and swore that LLM would simply sink into the moon. I suspect that these same ppl believe that we never went.

Just so that you know, Carmack and his rocket are real.

that's unfortunate but (0)

Anonymous Coward | more than 6 years ago | (#20322113)

Where's the video?

Re:that's unfortunate but (4, Informative)

XenoPhage (242134) | more than 6 years ago | (#20322263)

From John's post to the Amateur Rocketry list :

We have video that we will be releasing, but Matt had to leave for Germany the next day, so it won't be digitized for a week and a half.

So, it's coming, just not released yet.

Re:that's unfortunate but (1, Troll)

UbuntuDupe (970646) | more than 6 years ago | (#20322735)

Translation: Let's see if we can stall on releasing the video long enough for the bad publicity to blow over and everyone forgets about it.

Re:that's unfortunate but (1)

ferat (971) | more than 6 years ago | (#20323077)

Probably more "Lets delay the video until Slashdot forgets about it so our servers have a chance to survive."

Coming soon... (4, Funny)

faloi (738831) | more than 6 years ago | (#20322127)

Gravity will make you it's bitch!

Re:Coming soon... (4, Funny)

MyLongNickName (822545) | more than 6 years ago | (#20322247)

Gravity will make you it's bitch!

But grammar won't make you its bitch, will it?

Re:Coming soon... (1)

Colin Smith (2679) | more than 6 years ago | (#20322549)

Depends whether that particular grammatical rule makes sense and is consistent.

 

Re:Coming soon... (0)

Anonymous Coward | more than 6 years ago | (#20322629)

So you like to write m'y, you'r, hi's, he'r, thei'r and ou'r ?

You would want to be consistent after all.

Re:Coming soon... (2, Informative)

Just Some Guy (3352) | more than 6 years ago | (#20322677)

Depends whether that particular grammatical rule makes sense and is consistent.

It makes sense. "It's" is a contraction for "it is", like "he's" for "he is" and "she's" for "she is". "Its" is an possessive pronoun, like "his". You wouldn't apostrophize "hi's", and you don't apostrophize "it's".

Re:Coming soon... (1)

Toonol (1057698) | more than 6 years ago | (#20323035)

"It's" is a contraction for "it is", like "he's" for "he is" and "she's" for "she is". "Its" is an possessive pronoun, like "his". You wouldn't apostrophize "hi's", and you don't apostrophize "it's".

I already knew everything that you just said, but I don't think I've ever seen it laid out so simply and plainly. I may re-use that synopsis, if you don't mind.

Re:Coming soon... (0, Offtopic)

DanQuixote (945427) | more than 6 years ago | (#20322913)


I wish you wouldn't criticize people who don't spell "it's" the way you do (or the dictionary does).

People who spell it that way are just smarter than the folks who write dictionaries, IMHO. :)

Think about it:
Logic path A --> can not = can (erase some letters, put in an apostrophe) t
[therefore] it is = it (erase some letters, put in an apostrophe) s = it's!
Logic path B --> possessive = add apostrophe-s
[therefore] the property pertaining to it = it's!

It really just makes more sense this way, so forget the dictionary already!

Re:Coming soon... (5, Informative)

jonnythan (79727) | more than 6 years ago | (#20322265)

Carmack and Romero are two different people.

Re:Coming soon... (1)

Jesterboy (106813) | more than 6 years ago | (#20322603)

But come on, it is funny.

Although, I would've preferred opening with a Quake "rocket jump" joke, but there's still plenty of posts to go; I'm sure someone will bring it up.

Re:Coming soon... (0)

Anonymous Coward | more than 6 years ago | (#20322921)

I haven't seen a crash like that since Daikatana!

Current feelings: Conflicted (5, Funny)

0racle (667029) | more than 6 years ago | (#20322131)

On one hand what they were working on was completely destroyed, on the other the explosion was AWESOME!

Current feelings: Bloated (0)

Anonymous Coward | more than 6 years ago | (#20322199)

"On one hand what they were working on was completely destroyed, on the other the explosion was AWESOME!"

Overheard during Fiesta night at Taco Bell.

Re:Current feelings: Conflicted (1)

d0rp (888607) | more than 6 years ago | (#20322423)

Yeah, as much as I sympathize that they lost their vehicle, I also want to see the video of this thing blowing up!

Progress Comes At A Price (4, Insightful)

ircmaxell (1117387) | more than 6 years ago | (#20322157)

This is a good thing. It proves (again and again) that new technology is never perfect. Just think, no computer program is ever completely bug free the first time it's compiled. The first car is never perfect... There are always bugs in any system. The point is that the safety mechanisms in the system worked well (after all, acording to the inputs of the lander, it was falling). As with any "accident", there are many failures that lead up to those incidents. That's the price of achievement. Nobody was hurt, so learn, build bigger and build better. If you learn from it, it wasn't a "mistake"...

I wish I had his money (0, Troll)

Anonymous Coward | more than 6 years ago | (#20322161)

Cause then I could pursue all kinds of things I have no business pursuing.

Get back to your workstation, nerdboy, and write me up some cool games.

Easy really - LAG (1)

Skiron (735617) | more than 6 years ago | (#20322167)

Lag caused it... if he waited a few milliseconds it would have 'reset' back to the ground again.

youtube link? (1)

putch (469506) | more than 6 years ago | (#20322169)

it had to have been caught on video. who's got the youtube link? or perhaps an HD torrent?

Re:youtube link? (2, Informative)

XaXXon (202882) | more than 6 years ago | (#20322705)

blog posting said the guy who does the videos is out for a week and a half, but they will be posting it - presumably to their website.

John's forum post on the subject (4, Informative)

Reality Master 101 (179095) | more than 6 years ago | (#20322177)

It was a bad weekend for Armadillo. We set out to put some flights on Texel, the backup Quad vehicle, and it didn't go so well. We have video that we will be releasing, but Matt had to leave for Germany the next day, so it won't be digitized for a week and a half.

We started out with a normal 90 second elevated / tethered hover test, but we ran into a problem with the actuator power. We initially thought it was a bad main power switch, but it turned out to be the lithium-polymer battery pack cutoff circuit incorrectly shutting down at 16 amps of load instead of 40. This was a new battery pack ( www.batteryspace.com HPL-8059156-4S-WR), and it had passed all the individual actuator checks, but when the igniter started firing with both high amp NOS solenoids, the battery shut down (went to 0.3 volts indicated) after one second and stayed there until it was physically disconnected. Russ made a fairly heroic field repair, cutting open the battery pack and wiring around the protection circuit while sitting on top of the rocket. The total time spent on this after three attempts was 90 minutes, and enough lox had boiled off that the vehicle hit lox depletion at 60 seconds of flight. We got a few good data points from this: the batteries need to be checked at full current load, with vents open we boil off about two pounds of lox a minute, and lox-depletion runs are benign, if a little flamey.

For the second flight we were going to do a ground liftoff (still tethered for runaway protection) to test the automatic ground contact engine shutoff code. We have had several reasons to want to automate this: We get a fair bit of bounce on touchdown, because the engine is essentially keeping the vehicle weightless during the terminal descent. A computer controlled shutdown would be at least a half second faster than my manual punching of the shutdown when I visually see ground contact. The quads will just safely bounce around on the ground a bit if the engine just goes to idle and doesn't shut down, but the module, with the gimbal below the CG, will try to tip itself over when a landing leg becomes a pivot point, so there is extra incentive to get it shut off fast. You can see that in our XPC '05 vehicle flight. We also need to handle the case of the vehicle landing in a situation where I can't shut the engine off promptly, either because there was a telemetry problem, or when we are doing high altitude flights, it lands out of direct sight. There is a separate shutdownTime parameter that will keep it from sitting there at idle for ten minutes, but a telemetry abort could still have it on the ground and cooking for the better part of 220 seconds. We could still shut the flight safety fuel valve, which would result in just idle level lox pouring out of the engine, but that has its own problems.

I have been very hesitant to put in ground contact shutoff code, because shutting the engine down for some incorrect reason would be catastrophic, and I would feel awful if that ever happened. We had some switch based ground contact sensors on the old VDR, but they never got tested. We have concluded that the landing jolt, as seen by the IMU accelerometers, is a good enough ground contact signal. There is always the worry that combustion instability, or a nozzle ejection event, might trigger the signal level, so there are additional guards about it only functioning when you are within three meters of the ground (we must leave some slop for uneven terrain or GPS innacuracy) and trying to descend.

We loaded up again, being very thankful that we now pack three six-packs of helium for each test trip after we were forced to cancel the second flight on a previous test session due to insufficient helium after troubleshooting a problem forced a repressurization on the first flight. Liftoff and hover was fine, and at the 45 second mark (no sense pushing it on a ground liftoff), I had it come in for a landing. It hit the ground, and I saw it bounce back up. My first thought was "That didn't seem to help at all". My second thought was "Uh, that looks like it is accelerating upwards, not bouncing." My third thought was "How the heck did the ground contact code cause that?" My fourth thought was "Crap, its going to fly into the crane, I need to kill it".

After I terminated thrust, the vehicle coasted to an apogee of about 20 feet, and fell to the concrete. It made a fireball that would make any Hollywood movie proud, but the vehicle didn't launch itself back off the ground, make an earth-shattering kaboom, or throw any shrapnel. The fire truck moved into range of the crash and hosed down the vehicle until the fire was extinguished. Surprisingly, the flight computer was continuing to operate and transmit through all of this, but the sensor and wiring harnesses were shorted out in the fire, so we didn't have any sense of the state of the tanks. With trepidation, someone approached the vehicle and found that the lox tanks were still full and pressurized at the same pressure as when the vehicle was shut down. The Aspen Aerogel insulation on the tanks had prevented them from even warming up. We vented the lox, and started assessing everything.

It didn't take long to find out exactly what had happened.

On touchdown, the ground contact logic failed to activate at all. The IMU in Pixel is an older model Crossbow that was rated for +/-10 Gs, but reads to +/-14 Gs. That particular model was discontinued, and the newer IMU in Texel was only rated for +/-4 Gs. I had set the ground contact trigger value to 6 Gs, which I had some recollection that the IMU read to, but it turns out that it was maxed out at 4.5Gs.

What caused the upwards flight was a GPS issue. On ground contact, the GPS PDOP value went from our normal 200 or so up to a value of 1200. The very next frame, it went to 0. We ignore GPS updates when the PDOP is 0, and also some other cases where we know the data is bad, but after flight starts the rule has been that any valid GPS update is taken as authoritative for velocity. Between GPS updates, and if the GPS goes invalid, the IMU will use dead-reckoning to coast for a while, but the accelerometers in particular aren't accurate enough to do this for very long. The at-impact 1200 PDOP update contained velocity values that were significantly off, including a 5 m/s down velocity, which caused the vehicle to throttle up to try to regain the desired 1.5 m/s terminal landing velocity.

I briefly wondered if the GPS antenna mast might have actually fallen off on landing, giving a correct velocity before losing sat view, but reviewing the video showed that it clearly stayed in place through the bounce.

We have known these GPS receivers are vibration sensitive for a while, and we take several measures to protect them from the rocket environment, but this was the first time we had a shock related failure. I went back to the telemetry from the Oklahoma free flights to look for similar signs on touchdown, and found a corroborating point. On the second free-flight, the GPS PDOP jumped from 200 to 359 at the time of touchdown, as seen by the accelerometers. This effect has evidently always been there, and some combination of the still heavy propellant tanks from the shortened flight and just bad luck caused a jump all the way to 1200 PDOP and an unusable value.

The question of interpreting GPS PDOP values has always been an issue for us. The exact meaning has to do with uncertainties in the calculated GPS position value due to the angles between the currently tracked sats. Flights typically have a PDOP between 150 and 250. We have a no-go set at PDOP 300, and an in-flight abort set at PDOP 400. While the calculated GPS position can vary widely with higher PDOPs, the velocity value always seemed to stay fairly reasonable at high PDOP values, so I had intentionally elected to continue using velocity updates even if the PDOP was past the abort point. The situation I was worried about was having the vehicle at 60 meters altitude, and having the PDOP change to 450, forcing both an abort, and, if I couldn't use that velocity data any more, a reversion to coasting on the IMU updates all the way to the ground, which I wasn't very confident of. I'm not beating myself up about this original decision.

The change I am going to make as a result of this is to define an unacceptable PDOP value, initially set at 500, beyond which a GPS update will simply be ignored as if the PDOP were 0. If we are in a situation of degrading PDOP somehow, that should allow it to abort at 400 and still hopefully make it to the ground with GPS based velocity updates.

In hindsight, there are a couple other, more reasonable-to-expect, things that would have saved the day:

If we had tested the ground contact logic by actually dropping Texel at the shop, we would have found that it didn't trigger, and I would have adjusted the value until it did work. If that had been done, we might not have even noticed the GPS problem, because it would have shut down before any action based on the bad velocities was taken.

If I had planned on just shutting down the flight like I normally do, instead of just watching what the vehicle did with just the ground contact logic, it would have just looked like a higher-than-normal bounce, and we would have seen both the ground contact failure and the GPS issue when I looked at that part of the telemetry. If the ground contact logic had functioned, my pressing the button shortly after would have had no effect.

There were a number of data points learned from the experience:

The vehicle fell straight down after the failure, as it was supposed to do. In fact, three separate shutdown systems were activated. I commanded the shutdown first, followed shortly thereafter by the computer thinking it had flown outside the shutdown box due to the abnormal velocity and triggering another shutdown code, followed a second later by Russ hitting the remote flight termination button and causing the independent flight cutoff valve to close. The reaction times on all of this were quite good, especially considering the completely unexpected nature of the failure. It is one thing to be watching a specific line and hit a button when something crosses the line, and another thing to analyze a completely unexpected situation and make a high-cost decision under pressure. Everyone else reported that their first thought was "Why is John doing a touch-and-go?", and it took a little while to register that this was not intentional.

The vehicle had over 200 psi in the tanks when it went down. There were two pressure vessel failures, both on the fuel side: one of the stubby "feet" on the bottom of one of the fuel tanks hit hard enough to tear at the weld, and the fuel pipe that supports the cutoff valve broke under the force of the impact. All of the fuel was pushed out in very short order, and ignited by residual flames from the engine shutdown. No lox was vented, although there were high heat signs of a small oxygen leak around our lox dump valve during the fire, probably due to the fire cooking the valve packing. It would have been more dangerous if the lox tanks had also broken, but it is hard to say exactly how much. It might have been just an extremely hot fire that melted the vehicle to slag, or it might have been a significant explosion.

All three remaining tanks were bent in somewhat at the bottom where the "feet" were, but they still hold pressure. We will probably do some fatigue cycle tests on them now that the vehicle is scrap. If this had been over dirt, or the tank bottoms had just been simple hemispheres with rubber bumpers without the threaded mounts protruding, the tankage would have most likely been completely undamaged after a 20' fall. The top fuel pipe would still have broken, but that could be designed around if desired. It is possible to make a pressure fed vehicle that can survive a 20' drop.

The fireball and blaze were very spectacular, but the parts of the wiring harness that were wrapped in leather actually came through the blaze ok. With some more care, it would not be unreasonable to engineer a wiring harness that could actually live long enough inside a blazing wreck to allow you to do something useful, like vent pressure.

Everything inside the electronics box appears to be unscathed and still functional, although we know from a previous crash years ago that hidden faults may still lurk, so we aren't going to use them again. We can give an extremely strong endorsement to McMaster-Carr " 85925K423 Fire-Retardant Silicone Foam Rubber Sheet Adhesive Back, 1/4" Thick". We started covering our electronics boxes in foam years ago to give them a milder acoustic environment for the GPS units, but we managed to set the electronics box on fire a couple times when test stand engines misbehaved. We eventually moved to this material, and it not only didn't burn inside this fireball, but it protected everything inside the box quite well. The foam on the bottom of the box was hit by a high-pressure burning fuel jet from the plumbing rupture, but it didn't burn through.

We still have Pixel and Module 1 in flyable shape at the shop, so this doesn't have a critical impact on us, but it does change our testing plans for the next two months before the X-Prize Cup. We are cancelling the untethered 180 second flights for Pixel at OKSP. We will plan on doing two sets of back-to-back 180 flights under tether, but if we are going to risk a crash, it might as well be for the money at XPC now that we don't have a backup. We are going to finish up Module 2 in the next couple weeks so we have a backup for level 1. Modules 3 through 5 should also be at least frame constructed by XPC, but whether we get them wired and tested will depend on how our flight testing goes. If we manage to destroy a module in the next two months, we can crunch hard and get an extra one put together if necessary.

We will be flying again this weekend, and I hope to get at least the following tests in before XPC:

tethered back-to-back 180s with Pixel
tethered ground liftoff / touchdown tests with Pixel (using the double-shudown testing protocol)
tethered back-to-back 90s with Module 1
tethered ground liftoff / touchdown tests with Module 1
free-flight back-to-back 90s with Module 1 at OKSP
tethered back-to-back 90s with Module 2

John Carmack

Re:John's forum post on the subject (5, Funny)

Linker3000 (626634) | more than 6 years ago | (#20322631)

Executive summary:

"hotwired the battery...we don't need no stinkin' ground shutoff code...Sensors - never got around to testing them...we left some slop...ya think something rated at 4G would work up to 6G?...we know the GPS receivers are vibration sensitive so we stuck some bubble wrap round them and hoped...we checked earlier telemetry and yup - they're darn vibration sensitive...hold on lads; I've got an idea...The rocket has gotta return to the ground at some point; if only we'd done some testing on this...John's doing some fancy flying - oh, sh*t, he's not...now the tanks are scrap we're probably going to do some useful tests on them that we wouldn't have done with usable ones - heck those things cost money, baby...some of the wiring harness is wrapped in leather so we're going to alienate the vegan customer base...flammable foam catches fire."

I think I'll walk.

PS: The captcha I had to type in to submit this was "Piloting" - BWAHAHAHAHAHA

Re:John's forum post on the subject (0)

b0s0z0ku (752509) | more than 6 years ago | (#20322785)

.we don't need no stinkin' ground shutoff code

He said that ground shutoff could be manual, and he didn't want to add automatic shutoff code until it was triple-checked, owing to the danger of it triggering at the wrong time and destroying the craft.

ensors - never got around to testing them...we left some slop...ya think something rated at 4G would work up to 6G?

Actually, it very likely can work at 1.5x RATED load. Ever heard of "factor of safety?"

I think I'll walk

It's called "experimentation!" Sometimes things work, sometimes they fail. You learn from the failures. With your attitude, people would still be stuck in the dark ages -- after all, the fabric used to cover the Wright Flyer wasn't aeronautically RATED either!

-b.

Re:John's forum post on the subject (3, Interesting)

XenoPhage (242134) | more than 6 years ago | (#20322683)

For those that are not aware, this was John's post on a PUBLIC forum. John has continually posted information regarding his team's experiences and any important information they have learned. He's taken the open source mentality into the rocketry arena and many teams are all the better for it. This is the type of information that NASA would happily write a few hundred page reports on and they encase in cement and bury.

I've been lurking on the rocketry group for a while now and it's great to see the open discussions about everything from rocket design to safety. I've learned more in a few months that I ever did watching all those NASA shuttle launches over the years.

Too much reliance on GPS? (3, Interesting)

Radon360 (951529) | more than 6 years ago | (#20322791)

John goes on about the use of GPS in the control for acceleration for a bit. Understanding that where you have no nearby reference points, such as in space, this may be a good solution. At the same time, you usually don't have anything nearby to worry about crashing into (such as the ground). Although GPS can be very accurate, it often takes more datapoints that can be obtained in a very short timeframe to get that accuracy.

I wonder if there's a reason why they aren't using some means of LASER or RADAR rangefinding when in close proximity to landing for obtaining positioning (altitude) and velocity/acceleration information. The update rate could easily be several orders of magnitude faster than GPS could ever provide...especially since you need two position reports from GPS to find velocity and three to determine acceleration.

No kidding (1)

Man On Pink Corner (1089867) | more than 6 years ago | (#20322863)

Off-the-shelf GPS receivers and lithium battery packs are two things that do NOT belong in a mission-critical (or even money-critical) application.

Carmack is going way too far in the direction of "Use whatever seems to have the specs we need." He seems to be forgetting that if you have 100 components that are 99% reliable, your overall system is only 36% reliable.

I don't trust lithium batteries to meet their discharge specs on my iPod much less a rocket controller.

Re:John's forum post on the subject (1)

LWATCDR (28044) | more than 6 years ago | (#20322929)

Sounds like they should also add redundant GPS systems. Three with a hot back up would be a good start.

The carmack (0)

Anonymous Coward | more than 6 years ago | (#20322191)

The carmack FAILED!

Re:The carmack (5, Informative)

XenoPhage (242134) | more than 6 years ago | (#20322365)

Failed? I think not. Just so you're aware, Armadillo was the only team last year to even attempt the lunar lander prize, and except for some bad luck, would have walked away with it.

This year, there may be a few other challengers, but I think John and company will walk away with it. John and his team have taken this challenge in directions that the "big guys" have never tried, and it's working.

We'll see! Only 65 days left!

So (2, Funny)

CaffeineAddict2001 (518485) | more than 6 years ago | (#20322297)

When can we buy parts of the wreckage on ebay?

Re:So (1)

Eponymous Bastard (1143615) | more than 6 years ago | (#20322895)

Why is this funny? If its sister ship manages to win the Lunar Lander competition, they could probably sell parts of this to the spaceflight crowd. Not very expensive, but it could still be a bit of memorabilia. I'd like a piece :)

If armadillo eventually manages to do something historic, you can bet this ship will eventually end up in a museum display somewhere. Maybe on their HQ or research facility, but somewhere...

Why Ebay when you can buy directly from armadillo? (1)

Tmack (593755) | more than 6 years ago | (#20322917)

Well I say that and it appears they no longer sell armadillo droppings. At least I cant find it on their page anywhere. Back in the beginnings, they sold scraps/broken stuff as "Armadillo Droppings" for a donation to their cause, small donations got smaller droppings, like nuts/bolts/etc, larger donations got larger parts. I guess they discontinued so they could concentrate more on the actual research/dev stuff (if not for just legal reasons, you know, selling rocket parts to lybian nationalists might not look so good, even if they do turn out to only be used pinball machine parts ;) ).

Tm

X-Prize Cup (3, Insightful)

pavon (30274) | more than 6 years ago | (#20322485)

Ahh, I'm bummed out now. I was really looking forward to seeing them at the X-Prize Cup in October. They were expected to claim the prize (for the level 1 lander challenge), as they had already completed flights matching the profile on their own, and just had to repeat it a the cup for it to be official. I don't know if they'll have enough time to rebuild the craft in time for the event.

Most likely, there will be a winner (1)

WindBourne (631190) | more than 6 years ago | (#20322667)

First, Armadillo is not out of the race. Second, there are 8 other in this. One is new shepard who keeps VERY quiet. I believe that this year, there will be a winner.

Nevermind (1)

pavon (30274) | more than 6 years ago | (#20322733)

Yeah, after RTFA and John's forum post, it appears that they only crashed their smaller unit and their larger craft is still fine. I also hadn't heard that they were shooting for the level 2 with pixel. That should be interesting.

One of several Armadillo vehicles (4, Informative)

peacefinder (469349) | more than 6 years ago | (#20322813)

Texel was one of two essentially identical vehicles that Armadillo put together last year for the Lunar Lander Challenge. The other is Pixel, which is the one they actually flew last year (and that had a good shot at winning) at the LLC level 1 event. Pixel is still flightworthy. This crash of Texel doesn't take them out of the LLC race, although it will lower their chances of success; it is going to make them much more cautious about banging Pixel up ahead of the next LLC competition and therefore they'll get flight less testing in.

They're also working on a set of new vehicles they call Modules, of which I gather they have one essentially complete and five in production.

Re:One of several Armadillo vehicles (1)

pavon (30274) | more than 6 years ago | (#20323011)

Yeah, I should have RTFA before I posted. I'm surprised that I got modded up, especially after several people had already corrected me. What I think is most interesting about this is that rather than playing it safe and flying Pixel in the Level 1 competition, they are still planning on using it for the Level 2 event, and using one of the smaller Module crafts for the Level 1. That makes the upcoming contest far more interesting. I'm excited.

Re:X-Prize Cup (1)

FleaPlus (6935) | more than 6 years ago | (#20322841)

Ahh, I'm bummed out now. I was really looking forward to seeing them at the X-Prize Cup in October. They were expected to claim the prize (for the level 1 lander challenge), as they had already completed flights matching the profile on their own, and just had to repeat it a the cup for it to be official. I don't know if they'll have enough time to rebuild the craft in time for the event.

That was just one of their vehicles. They still have Pixel. From Carmack's post:

We still have Pixel and Module 1 in flyable shape at the shop, so this doesn't have a critical impact on us, but it does change our testing plans for the next two months before the X-Prize Cup. We are cancelling the untethered 180 second flights for Pixel at OKSP. We will plan on doing two sets of back-to-back 180 flights under tether, but if we are going to risk a crash, it might as well be for the money at XPC now that we don't have a backup. We are going to finish up Module 2 in the next couple weeks so we have a backup for level 1. Modules 3 through 5 should also be at least frame constructed by XPC, but whether we get them wired and tested will depend on how our flight testing goes. If we manage to destroy a module in the next two months, we can crunch hard and get an extra one put together if necessary.

Not on Mars (1)

97cobra (89974) | more than 6 years ago | (#20322505)

Perhaps it would have survived had it only fell 6 feet instead of 6 meters.... Oh wait, this isn't NASA and this isn't Mars

Harsh (4, Insightful)

Hawthorne01 (575586) | more than 6 years ago | (#20322627)

I have no opinion on Carmack one way or another, but tagging this story with 'haha' and 'hesnorocketscientist' seems a tad mean.

So he's a game designer dabbling in space exploration. It's not like he ran a bicycle shop [wikipedia.org] or something. Now *there's* a logical starting point for a career in aeronautics!

Mod Parent Up (1)

sam_paris (919837) | more than 6 years ago | (#20322747)

I totally agree.

In fact I believe that Carmack is more qualified than most. For a start he's a maths genius, having spent his life creating the most advanced graphics engines from scratch. He's single handedly developed many of the graphics techniques that are used in virtually every game.

When someone is that smart, it's clear that if he puts his mind to it, he could easily learn the finer details about rocket science.

Re:Harsh (3, Informative)

SCHecklerX (229973) | more than 6 years ago | (#20322957)

So he's a game designer dabbling in space exploration. It's not like he ran a bicycle shop or something. Now *there's* a logical starting point for a career in aeronautics!


Actually, it is. Bikes and airframes are VERY similar. You are trying to get a very strong structure with as little weight as possible. With a bike, as with an airplane, you can't just slap a factor of safety of 9 on the thing. You have to really design it, and pay attention to materials science. (Hint: Bikes, like planes, take advantage of lightweight aluminum alloys, carbon fiber, high torsional rigidity, etc).

Then there is knowing how to use the right materials in the right places for minimum cost/weight, or for rigidity / flex.

Today's bikes are what they are mostly from Aerospace research.

GPS for a lunar lander? (2, Interesting)

klossner (733867) | more than 6 years ago | (#20322727)

But the touchdown did have a big enough effect to jostle the onboard GPS unit that Texel relied on to track its motion.

Why would a candidate for a mock lunar lander be designed to depend on GPS? There won't be GPS service on the moon in the foreseeable future.

I bet nothing like this happened... (1)

Schnoogs (1087081) | more than 6 years ago | (#20322751)

while Carmack was working on IDTech5!! Carmack is too valuable to the gaming industry for him to be involved in such potentially dangerous hobbies. Now that Epic is tied up with the backlash against Unreal 3 this is id's opportunity to become the engine of choice again. Put down the rockets John and go back to the keyboard where it's safe!!! On a more serious note I'm glad to hear that noone was hurt.

Doomed (0)

Anonymous Coward | more than 6 years ago | (#20322771)

"That's one doomed Aerospace Rocket."

(In the words of Duke.)

Article title incorrect (NITPICK) (1)

b0s0z0ku (752509) | more than 6 years ago | (#20322805)

Minor nitpick -- this was NOT an X-prize competition (Rutan already won X-prize last year). This was a NASA sponsored competition for design of a lander.

UAC (5, Funny)

samwh (921444) | more than 6 years ago | (#20322843)

Shame, I was already to invest in his new company, dubbed the "Union Aerospace Corporation"

article tags are ridiculous (1, Interesting)

posterlogo (943853) | more than 6 years ago | (#20322867)

The tagging system on Slashdot is getting really pathetic. What kind of jerkoff tags this posting "haha". You think you can do better?

Cover the basics (4, Insightful)

Dan East (318230) | more than 6 years ago | (#20322903)

I'm hesitant to criticize a group that is breaking so much new ground so quickly, but this sounds like some really amateurish mistakes when it comes to electrical engineering. Basically they added new sensors to detect when the craft impacts the ground. The computer monitoring the sensors was expecting a signal of a certain strength to indicate it had touched the ground, however the value the computer was expecting was higher than what the sensors could physically produce. So it sounds like they either engineered the electronics wrong making it impossible for the sensor to produce a meaningful response, or they misread the sensor datasheet which resulted in flawed software.

Now it's one thing to make an engineering mistake, but it couldn't have taken them an hour to rig up a simple test rig they that they could drop onto the ground, or tap with a mallet, or something similarly simple, to see if the computer could register a landing.

I just can't imagine strapping something new onto an entire rocket assembly, going to all the risk and expense to actually launch the thing and fly it around, hoping that all the new circuitry and software will work perfectly the first time.

It makes me wonder about the whole process NASA has in place with these contests. Even if a craft can meet various flight goals, does it result in anything of worth to NASA? For example, take a piece of software. Say there is this program that really does something impressive (game engines come to mind). So you take a look at the source, and find it is a total and complete mess. Maybe it is full of memory leaks and other bugs, so it just can perform a specific task right, but given other scenarios it crashes. Maybe the code is insecure, or is not scalable, or cannot be extended, or is not maintainable, or is not portable to other platforms. Any of those things could practically render the sources useless. But yet the program does a specific task and does it really well. For some reason I feel that NASA is going to end up with crafts with similar engineering caveats.

Dan East

Please Direct The Next Launch (-1, Troll)

Anonymous Coward | more than 6 years ago | (#20323009)


to American Taliban Headquarters [whitehouse.org] .

I hope this helps "The Surge".

Thanks for contributions to world peace.

Cheers,
K. Trout, C.T.O.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...