Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Stunts, Idiocy, and Hero Hacks

samzenpus posted more than 3 years ago | from the when-it-just-has-to-get-done dept.

Idle 208

snydeq writes "InfoWorld's Paul Venezia serves up six real-world tales of IT stunts and solutions that required a touch of inspired insanity to pull off, proving once again that knowing when to throw out the manual and do something borderline irresponsible is essential to day-to-day IT work. 'It could be server on the brink of shutting down all operations, a hard drive that won't power up vital data, or a disgruntled ex-employee who's hidden vital system passwords on the network. Just when all seems lost, it's time to get creative and don your IT daredevil cap, then fire up the oven, shove the end of a pencil into the motherboard, or route the whole city network through your laptop to get the job done,' Venezia writes."

Sorry! There are no comments related to the filter you selected.

Rubber Band (3, Funny)

Gotung (571984) | more than 3 years ago | (#34535984)

I once fixed an issue that was holding up the operations of a $50 million dollar a year company with one well placed rubber band.

Re:Rubber Band (3, Interesting)

ColdWetDog (752185) | more than 3 years ago | (#34536018)

I once took out most of the internal network of a major hospital by innocently tugging on some duct tape while waiting for a Novel server to reboot. But I think we're not supposed to talk about those sorts of 'solutions'.

Re:Rubber Band (-1)

Anonymous Coward | more than 3 years ago | (#34536070)

I once farted a whole plumb. I was plumb shocked!

Re:Rubber Band (2)

Yvanhoe (564877) | more than 3 years ago | (#34536592)

I once saw people hand-starting a critical hard-drive whose motor had failed. It was opened so they knew it would die quickly but they did it to quickly copy a few critical MB of acounting data.

Re:Rubber Band (3, Insightful)

FatdogHaiku (978357) | more than 3 years ago | (#34537182)

I have packed drives in dry ice to get one last read out of them. Once the drive is very cold it's a race to get the data before condensation builds up on the circuit board. Side note, bare metal "screams" when pressed against dry ice... gas hammering against the metal.
My normal MO for this is stick cables on the drive, stick the drive in a heavy freezer baggie with the cables sticking out the open end of the bag. Rubber band or tape it shut as best you can with the cables sticking out. Put a block of dry ice on the top and bottom of the drive (outside the bag) and wrap it with something like a thin towel or paper to keep the dry ice in contact with dirve. Stick the whole thing in the freeze for at least half an hour before trying to spin up the drive. Leave the dry ice in place and hook the drive up (I have done both USB and IDE connections). If you are lucky you can HURRY and copy your data... If not you're out $5 for dye ice.

Re:Rubber Band (1)

Anonymous Coward | more than 3 years ago | (#34536030)

I once fixed an issue that was holding up the operations of a $50 million dollar a year company with one well placed rubber band.

Let me guess: the "operation" was a porn shoot, you were a fluffer and the stud couldn't keep it up. Am I close?

Re:Rubber Band (1)

rtyhurst (460717) | more than 3 years ago | (#34536076)

Oh you kids!

I remember when you could unlock the multiplier on an Athlon XP CPU by drawing four little lines with a pencil lead!

Now get off my lawn!

Re:Rubber Band (2)

AndGodSed (968378) | more than 3 years ago | (#34536140)

You could actually do that with the older Athlon chips too.

Nice lawn btw.

Re:Rubber Band (1)

TheThiefMaster (992038) | more than 3 years ago | (#34536208)

You could also do it to enable the MP capabilities of an XP chip.

I don't have a lawn :(

Re:Rubber Band (1)

Anonymous Coward | more than 3 years ago | (#34536682)

I think that was a Celeron 300A hack that was required to enable SMP. Man, those Celeron 300As were bad ass chips... One of only a few batches of CPUs that I've ever been able to successfully run 50% overclocked with 100% stability!

Re:Rubber Band (1)

imboboage0 (876812) | more than 3 years ago | (#34536530)

Oh you kids and your fancy pencils. I took the liberty of stipping stranded copper wire, and painstakingly placing individual strands between pins inside the socket of an Athlon XP-M 2600+ to overclock it to 2.8GHz. And we liked it that way.

Re:Rubber Band (1)

Gaygirlie (1657131) | more than 3 years ago | (#34536598)

I had once a motherboard that was sold in pro and cheap versions. There was physically no difference as both had all the same chips and connectors, the cheap one just had only stereo sound and Serial-ATA disabled. Just smudge a disconnected bridge with a pencil and you got the pro version. I never understood how it was cost-efficient for them to do it that way, but I didn't complain.

Re:Rubber Band (1)

Gordonjcp (186804) | more than 3 years ago | (#34536998)

It probably failed on test and was sold as a "normal" one. That's why you could overclock CPU chips, too - they're tested then the whole batch is labelled with what they're capable of handling.

Re:Rubber Band (1)

Jay Tarbox (48535) | more than 3 years ago | (#34536620)

I was doing this (actually with the window defroster repair kit trick) and that was when I realized my eyesight was beginning to go farsighted. I couldn't see the traces when I held them closer to my eyes! Weird!

Re:Rubber Band (1)

pilgrim23 (716938) | more than 3 years ago | (#34536238)

WAY back in the day when Mainframes ruled the land, data was written to 9 track tape. Occasionally a data check would occur when data was written to a bad spot (crystal int eh ferrite mix coating the tape, wrinkle, etc) on the tape. Old Operator trick: Pop open the Vacuum door on the tape drive, pull out the tape, Lay the spot on that tape over the top of the tape drive door, briskly rub with a big square rubber eraser kept for this purpose, put tape back and hit START on the drive. Usually the tape then works and payroll goes through with no one the wiser

Re:Rubber Band (1)

clone52431 (1805862) | more than 3 years ago | (#34537008)

What did that actually do?

Re:Rubber Band (4, Interesting)

pilgrim23 (716938) | more than 3 years ago | (#34537872)

it would smooth out crystals in the ferrite matrix of the tape. Seriously. Data at that time was measured as 1600BPI or 1600 bits per inch of tape recorded in 8 discreet tracks or "not very much". If a spot on the ferrite coating bridged the "tracks" this caused a data check (and bridging was the usual cause of issues). Running the eraser over the tape smoothed and broke this connection, resorting in the tape drive being able to read this bad spot. We are not here talking about the femto-micron gap between bits on a modern hard drive

OT: Obamacare unconstitional (-1)

Anonymous Coward | more than 3 years ago | (#34536034)

Opinion here [] PDF warning

Floppy disk in the wash (4, Interesting)

wandazulu (265281) | more than 3 years ago | (#34536050)

I got "lucky" to solve a problem for someone back in college: she had written her thesis on a 3.5 floppy, had no backup (this is when you had to go to the "computing center" to work, as practically no one had a machine of their own, so you had to take all your stuff with you), and had run the disk through the washing machine.

She came in, crying hysterically (it actually took a few tries just to figure out what was wrong), and realized what had happened. I had one of the few "eureka!" moments of my life, and grabbed another floppy, carefully cut it open, did the same with her disk, then air-dried it. I put the platter in the "new" disk, with its dry fabric covering (whatever that stuff was...), taped it shut, and put it in the Mac ( hd) and yep, the disk was readable and I was able to get her thesis off and onto a network drive, then we copied it back onto a new disk and assured her I'd hold onto the thesis on the network drive until the end of the semester.

Funny thing, she kept the disk I had used, taped around the edges, and the next year I saw her again and asked how things were, and she was still using it. Go figure.

Re:Floppy disk in the wash (5, Insightful)

LMacG (118321) | more than 3 years ago | (#34536104)

I don't think you understand the term "got lucky." Oh right, I'm reading Slashdot ...

Re:Floppy disk in the wash (1)

MetalliQaZ (539913) | more than 3 years ago | (#34536312)

+1 for the laugh!

Re:Floppy disk in the wash (3, Funny)

Ihmhi (1206036) | more than 3 years ago | (#34537268)

Hell, with that kind of IT heroics, who's to say he didn't? He might just be doing the gentlemanly thing and not talking about it.

I mean, really, "I saved her thesis from being lost forever and then banged her brains out with her leaning against a blade server" is a tad bit uncouth, wouldn't you say?

Re:Floppy disk in the wash (1)

Skater (41976) | more than 3 years ago | (#34536528)

My girlfriend (now wife) managed to snap the USB connector off of her thumb drive...of course, without backups. I was able to solder it back on and it worked. (Before I started, I wasn't sure how hard it would be, but the soldering went pretty well, so I wasn't that surprised when it did work.) I copied the data off of it, burned the data to CD, and promptly threw the USB drive away to prevent any temptation in using it again!

In hindsight, I probably should've at least taken a picture of it.

Re:Floppy disk in the wash (4, Informative)

CAIMLAS (41445) | more than 3 years ago | (#34536866)

Did you get laid for that one? I had several such opportunities resulting from obscene levels of gratitude.

I came to a school as a sophmore, and ended up staying in a co-ed mixed year dorm. The school didn't have a heavy (any) IT/CS focus, and this was right around when the quality of floppy disks and drives was "questionable" - on a good day, with very careful handling (1999-2001), you might get a disk to work in a drive that didn't write it. All it took was one successful dd recovery of a floppy disk and the word got around.

I loved that old Toshiba floppy drive: it was so much more reliable than the drives of the era, ran quickly, and could read pretty much any 'corrupt' data. Very rarely was anything unreadable.

As someone else said, being poor, on a time crunch, or limited by other people's failure to plan does seem to result in some pretty good 'hacks'. I didn't think the list they picked was all that spectacular: many of the "unconventional" ones have been done before by many others, I'm sure.

* Riser card creep? Hot glue (I always keep a gun handy now)
* Routing traffic through a Linux laptop? That might be a "jackass hack", but many people do it on a planned, regular basis, and have for the better part of a decade. Move along...
* Cook your drive? I've never heard of that trick, though I have frozen drives to recover data (bucket of ice, water, and a little water purifier salt, with a triple-bagged hard drive).
* enable password on the network? Anyone using rancid and no encryption does this; I'm sure there are others.
* heartbeat - I had to do this temporarily (or something like it). I used netcat.
* The timezone settings? Pretty sure that there's nothing 'jackass hackish' about that - that's just a common part of remote system deployment. I've worked at several places which have done things in a similar fashion.

Other "jackass hacks" I've done (that I don't think are all that incredible):

* Expensive network MFD printer's built-in ethernet died - but it had USB. Hooked a laptop up and shared the printer, with scans getting automatically dumped to a shared path until a replacement could be acquired (small office).
* Could not get a back plate adapter for a supermicro tower chassis from them on time to use a standard, quality ATX PSU, as I'd already had multiple (shit) PSUs from SM. Spent an hour that night at home cutting one from an old Dell Optiplex case via a cardboard stencil so I could get the system back up.
* Plastic CPU mounting bracket used for the HSF on a first-generation Opteron cracked due to the OEM tension bracket being too tense. -Carefully- drilled 4 small holes in the board and attached the HSF via a cat5 insulated strand garrote, tensioned on the back of the board.
* Modem bank had modems that were hanging with regularity. Better airflow helped, but no cigar. Determined the wall wort PSUs were getting warm and causing the modems to crash. Wired up an old(er) tower PSU to provide the power to the modems directly, and threw in a couple 12v fans for good measure. Problem solved.
* Customer complains about server noise. Five minutes and a drop of machine oil on the CPU fan and the problem is 'fixed' - no charge, and the customer is happy.
* Virtual server had a controller + disks blowout... put the system's VMs on a remote network share from backup and booted it from USB. Had it back up in slightly more time than it took to copy the images over.

These are just the tricks of the trade: we make hackish decisions like this every day to "just get the job done". Hopefully we can go back and fix them properly at a later time.

Re:Floppy disk in the wash (2)

skids (119237) | more than 3 years ago | (#34537178)

Well, personally I've got plenty of such war stories, but you never seem to have an opportunity to get laid for looping back unused OC-3 ports to boost signal when a fiber degrades, putting spare optical amps into a line to graph signal level and verify the cause of an intermittent outage occurs exactly when outdoor temperature crosses 0C, or taping a pencil and a mouse to a cd tray in order to develop input drivers for a machine located miles away (think "eject -c").

Occasionally I wish I'd stayed in user-help related jobs, if just that more human contact would help my mood. Then I see the worn-out, beat down faces of the help-desk staff as they leave work and remember something about the color of grass as it relates to fences.

Re:Floppy disk in the wash (1)

JamesP (688957) | more than 3 years ago | (#34537152)

This reminds me you could bore a hole in a 720k disk to "turn it into" a 1.44Mb disk. WOW HEY!

Not very reliable though...

Re:Floppy disk in the wash (1)

i.r.id10t (595143) | more than 3 years ago | (#34537688)

Very reliable - just flip it over, use a second disk as a guide for where to punch the hole, and punch the hole using a hole puncher.

Of course, when the 45 minute audio tapes came out I was all happy about having 50% more data storage, but the new tape stretched too much for data use...

Re:Floppy disk in the wash (1)

imgumbydamnit (730663) | more than 3 years ago | (#34537850)

Just as could convert a single-sided 180K disk into a "flippy" 360K disk, with a paper hole puncher or an exacto knife..

when all else fails (2)

NiceGeek (126629) | more than 3 years ago | (#34536062)

I don't have a story on the the scale of any of these but I remember fixing my Commodore 1541 drive by adding an extra screw. The drive belt had gotten stretched somehow and I was getting all kinds of read errors, and being poor, decided to attempt a repair. Turns out there was an empty screw hole near the drive belt, I put one in, stretched the belt around the screw to take up the extra slack and that drive was still working when I finally got rid of my C64 a couple years later.

I know, cool story bro.

baking the drive (2)

digitalsushi (137809) | more than 3 years ago | (#34536088)

I worked with Paul a long time ago at a mom&pop in NH. And I know that he personally did the drive trick and it worked. It was a 9 gig scsi drive with an smtp mqueue on it. He was extremely elated that it had worked, and his portrayal of the story to a wide-eyed netadmin noob (me) was one of those late-night, sipping coffee at the Red Arrow while the raid rebuilds sorta memories that you'll take to your grave.

Re:baking the drive (1)

cinderellamanson (1850702) | more than 3 years ago | (#34536526)

You can also freeze them, it's only worked once that I have tried it though.

Re:baking the drive (1)

AK Marc (707885) | more than 3 years ago | (#34538144)

Freezing can work if they spin but don't read. Cooking can work if they don't spin but would have read.

Computer Tech (4, Interesting)

cosm (1072588) | more than 3 years ago | (#34536112)

Back when I was a computer tech for one of the big retailers, I had a customer bring in a machine that wouldn't boot. After interrogating the customer a little more, it turned out he had tried 'upgrading' his CPU, and in the process had broken off one of the Athlon XP's (shows age) pins by inserting the CPU in the wrong orientation.

The dude couldn't afford anything new, so I offered my most MacGyver-ish attempt. I went over to the car-audio shop, grabbed some speaker wire, spliced out some copper about the same size as a pin, and voila!

After bending some of the pins back with a mechanical-pencil tip, and inserting the new 'pin' into the socket below the missing pin on the CPU (cut to semi-correct length), it booted right up! He took it home and all was well. I don't work for said company any more, but how long that 'fix' lasted is questionable.

Never told the boss about that one.

Re:Computer Tech (1)

citylivin (1250770) | more than 3 years ago | (#34536634)

HA! that reminds me, i just did that a few months ago. I was straightening some pins on a CPU i had stupidly dropped and suddenly one of the pins broke!

I gasped, but thought about it and cut a pin off of a pentium pro cpu i had on my desk. Put it in the socket where the regular pin should be, slap the cpu on top and lo and behold the machine booted and ran fine!

The machine is still working in my company right now. I wonder how common it is of a hack. I was quite surprised that it actually worked, and stayed fixed!

Re:Computer Tech (2)

Jeng (926980) | more than 3 years ago | (#34537792)

I wouldn't be surprised if hacks such as that were the grounds for Intel developing their LGA design.

Re:Computer Tech (1)

gfody (514448) | more than 3 years ago | (#34538118)

I broke a pin off an old 486 chip once. Didn't do anything about it and the computer worked fine.

Depends on the point of view (5, Interesting)

arivanov (12034) | more than 3 years ago | (#34536142)

The first "stunt" depends on your point of view. If you have nicely brainwashed and duped by marketing material that "Vendor gear good, PC bad" that may sound as a stunt. If you actually know what you are doing you can run networks for years on this.

Nearly any laptop today has the forwarding grunt of an upper end of a 3800, there are plenty of servers that are on par with a 7200 or low end 7600 and most supervisor modules. You can run a network on this on a daily basis and do a _LOT_ of things a Cisco cannot do or cannot do at sufficient performance.

To put the so called "stunt" into a perspective, I used to run a production installation with 20+ 802.1q trunks via 800MHz Via EPIAs with 600+ entry ACL lists including content filtering with VRRP failover, load balancing to multiple upstream uplinks, OSPF, hardware accel-ed openvpn and ipsec, 16+ class hierarchy CBQ QoS and a few more bells and whistles. For years. Not for 48 hours.

Nothing wrong with it if you can do it. If you cannot - well, not everything in life is learned on CCXX and RedRat certification courses. C'est la vie.

Re:Depends on the point of view (1)

cinderellamanson (1850702) | more than 3 years ago | (#34536606)

Yea, you want to have your intelligence really insulted try infoworld's programmers IQ test. I hate to judge as they are pretty open minded about open source, but seem to be a little daft here and there.

Re:Depends on the point of view (4, Interesting)

Minwee (522556) | more than 3 years ago | (#34536714)

If you actually know what you are doing you can run networks for years on this.

Or, depending on where you source your notebook computers from, the whole thing could fall over in a few hours.

A company I worked for did a similar stunt a few years ago by repurposing some old Latitude D600s as a development cluster when they ran out of money for real servers. On the surface it looked like a smart idea -- The hardware was already paid for, had a small form factor and every single one had its own built-in UPS. What could possibly go wrong?

The answer to that is that every few days at least one of them would die and need to be rebooted, reimaged or simply beaten with a club. Some things are designed to sit in racks and run non stop for years at a time, others are designed to sit on a table at Starbuck's and run for a few hours before shutting down. The trick is in knowing which ones are which.

Re:Depends on the point of view (1)

skids (119237) | more than 3 years ago | (#34536794)

Really it's a matter of whether your employer has the ability to recruit Linux networking talent. If you're a rare hire, and they cannot usually get that talent, best to stick with solutions a Cisco Certified Network Alpaca is familiar with.

Limp Mode (0)

Anonymous Coward | more than 3 years ago | (#34536160)

I once had the servers using a backup database on a machine slower than the main database! ...
Okay, that's pretty tepid.

I did figure out how to recover a day's worth of offsite data saved on a corrupted USB key. I've had to start ancient-but-still-used hard drives by repeated power cycling after the power went off long enough for the UPS to lose battery.

Nothing too exciting.

Speaker Wire (2, Interesting)

Anonymous Coward | more than 3 years ago | (#34536166)

I had non-booting server board back in the late 90s and managed to track the problem to a scratch through one of the traces on the bottom of the board. Something had fallen between the board and the offsets and had worn through the circuit.

Having nothing to lose, I fired up the soldering gun and pulled out the only wire I had from a pair of speakers and sure enough, once the circuit was made the board booted and remained stable long enough for us to order and install a replacement.

socialist ? (-1)

Anonymous Coward | more than 3 years ago | (#34536214)

jesus would be a socialist.

he was (1)

unity100 (970058) | more than 3 years ago | (#34536262)

he said share your bread. not charge for it.

Re:socialist ? (1)

Anonymous Coward | more than 3 years ago | (#34536804)

If he existed.

Paperclip, tweezers, crowbar & a strong flashl (2)

digitaldc (879047) | more than 3 years ago | (#34536234)

Trust me, they will come in handy in the lab at some point. Even for sudden headcrab infestations.

Re:Paperclip, tweezers, crowbar & a strong fla (1)

skids (119237) | more than 3 years ago | (#34536934)

End cutters. Don't forget the end-cutters. The flat claw kind, not the slanted ones. Great for stripped bolts.

Alas no recognition. (4, Insightful)

AndGodSed (968378) | more than 3 years ago | (#34536240)

I once took my laptop and used it to set up an Apache + DNS server while replacing a webserver that died. All I did was to post a "Emergency Maintenance" page while we swopped out the server.

Every IT guy who has been in the trenches for 10+ years has "I once" stories. Oftentimes they salvaged hundreds of thousands of rands of damages for the company, or helped mitigate a bad management decision.

The thing is, one of several scenarios invariably happen:

1 - You get no recognition because no one understands what you did. ("Oh, you had another web server running on your laptop, that's dandy!")
2 - You get an accusing look. ("How was it possible that this happened? Sure you fixed it but this should not have happened, make sure it doesn't happen again.") - I saw something like this happen to a senior network admin once, something totally out of IT's control that occurred due to a bad management decision not to buy a spare router. We used an old PC with IPtables to route traffic on a network over a weekend while our suppliers tried to source one.
3 - The dark suit analogy: Doing a good job is like spilling coffee on a dark suit, you feel warm all over, but nobody notices.

Being in IT is a bitch, and management doesn't help - IT is honouring the impossible promises of management to unthankful clients.

As opposed to every other job in the world... (0)

Brannon (221550) | more than 3 years ago | (#34536348)

where all employees are rewarded exactly in proportion to their value?

Do you have any idea the thankless heroics that school teachers, lifeguards, EMTs, nurses, firefighters, etc., etc., pull everyday?

IT has got to be the whiniest fucking field out there.

Re:As opposed to every other job in the world... (1)

AndGodSed (968378) | more than 3 years ago | (#34536390)

Lemme guess, you are not in IT huh. Or an IT manager.

Re:As opposed to every other job in the world... (2)

Ynsats (922697) | more than 3 years ago | (#34536710)

Looks like somebody doesn't want email access today, huh? (End BOFH mode)

Re:Alas no recognition. (4, Funny)

jjohnson (62583) | more than 3 years ago | (#34536404)

"How did this happen?"

"We were running without a spare router because you turned down the request I made to purchase one. I can forward you and your boss all the documentation on my request, including the cost analysis for suffering an outage like this because we didn't buy the router, if you like."

Re:Alas no recognition. (4, Funny)

ArsonSmith (13997) | more than 3 years ago | (#34536630)

That'd be a nice story to tell to the other people in line at the unemployment office. You can even show them the printouts of the email you made before you were walked out.

Re:Alas no recognition. (3, Insightful)

AndGodSed (968378) | more than 3 years ago | (#34536900)

"Yes but you should have made sure we made the correct decision. You are an expert in your field why did you not push your position harder?

Should I get another IT guy willing to take responsibility for his department or are you going to make sure we make the correct decisions in the future?

You are an adult, you should be aware that people can make the wrong decisions and be prepared for any eventuality.

Now go and get me a quote on replacement hardware so that I can make a decision and take it up with the MD."


Re:Alas no recognition. (1)

Ihmhi (1206036) | more than 3 years ago | (#34537358)

Speaking from experience, IT is one of those jobs where nobody notices or cares about you until something goes wrong.

Re:Alas no recognition. (1)

AndGodSed (968378) | more than 3 years ago | (#34537588)

And the question is invariably along the lines of "Hey why can't we access our email?" or "Why is the internet down." as opposed to "Hey were you aware that..."

IT guys are not bloody clairvoyant.

My response is usually either "I wasn't aware that [x or y]" OR "Working on it, talk later, cheers."

Car Battery (5, Interesting)

Ynsats (922697) | more than 3 years ago | (#34536242)

I once repaired a critical UPS that was attached to a critical database server actively recording data in the middle of a test shot with jumper cables and the battery from my truck. All that just to replace a fan that kept sending the UPS in to panic mode for an overheating battery and trying to start a shutdown sequence on the database server.It was a 12v power source for the UPS (old, old equipment) coming out of the AC to DC power supply. The UPS was part of a suite of equipment that included the database server, the array, a backup device, a network switch and the UPS hardwired to each of them in it's own rack. Don't ask me who made it. All I know is it was an Informix based DB and the maker was some esoteric, specific solution company I never heard of and before my time anyway. All I knew was the replacement parts had a 2 week lead time and I have no idea why this company chose to hold up such critical data with such arcane and unsupportable equipment. But, I had to shutdown the UPS to do the work but the battery didn't have enough juice to support the 30 minutes it was going to take to do the work. The battery power would have been killed once the unit was off anyway.

So I attached my jumper cables and the 600 amp battery from my truck to the output rails on the UPS, after the control switches. From there it was just juice to the rails and then to the server and it's data array. The car battery had about 45-55 minutes of juice for the suite to run on full-tilt. So I shut the UPS down and the servers, thankfully, stayed up! Had a box fan blowing on the battery and jumper cables. I disassembled the UPS case, cut the bad fan out and spliced the old connector on to the new fan I got at a local surplus store for $3. Plugged it all in, reassembled and turned the UPS on. It went through diagnostics and everything went green. Then the overload light started blinking and the warning chime came on. I pulled the jumper cables off and the overload warning went away and things stayed stable. The fan stayed on and nothing went down.

I probably should have gotten an award for it because it was a test shot for a multi-billion dollar contract but I was more afraid of disciplinary action over the risk than getting any praise for it. As far as I know, to this day, only two other people at that company know what happened

Re:Car Battery (0)

Anonymous Coward | more than 3 years ago | (#34536658)

You really thought the 600 amp battery and jumper cables needed a box fan to keep them cool? I guess this really was quite the hack for you...

FYI batteries and cables only need cooling if they are pulling close to their limit. Your server should be nowhere near (~2 orders of magnitude) the 600 amp limit of the battery and jumper cables.

Re:Car Battery (4, Funny)

Ynsats (922697) | more than 3 years ago | (#34537816)


Jumper cables are designed for short bursts of high draw. "High draw" being around 30-40 amps. Most starter motors in cars draw between 30 and 60 amps max, some diesels will draw up to 120 amps.

However, they are not designed for a constant draw of 30-60 amps. The cables will get hot from just trying to jumpstart a car. Having a rack with a server, a disk array, a network switch and a backup appliance draws a considerable amount of power. Even if each of them were all running at the typical 15 amp draw like you see from a 120V circuit, that's still 60 amps of draw (reality was more like 90 amps and 15A on 120VAC doesn't really equate to 15A on 12VDC). Twice as much as what a standard starter motor draws on jumper cables. Add to that the fact that it's a constant draw over a half hour or so and the thermal properties of jumper cable insulation becomes a factor. If you had more than a passing knowledge of automotive charging and starting systems, you'd know that.

The fan was already there and running to move air across the ailing UPS. Obviously in place to handle the heat problem caused by the failed fan. Whether the cables and battery getting hot would have been an issue or not wasn't a concern because the fan was moving air and dissipating heat. Even just the calming effect that a perceived reduction in risk due to the operation of the fan on the cables and battery does wonders for performance of the tech trying to fix the problem quickly.

A box fan, whether it's effective or not, is a small cost of insurance to eliminate a condition that is easily avoidable. Even if you are flying by the seat of your pants and operating on a UPS system with unregulated power coursing through the output rails. Why that is a point of contention, I'm not sure, but in true Slashdot commenter fashion, you've managed to nitpick an insignificant part of a story with incomplete information just to discredit and insult a poster over something of no consequence to you. Good job!

Slashdot takes the "fun" out of dysfunctional.

Re:Car Battery (4, Interesting)

digitalhermit (113459) | more than 3 years ago | (#34536734)


After a hurricane had wiped out power to Miami, I had to drive into a facility I maintained to get their email servers back online. It was critical for their remote employees to send in orders and time sheets. This was back before outsourced email services such as Google or Yahoo were available.

When I got there the power was still off. I had to rely on a 300W inverter plugged into the owner's truck battery. We ran a high gauge extension cord about 50' to the truck parked outside. Next we added a power strip to the end of the extension and plugged in the modem, server and monitor. On powering up the fuse for the cigarette adapter blew. We clamped up directly to the battery then. Powered up the monitor, modem, then PC. Everything worked for about 3 seconds until the BIOS splash screen turned on. Then it all went dead. The 300W inverter was not enough to power on both the server and the old CRT. We had the bright idea to charge a UPS for 30 minutes. With the monitor plugged into the UPS, we had just enough juice to see that the server has hanging on a bad filesystem. Then it died.

This is where it got fun.

I unplugged the monitor. As the system booted, I replayed in my head the steps I needed to bring the filesystem back. I knew that needed to login to maintenance mode first. I knew this by entering the root password then typing (blindly) "touch /tmp/foo; find /tmp -name foo". When I saw the hard drive light flicker when I pressed enter I knew I was at the shell.

I had to check the filesystems... I didn't remember what partition it was on, so on a piece of paper I wrote out an awk script that would peek through /etc/fstab, grab the relevant filesystems and the appropriate /dev entry, then pass that to stdout. I piped that output to a file then used that file to run fsck. All of this was done without seeing my commands or the output from those commands.

When the remote user was able to connect via mail then I knew it was working..

It wasn't particularly ingenious, but the circumstances made it memorable. Missing pieces of the room, navigating around downed trees to get to the site, complete darkness except for a door propped open on the other side of the room (server room was the farthest room in the office and had no windows or doors to the outside), hot hot hot hot hot (Florida weather), and users calling every five minutes trying to connect... Power came up later that day, but what an experience.

Unreadable CD/DVD (4, Interesting)

xded (1046894) | more than 3 years ago | (#34536256)

Look for scratches on the bottom side, brush with toothpaste (the plain one, no additional abrasive ingredients), rinse, read.

Re:Unreadable CD/DVD (1)

Anonymous Coward | more than 3 years ago | (#34536778)

A funny joke but as a person who has actually tried this on light to heavy damage discs I can say without a doubt...

It does not work.

Re:Unreadable CD/DVD (1)

gfody (514448) | more than 3 years ago | (#34538270)

worked for me have you ever seen a cd/dvd repair kit? it's just rubbing compound (about the consistency of toothpaste) and a buffing pad. extra gritty toothpaste could be a problem though

Re:Unreadable CD/DVD (1)

RogL (608926) | more than 3 years ago | (#34536916)

Look for scratches on the bottom side, brush with toothpaste (the plain one, no additional abrasive ingredients), rinse, read.

Or as happened repeatedly with a former boss:

Rush out a data CD for him to test, he tests it during lunch & it's unreadable...

Look for scratches on the bottom side, take it ito the men's room to wash off the peanut butter & jelly he'd gotten on it while eating, gently wipe dry & have him try it again. Deliver a stern lecture on the proper handling of CDs containing the master copy of the company's chemistry databases.

Re:Unreadable CD/DVD (1)

Spazztastic (814296) | more than 3 years ago | (#34538032)

Even if this was years ago, why would you keep something so important on a CD? Don't you have a network?

First thing title made me think of... (1)

horza (87255) | more than 3 years ago | (#34536264)

Stunts, Idiocy, and Hero Hacks

With a title like that, I was sure it was going to be another Wikileaks story.


crazy reasons of failure... (1)

excelsior_gr (969383) | more than 3 years ago | (#34536420)

I remember once back in the day, a friend of mine called me to help her with her new computer because there was some problem with the modem. The box had been delivered a few days back and everything seemed to be working perfectly but the modem would not connect to her ISP (dial-up). I tried *everything* on both the software and the hardware side. The support guy also had no idea. Everything seemed ok on their side, too. The drivers were ok, the hardware also, we were dialing the correct number, the username/password was active. Then I set the volume of the modem real loud. And lo! The modem would whine and purr, but amongst the whines and the purrs there was something like a female voice coming out of the metal box...

Then my friend suddenly stood up, pounded her head against the wall (metaphorically speaking) and explained: the part of the town where she was living had a *really* old telephone network: if a neighbor was making a long distance call you could easily get charged for it and visa-versa. To avoid such problems she had called the telephone company and had a barrier placed that would block all numbers starting with a zero! When the modem dialed the ISP number that also started with a zero but would give her cheaper internet prices instead of normal call prices a recorded female voice would answer the call explaining that a barrier had been placed for such outgoing calls! Of course, the modem would ignore her and try to mate with her with no success...

Re:crazy reasons of failure... (1)

AndGodSed (968378) | more than 3 years ago | (#34536624)

30% of /. would try to mate with her with no success...

The rest would never even think of trying.

Recover deleted data (1)

patrick_leb (675948) | more than 3 years ago | (#34536436)

After about 4 weeks into my first programming job (~15 years ago), I lost all the source files in my cwd by mistakingly typing "rm *>o" instead of "rm *.o".
Of course at that time there were no tape no backups and my last commit was about a week or two earlier. I went to see the sysadmin and explained my situation.

In about 2 minutes he wrote a C program that opened and read from /dev/hda in blocks, looking for some variable/function names that I had provided him. This
yielded a big text file, and it took me about a day to untangle and reconstruct the various source files. I remember that had impressed the hell out of me.

Re:Recover deleted data (1)

ArsonSmith (13997) | more than 3 years ago | (#34536692)


strings /dev/hda > text.out

Re:Recover deleted data (1)

patrick_leb (675948) | more than 3 years ago | (#34536800)


strings /dev/hda > text.out

That gives only the strings, I needed the entire text blocks to reconstruct the source files.

Anonymous Coward (0)

Anonymous Coward | more than 3 years ago | (#34536458)

Had a similar story, worked as an intern in a plant, plant security system (286?) had a 10 MB disk that DIED (would not POST). Replacement unit was $1200 from the supplier. Managed to 'unstuck' the armature by 'twisting' the exposed armature shaft (allowed the machine to boot). System ran for months until it was replaced.

City Sticker (1)

GerryHattrick (1037764) | more than 3 years ago | (#34536460)

Late at night doing the stock prices, if one card of the COBOL pack was wrong, you'd find a punched-out confetti on the floor, and stick it back in the errant punch-hole, using a tube of polystyrene cement. Quick dry, no snagging, no delay. Just don't run those packs if you find them crisp in some archive.

Shaked not baked. (1)

fahrbot-bot (874524) | more than 3 years ago | (#34536534)

From stunt #2 (bake the hard-drive):

...the disk presented to the SCSI controller just fine -- but it also didn't seem to spin up at all...

Ya, I had that happen once, but I simply rapped my SCSI drive with the handle of my screwdriver - hard - right on the spindle head while the controller was trying to spin it up - "WHACK". Sucker spun up on the second hit. Still works fine.

Re:Shaked not baked. (1)

clone52431 (1805862) | more than 3 years ago | (#34537250)

Sounds familiar. I had an external hard drive with a lousy USB cord that wasn’t delivering the full juice needed to spin up the drive. When it was plugged in, it just made a sad clicking noise. Hold it in the palm of your hand, though, and a gentle twitch of the wrist would start it right up.

Xbox 360 (0)

Tokolosh (1256448) | more than 3 years ago | (#34536540)

Let me tell you a crazy story about how I got my Xbox to work after a RROD!

I involves chewing gum, rubbing alcohol, a tweezer, a prophylactic and a wire coathanger...

Re:Xbox 360 (1)

evilbessie (873633) | more than 3 years ago | (#34537552)

Black Jack and Hookers?

Done a few splices in my day. (1)

Nadaka (224565) | more than 3 years ago | (#34536572)

Simplest was splicing in a 4th drive power cord for a machine that needed another drive.

Also wired an ISA card directly to the motherboard after the socket snapped off.

Wired a laptops power supply to the motherboard, again after the socket ended up broken.

Had an old sparc station that had a pin broken off of the keyboard/mouse cable and had to wire that together as well.

Re:Done a few splices in my day. (0)

Anonymous Coward | more than 3 years ago | (#34536970)


Re:Done a few splices in my day. (1)

Nadaka (224565) | more than 3 years ago | (#34537674)

Wow? Really? It really isn't impressive at all. Just judicial application of bits of wire and solder. Compared to a lot of the other hacks it seems like pretty small stuff.

Re:Done a few splices in my day. (1)

clone52431 (1805862) | more than 3 years ago | (#34537924)

Not sure what exactly the Anonymous Coward was reacting to, but that ISA card... damn, that’s a lot of pins.

My soldering exploits have been limited to more small-scale hacks, like re-attaching the plug on a USB thumb drive which had cracked the solder joints attaching it to the drive, or replacing the mechanical switch in the primary button of an optical mouse (which I’d worn out playing Minesweeper... how many people can say that, hmm?) with a similar mechanical switch cannibalized from a rolling-ball mouse. Well, that and scavenging the switches, LEDs, and various electronic components from just about anything electronic that we threw away when I was a kid... in fact just last week my dad called me up and told me he’d needed a small switch or something like that and remembered my old stash down in the basement and he’d found just what he needed in it.

Jackass #2 related (3, Interesting)

Caerdwyn (829058) | more than 3 years ago | (#34536678)

In the Dim Times, my company had a couple of hard drives (those newfangled 3.5" Scuzzy drives) that wouldn't spin up and had critical data on them. My solution:

  • 1. Find a long internal-type SCSI cable (about 30").
    2. Hold the drive in my fingertips (so the platters were parallel with my palm)
    3. Power on the computer, then "snap" the drive with a twist parallel to the platters, relying upon inertia to break the stiction.
    4. Recover data from now-spun-up drives.
    5. Power down, then physically destroy the interface pins on the drive to ensure nobody tried to use it again.

Since then, I've used that trick several times on dead/dying hard drives. As long as the heads are trying to move (indicating electrical life), it's worked every time.

Nine-track cat-o-nine-tails (2)

OmniGeek (72743) | more than 3 years ago | (#34536712)

I once was a sysop for a small company's Data General system, where large datasets were stored as TAR archives on nine-track tapes; some poor soul had copied TO the tape instead of FROM the tape, and desperately needed to recover a file that was still there on the part of the tape beyond the end of the inadvertent write. You could read up to the added end-of-tape marker, but the tape just wouldn't read any further. Screwed, yes? Well, not quite. I set the system to rereading the damaged tape, waited 'till just before it reached the offending end-of-tape marker, and briefly put my thumb on the roller that measured tape travel, causing the drive to jump the tape ahead ('cause the sensor said "the tape is not moving!") and right past the EOT marker. Voila! The system read out the rest of the files on the tape, fortunately including the one they really needed, and I was briefly a hero. Hero never lasts, of course, but it was fun.

Stubborn Hard Drives (2)

beadfulthings (975812) | more than 3 years ago | (#34536720)

I'm surprised they didn't mention the technique for unsticking recalcitrant half-height RLL and MFM hard disk drives by slamming them gently, but firmly, down onto a smooth horizontal surface (like your desktop). They would occasionally stick when the heads became goo-ed to the platters due to breakdown (or solidification, I was never sure which) of the lubricating material. When all other hope was abandoned, and you knew the drive was headed for the graveyard, a good, solid (but gentle) whack would often get it spinning again. The idea was to keep the drive as parallel as humanly possible to the horizontal surface. It was one of the few hardware tricks I had to summon male assistance to handle--my hand was not large enough to get the necessary firm one-handed grasp on the drive. Boy, do I feel old. Probably because I am old.

Re:Stubborn Hard Drives (3, Funny)

dargaud (518470) | more than 3 years ago | (#34538092)

Like they say: "Hardware is the part of the computer that you can kick..."

No Cable, no problem (0)

Anonymous Coward | more than 3 years ago | (#34536750)

This was about a decade ago. I had to get a serial cable working, but did not have the proper cable. I got the doco for the massive UPS I was working on (required strange cable config) grabbed a CAT-5 cable and tore apart an existing serial cable. One of the wires had to be a loop back, which I had a devil of a time doing. Eventually, I connected it by using my thumb. Baked the inside of the thumb actually. It was quite uncomfortable for two or three days.

dont use print links (0)

Anonymous Coward | more than 3 years ago | (#34536782)

I once used the articles full url, instead of the print URL, so that the company who paid the writer, the web developers, and hosting company might actually get a little benefit from my visit.

The carousel from hell (1)

Combatso (1793216) | more than 3 years ago | (#34536864)

In a previous life as a network admin, I went to check out a warehouse that my department was taking over.. Everything was normal, as I made my list of comptuers, and started writing a to-do list for myself.. on about the third day I noticed the picking line had stopped.. I went over to the pickers area and saw the local IT guy opening up a computer.. so I asked what was up.. He said "oh this hard drive sticks now and then"... it was an non-network 486 running Windows 95 that had the entire carousel database on it.. without the info on that drive there was no way to get ANY parts out of the carousel, let alone run the carousel... anyways, he popped the case off, and the harddrive was already opened up, he then spun the platter by hand to "kick start" it... I asked how long it had been like this, and he told me "a few years now, but in only happens every couple of months"..... long story short, I replaced that computer that day, put the DB on the network and when the acquisition was complete, I didn't offer him a job,.

Fire Axe (5, Funny)

PPH (736903) | more than 3 years ago | (#34536896)

Back in my days as an engineer at Boeing, I supported some automated test equipment on the factory floor. One day, one of the ATE failed to download the required s//w update, so I was called out to investigate. It turned out that the network drop adjacent to the equipment had been disconnected in the nearby network closet. (locked, of course). So I, with the factory manager in two, called the IT department to get it plugged back in.

Me: "I'm in the Renton plant, at column XYZ and we need this network drop reconnected. Production has been halted."

IT Operator: "OK. We'll start a ticket on that. But standard turn-around is 24 hours".

Me: "We can't wait 24 hours. We need to get this equipment updated to get the line up and running. Is there any way to escalate this?"

IT Operator: "Sorry. That drop is was identified as being inactive and was unplugged."

Of course it was inactive. The ATE is only powered up when needed. At other times, the little light on the switch in the closet would be off.

At this point, the factory manager asked for the phone. Very calmly, he spoke to the IT operator.

Manager: "You can cancel that ticket. My engineer assures me that he can reconnect the drop once he gains access to the network closet. The plant fire department is just downstairs and we'll have them bring up a fire axe to open the door."

The IT department dispatched a tech who arrived within 15 minutes.

Motherboard Troubles (0)

Anonymous Coward | more than 3 years ago | (#34536980)

I once heard a story from an individual who worked for a motherboard manufacturer. One day his boss came in with a frown and a few motherboards. The company had manufactured a whole bunch of these boards and NONE of them worked (I got the feeling this was from the "50MB HD, 60Mhz processor days). For the next few days he looked over the schematics and boards. Until he noticed that there was a circuit on the board that wasn't on any of the plans. He raked a screwdriver through the circuit and then plugged it in, it worked fine. For the next few weeks there were several employees whos sole task was to do the same thing to box after box of motherboards. I imagine he got quite a pat on the back/bonus from his employer, but probably more than a couple angry lears from his fellow employees for the next few months.

Hard drive in the freezer - it worked! (1)

Dr. Spork (142693) | more than 3 years ago | (#34537098)

I don't think my parents were ever as impressed with me as on the day when I rescued their data, including many years of precious photos, from their crashed hard drive. After diagnosing the click of doom on their drive, I wrapped it in a towel, then two bags of that blue freezer gel, another towel and a plastic bag, and in this state left it to freeze overnight. It had a SATA cable and a SATA power adapter cable sticking out, and I did my best to seal the plastic bag with tape to avoid condensation once I took the thing out. The next day, the wrapped, frozen hard drive actually booted and was able to transfer a few gigabytes of the data onto a new drive. I repeated the procedure until I rescued the rest. I still can't believe that worked!

Harddrive in the freezer (1)

arhhook (995275) | more than 3 years ago | (#34537274)

One of the coolest [no pun intended] I did was to place a hard-drive in the freezer overnight so that I could recover data from it the next morning. It took about 3 or 4 days to completely recover the data I needed to, but it definitely worked wonders.

Re:Harddrive in the freezer (1)

arhhook (995275) | more than 3 years ago | (#34537510)

One of the coolest [no pun intended] I did

I accidentally my own comment.

One of the coolest [no pun intended] *HACKS* I did

Minuteman Missile System (4, Interesting)

Sanat (702) | more than 3 years ago | (#34537410)

Back in the early 60's I was on a three man combat targeting team and we had two minuteman missiles to startup and target one day. So we went to the first site and the maintenance team had just finished installing a new guidance and computer package and the nuclear warhead. They closed the 80 ton door that protects the missile and so it was out turn to perform.

We started up the on-board computer and ran some checks and then began loading in the targeting data such as whether it was a air burst or ground burst and all of the war-plans associated with it as well as the launch codes and targets.

After this is accomplished then the guidance package goes through some testing and self calibration and finally becomes "ready"

Ready is actually called "Strategic Alert" and lights a green light on our console.

The missile system sat in strategic alert for a few minutes and so we figured we had completed our job and would button up the site and head to our second site.

Suddenly the "Launch Commanded" light lit on the console and a fraction of a second later the "Launch in Progress" light also lit.

I quickly popped out a bunch of the circuit breakers on adjoining panels causing the support equipment to stop functioning.

At this stage we did not know if we had a bad console (portable between sites) or a computer failure on-board. Anyway the missile did not blow the umbilical nor launch so we believe we stopped it just in time. If we tried to check our technical data then we would have been dead most likely.

We contacted job control and they agreed not to attempt a restart and rather have maintenance replace the guidance/computer package yet again and return it to Autonetics for repair.

The next site we went to for startup went perfect and the console worked flawlessly...

That has been nearly 50 years ago now and i still occasionally wonder if the missile had actually entered "launch" or if the on-board computer was giving erroneous launch status.

Real IT heros (1)

fishbowl (7759) | more than 3 years ago | (#34537796)

Real IT heros have things like routine backups with offsite storage rotation, N+1 replacement hardware policies, and sensible password policies.

Hacks aren't limited to computers (1)

Anonymous Coward | more than 3 years ago | (#34537830)

The (only) supply pipe from the lake reservoir in my city got a hole in it about 15 years ago. Ever since, it's been held together with a 2x4 and metal strapping.
I love the look on the faces of the new city councillors when public works explains why we need a new pump station.

Harddrive not in freezer but ice water bath (1)

bemenaker (852000) | more than 3 years ago | (#34537864)

I had to recover a drive from a test machine that was holding up about $300K worth of product from shipping. I didn't have a backup of this machine because someone deleted a folder on the file server recently. (I still don't know who did that). I tried freezing it, but the drive heated up too quick to get the data off. So, I took the drive, put it in a ziploc bag, submerged it except for the opening of the bag with the cables coming out of it in a bath of ice, water, and salt. The combination kept the drive cold enough to run a full disk to disk copy. We got the shipment out that night.

Stunt # 6 is inelegant. (1)

nuckfuts (690967) | more than 3 years ago | (#34537986)

A small addition was made to the autoexec.bat on the client, simply to run curl to access the Perl CGI script, then feed the output to the settz utility, thereby properly setting the time zone of each client every time it booted

Being able to modify the autoexec.bat file, they could have written a solution that required no third-party software. I used to change all manner of systems settings via .bat files, even modifying registry settings by creating .reg files on the fly and calling regedit to load them.

eeprom eegad (1)

lysdexia (897) | more than 3 years ago | (#34538080)

The old pizza box NeXTstations, if left to sit powered down too long, would sometimes lose their boot proms. My friend Jake had one go out, so he booted machine a, took the prom out, put the prom from machine b into it, wrote the bootloader back into the prom, then put prom a's prom into machine b. They both booted on next power cycle. Much sighing of relief and drinking of tea followed.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?