Journal Alioth's Journal: The worst week 1
The worst week of my working life is now over.
Next week can go either of two ways: the second worst week of my working life. Or it could be even worse than this week. Fortunately, it's not something that's my fault, I'm just involved in the recovery and developing the safeguards that have to come from this. (My part in all of this is development of back end systems and DB administration, none of which was involved in the monumental cockup). One of the things we do is process order forms for our biggest customer. We process around 120,000 forms per week (of which about 20% are order forms). Due to a conjunction of circumstances, some 6000 forms were processed incorrectly, and 6000 incorrect customer orders went out. Part of the problem is the form processing software has a highly unexpected behaviour when processing double sided forms - it does NOT treat the two images from the duplex scanner as an "inseparable unit". It tries to smartly link them by recognising the form, and if it gets it wrong it can associate the wrong front of the form (which has the order) and someone else's back of the form (with the customer details). A conjunction of this unexpected behaviour, a lack of adequate user training, too much comfort over the process, and human error resulted in things going tits up.
If we finish the recovery next week, then it'll only be the second worst working week I've ever had.
If the customer also fires us, it'll be by far the worst - a catastrophe. It's cold comfort thinking how I'm not to blame while seeing 30 fellow employees getting laid off just before Christmas. I was only half joking when talking to my boss about buying a one-way ticket to South America and never coming back.
Small comforts... (Score:2)