mandelbr0t's Journal: Aging City of Calgary Network Infrastructure

My previous entry mentioned the creation of the new IIS business unit at the City of Calgary. Mayday marked their first spectacular failure. My source reports rumors that the tax bill print run failed twice over the period from 30 April until 5 May. Apparently, the entire tax database needs to be put into read-only mode for the entire duration of the printing of over 200,000 tax bills. This was done on Friday. The first run failed on Sunday, leaving the tax database in read-only mode at start-of-business Monday. The second attempt failed, with an Out-of-Memory error. Read-write access was restored only at end-of-business Wednesday. At twenty City employees had work backlogged as they require read-write access to do their work. They were delegated to following up on the backlog by contacting each and every customer affected, and advising them that their request would be delayed. Of course, now that read-write access is restored, those twenty employees will have to go through each and every request again, to do the work they should have been able to do in the first place. Minimum total cost to the taxpayers: 20 employees x $200/day (guessing) = $4,000. Maximum could possibly be 3 to 4 times that.

An out-of-memory error is a thing of the past (or a sign of really bad programming). Are CoC servers really that old? Is the processing of 220,000 tax records such an encumbrance to them that they require 5 full days to make it happen? I hope this is not a sign of things to come. If every single mistake from IIS has this much of an effect on taxpayer cost, perhaps the new business unit was a mistake. My bad feeling has gotten much worse; chaos will reign soon if they can't make their aging infrastructure reliable.