cmacb's Journal: Open Source Software in Government

Comment originally made to an article, saved here for "posterior":

I don't think the issue is so much with individuals and what software they choose to use. If you are an avid game player and have to have Windows to run your games that's fine. Do so at home, with my blessing. In business and government, it's another matter, particularly in government. Here are my two first hand experiences... compare with your own...

At the Department of Energy I worked with a group of 100 or so OS/2 users. This was back in the early 90s. They loved OS/2 and had no desire to change. As an autonomous department, they technically could run any software they wanted. However when it came time to upgrade their PCs to a newer generation of hardware they were given an ultimatum: Switch to Windows or keep your old PCs. They eventually switched.

At another well known federal agency there is a group of people who love IBM computers and operating systems. There is also a group of people who hate IBM and anything connected with them. More importantly these two groups hate each other and are in constant conflict. Since their systems have to talk to one another there is ample opportunity to stab each other in the back... cause something to fail (or just wait for a natural occurence) and then try and blame the other group.

Many years ago the anti-IBM crowd decided to build a system based on Wang mini-computers. The system basically sucked, but it wasn't a good career move to say so. The only reason they migrated off the Wang systems was that Wang went out of business. In fact they ran the system out of used parts for quite a while before declaring the situation an "emergency" which meant that huge amounts of money were spent for a quick conversion effort that should instead have been carefully planned.

They picked Windows as their new target architecture. I'm not sure that this was necessarily a bad decision, and in fact there were parts of the "plan", such as it was, that tried to encourage the use of "standards based" softare. This means that you write your programs to use, to the extent possible, generic SQL (for example) rather than Oracle, DB/2, or SQL Server syntax. Because it was an "emergency" however, these sound business concepts were ignored and the system became locked into specific DBMS/Compiler/Operating system ways of doing things. Seven years later and the system is still buggy as hell. The application is written in a now non-supported programming language, but the only fix for this would be another total re-write.

At one point a group I was involved with was asked to recommend some statistical analysis software to allow for ad-hoc queries of this 7 years worth of data. Using live data the analyst compared several potential products and rated them. As part of the summary he pointed out that while several of the proposed products were quite capable, he had noticed during the tests that almost no column of values in the database had sufficiently enough valid data points (both missing and mangled values) to draw any statistical inferences, no matter what product they picked. Both analyst and report were "shuffled off to Buffalo" never to be seen again.

I was there for another year or so after that. There was begrudging talk about the lack of wisdom in continuing to rely on non-supported components. Jokes about the similarity to the Wang systems were getting too common. They went to one of the top consulting firms for independent outside advice (a very good idea in my opinion). After months of study, they issued an analysis of just one of the many applications there. Not surprisingly they said it was too dependant on the quirks and features of a particular DBMS. It also was using outmoded client/server methodologies and of course the non-supported compiler was full of bugs as was the resultant application. They also threw in some concerns about Windows security, which was just starting to show up on the radar for large geographically dispersed organizations (network dependencies).

I was encouraged by the accuracy of this report and looked forward to their final finding that would make recommendations about how best to clean up the mess that had been made. Thirty minutes into the presentation of those final recommendations (it was a small "closed" meeting of about 8 people) it was clear that the recommendation was going to be to move to Java, standardized SQL syntax, and to *consider* alternatives to Microsoft for server components. The presentation was interrupted however and after a few Q&A type interactions it was adjourned by the government employee who sponsored it. He indicated that he didn't want to embarrass his boss by reporting back that there were any major problems with the path we had already taken. The report was "corrected" to indicate that everything was just fine as it was and now they are sending people to classes to learn how to use .Net (even though there is no .Net interface to the broken compiler they are using).

Not only our tax dollars but our very LIVES may depend on the quality of decisions made in meeting such as this, but I am quite sure that petty personal politics and ego play a larger role than anything else. I think the world will get this right. For once I have to say that Europe seems to be on the right path. China, Brazil, India too while they may be choosing Open Source for different reasons (lower cost, less dependency on U.S.) will ultimately be in a position to make the right choice on purely technical grounds. I'm not at all sure we (the U.S.) will be in such a position. I see us locked into a monolithic way of doing things based on back-room deal making rather than good technical decisions. The government people making these decisions in many cases have NO technical background and are more interested in selecting a technology that supports their digital camera or game playing habits than anything else. I have no doubt that the basis for many of these techniccal decisions are made on the golf courses of expensive country clubs and not in meetings that include a technical point of view.

Not only does Open Source need to be on the table during these decisions (which implies that there is a table involved and not a 4-iron), but the decision making process needs to be open as well. If you think there is a compelling national interest in running only Windows based software for programs that affect national security I'd like to hear what it is. My own inclination from what I have seen first hand is just the opposite. As a tax payer and citizen I am outraged by what I have seen and more than ever suspicious of decisions that I only read about in the news. If you care about your country, you should be too.