When Our Systems Make Us Stupid

Photo: Gopal Kapur. This article is edited from a speech given by Gopal Kapur, and it first appeared in Government Technology's Visions Magazine in Sept. 1997.

For years, the River Nile flooded each year, and rich sediments came down the river. Farmers would use this as free fertilizer, making the Nile River Valley one of the most fertile in the world. The Aswan Dam -- built at enormous expense -- stopped that flooding.

So now there were no natural fertilizers for the farmers. Gigantic factories were built to replace the fertilizer. Unfortunately the factories now use nearly all the electricity produced by the dam. The chemical fertilizer is too expensive for most of the farmers, because they were used to getting it free. In the meanwhile, the dam is filling up with sediments.

Why We Fail

The first reason we fail is the belief that any system set up will solve our problems. The second is the belief that complex systems can be made to function to achieve its objective. Both of these are faulty beliefs, because when a system is set up, a new entity comes into being whether or not it solves the problem. And no matter what the purpose of the system, embedded in it is what we call "system behavior," and the first system behavior is that complex systems tend to oppose their own proper behavior.

Another example of a complex system opposing its own proper function is a multi-million dollar project of the Motor Vehicle Department in the state of New Jersey a few years ago. Within days of setting up this system, the DMV found it did not have enough time at night to process all the tickets that were issued by the Highway Patrol, and the system was too slow.

Within a few weeks, 1.4 million transactions backed up. So how do they solve the problem? The governor of New Jersey was forced to issue an Executive Order to the Highway Patrol not to cite people for traffic violations and this included driving at high speeds.

The moment this Executive Order came, drivers from neighboring states were quick to discover his gift. They began to drive through New Jersey at high speeds just to irritate the Highway Patrol.

Another example is that at 2 a.m. Sunday October 27, 1985, Amtrak came to a halt all over the country and remained motionless for one hour. Passengers were notified that the nation was switching back to standard time from daylight savings time and the trains were waiting for the clocks to catch up. Aren't we glad the same people did not program the airline flight control systems? Now that was in 1985. You could say "Well, that was the first time, we'll learn from it, we won't do it again."

Now consider an example from San Francisco's Bay Area Rapid Transit (BART) system. In an attempt to run more trains per hour, BART upgraded its hardware, wrote new software and redid its computer system.

During the first few days after switching to a state-of-the-art system, if a train was behind schedule, it would run right past the station. That is exactly how they programmed their computer.

This feature was not appreciated by riders, who missed their station and had to get off and go back. A new term was coined: station deja vu, as in "I have seen this station before." However, once you got on the train to get back to the station which was behind you, there was no proof that it would stop there. As a result, most trains ran on time -- however, most customers were late getting to work. Now this is the clincher -- BART management made this change on one of the busiest days of the