Why IT Projects Fail

The large number of projects that are cancelled every year tell us that something is not working well in software engineering. What is it?

Software engineering is a discipline that is suffering from a cancer, a cancer of naivety in project planning. When we start to construct a building, we don't abandon it half way through because it doesn't meet the requirements of the users. When we undertake the construction of a canal, we don't give up saying that it failed to deliver water to the appropriate areas. Or if that does happen, there are a lot of people who get into real trouble.

Software engineering, however, is different. Each year thousands of failed projects are cancelled.

I am not going to try to offer a silver bullet in an article as short as this for the problems that hundreds of  illustrious experts, such as those in the Project Management Institute, have spent decades studying. But I am going to attempt a summary of the hidden causes for the failure of projects that apparently have been well designed and planned.

1. Programmers' common sense says that if the specifications are good, the design is good, the implementation is good and if quality control is good, then the project will be a success. But this is false because a project is a project, and not a set of specifications or an architecture. A project is a mission shared by a group of people.

2. Sibylline withdrawal of resources. The majority of large projects that fail do so because the resources necessary to complete them are reduced in a subtle way. Every bricklayer knows that there is a correct ratio between lime and Portland cement, and that you can't remove 5% of the steel in a building just because the price of steel has gone through the roof. By contrast, in software engineering it is normal practice to contract someone with 3 years of experience instead of someone with 5 years, and sometimes it is not even felt necessary to recruit someone with an IT qualification. It doesn't seem to matter if 9 months are shortened to 8 or a budget of 100,000 euros reduced to 90,000. Little savings are made all over the project until they reduce the probability of success.

The difficulty of coordinating effort. Another unacknowledged reason for failure is the difficulty of coordinating a large number of independent parties with conflicting interests. The greater the size of the group the less chance there is for them to work together. This is especially true in the case of external suppliers who, by their very nature, are looking out for themselves and will try to eliminate any potential competitor. This struggle often results in the disruption of the team environment and the death of the project.

3. Artificial obstacles. The third reason is the blocking of initiatives at a given moment that could save the project. This could be for political reasons, or because someone's ego depends on things being done in a certain way. The problem is that these do not allow the software engineers to work. Absurd restrictions and insane kafkaesque procedures are imposed on programmers that obviously were not foreseen in the initial plans.

[Translator's comments. I am sure any developer involved in a large project can identify with these points. My only issue is:

People often try to draw parallels between constructing a building and constructing a software system, and then point to the high rate of failure of the latter as something that needs to be fixed. This is not an appropriate parallel. The correct parallel is between the design of a architectural artifact and the development of a new software system. Both are subject to frequent changes as more information is revealed during the development and in neither case is failure unexpected or even unwanted. Once a software system has been designed, then its implementation in a new environment is often just as reliable as the construction phase of a architectural artifact, given the relative immaturity of the discipline of software engineering.

See also the superb article by Joel Spolsky, Hitting the High Notes, on a similar theme.]