Why Projects Fail? The Devil is in the Detail
Some Hard Lessons from Government IT Projects
This article presents some practical insights and challenges encountered during the implementation of major IT projects in the government sector in Arab countries. The primary purpose of this article is to point out the identified pitfalls to the existing body of knowledge from a practitioner’s standpoint, as many of the articles published in this regard are published by vendors, consultants, or academics. Each item is discussed to highlight how it impacted the management and the overall performance of projects. They are believed to contribute significantly towards the successful management and implementation of projects, and as valuable lessons that should be recorded in an organisation’s knowledge and watch list repository.
It is widely accepted in the literature by both academics and practitioners that information technology projects have very high failure probabilities and that between 60 to 70 per cent do actually fail. Many other researchers argue that the actual figure might be far more frightening since many organisations tend not to disclose such experiences, due to fear of criticism by audit or the media (Collins, 2006; Cross, 2002; Fichter, 2003).
Perhaps this may be attributed to the fact that current information technology is more complex and diverse than several years ago as it is moving out of the back-office and into more mission-critical business process e.g., customer interfaces, electronic commerce, supply chain management, etc. (Gartner Group View, 1999). Besides, many researchers pointed out that many of today’s failures are avoidable (Avison & Wood-Harper, 1990; Bentley, 2002; Berkun, 2005; Broder; 1999; Curtis, 1998; Lam, 2003; Radosevich, 1999). They argue that many of the projects fail because of foreseeable circumstances and that organisation’s need to give careful attention to several factors to reduce failure.
The findings of this article correspond with the often quoted statement in the literature contributing to failure and that is related to the fact that organisations tend to treat IT projects from purely technological perspectives, and do not give much attention to other organisational issues. Almost all challenges and pitfalls reported in this article were organisational issues related to management and people. Figure 1 provides an overview of the pitfalls.
Figure 1: Projects Pitfalls Overview
The identified elements hindered the progress of projects and delayed them from hitting the planned go-live milestones repeatedly. The elements pointed out here are considered to be some valuable lessons learned during the implementation of several government IT projects and that if understood thoroughly could minimise the potential problems with the management and resultant delays in similar projects elsewhere and smooth their implementation. They need to be comprehended and understood by the key stakeholders in projects and organisations, and not only the project management professionals. The following sections look at each encountered project pitfall individually.
2. Holistic Approach in Planning the Projects
One of the key problems with especially IT projects is the fact that the basic business principles associated with each project are sometimes underestimated or totally ignored. The reason often being a total lack of knowledge or ignorance in regard to the simple principles of sound Public Administration. No project can be implemented in a smooth and time scale manner if the total business process is not determined or comprehended up front. This entails the identification in detail of the different processes, supporting legislation, rules and regulations to be applied, what will be required in terms of finance, human resources, accommodation and related requirements. The IT divisions should only be tasked with execution once these basics have been determined. For example, once the different processes required by the project are determined in detail by the business experts, it becomes easy to determine the required skills necessary to attend to all the different aspects of the project.
Personnel working on the projects should therefore consist of a blend of business as well as IT experts. Decisions regarding the business issues will be taken by business experts and not vice versa. IT experts will therefore be given a clear indication of what is required by the business experts and can apply their expertise to execute it in the best possible way. Through this interaction between the business and IT experts the best possible solutions and/or decisions will be implemented.
As a result of poor planning, all the relevant aspects of the project are not taken into consideration, resulting in unrealistic target dates being set. Often when these target dates are not reached, a scapegoat is searched for to carry the blame. It is in this instance very often seen that the magnitude of a project is underestimated to such an extent that unrealistic timeframes and closing dates are set for the project. This is typically the result of inexperienced persons planning a project.
Without a clear understanding of what the project entails, it is often found that the wrong skills are appointed or they are appointed at a very late stage in the project and there is always the tendency from these appointees to re-invent the wheel. This results in endless disagreements from both sides and causes delays without real value being added to the project. It is therefore of the utmost importance to appoint the required skilled personal on the business and IT sides from the outset, in order to avoid these fruitless differences and delays.
Provision is sometimes not made in advance to accommodate equipment, personnel working on the new project and for the public to be served in a user friendly environment. This also leads to endless delays and deadlines have to be postponed on a continuous basis.
There should also be a clear line of communication between the project management and the sponsor of the project. Without that, vital information is sometimes “withheld” from the sponsor or his decisions are anticipated only to be rectified once he becomes aware of it.
3. The Dilemma of Determining Realistic Timeframes
In almost all of the implemented large scale projects, each conducted review of the projects schedule confirmed a delay of several weeks and in others months related to the target date for the formal project kick off. The following factors were the prime delaying factors that obstructed the ability to maintain the published schedules:
– The amount of work that had to be performed was underestimated and the process to be followed was not clear,
– Underestimation of the requirements of the projects up front often leads to an inappropriate solution offered by the Vendors. This solution is then included in the contract and vendors are often, with good reasons, reluctant to deviate from the contract for a fear of scope creep. The requirements should therefore be absolutely clear and signed off by a skilled business expert who should take responsibility for the project. The proposal and associated contract should likewise cater in detail for all these requirements. This will ensure changes to the system and contract will be restricted to an absolute minimum,
– The identification of project activities was not easy and the time required for their accomplishment was too short,
– The approval of system specifications took longer than specified. This was mainly due to an unrealistic date set for this as a result of inadequate specifications, the complexity of the system and also to the unavailability of business experts, some project staff and IT experts for decision making which consequently caused a postponement in the deliverables’ dates for many specification documents and project activities,
– Far too much emphasis was often placed on the security of the systems from an IT perspective resulting in a closed system being developed by the vendors. The systems were therefore difficult to operate, not user friendly to the public and it was very difficult to affect any changes at a later stage,
– Team members not skilled for the task were sometimes compelled to take important decisions on certain issues only for these decisions to be revisited. An example is when IT experts have to make business related decisions and business experts have to make decisions on technical issues,
– Decisions were sometimes taken in good faith by management team members only to be reversed when more senior members or the project sponsors become involved. It was therefore of the utmost importance to ensure that decisions on critical issues were cleared with higher authority before it was implemented. There should be a structured way in which these issues are dealt with.
– Vendor system development process was complex and could not address ad-hoc modifications to the system,
– Outstanding contractual issues that could not be resolved at the technical levels took longer periods until they were resolved on the executive management level,
– The legal requirements for some organisational aspects related to sharing of data with other government organisations ran through long government cycles,
– Recruiting the required staff was a big challenge especially for those jobs requiring highly skilled and knowledgeable candidates and was often finalised at a very late stage;
– Communication and coordination with other government entities were a daunting activity and the delays in reaching consensus and getting approvals had an impact on the completion of project activities, and
– Consultants working on the projects were often experts in specific areas of the IT environment but regarded themselves as universal experts even in advance business processes. This resulted in the system often being steered into a closed and secure IT project with little room for change. This was especially a problem as some consultants were part of the team negotiating contracts with vendors. Further, western consultants had a tendency to cleverly, but purposefully, add additional requirements to the system which were time consuming, resulting in endless discussions and more than often required a rework of the system. This caused “legitimate” delays and extended the project to an extent where the contract of the consultant also needed to be extended for a further period of time.
Besides, project managers tend to either produce plans that were too broad in scope, with insufficient detail or which were too detailed. Large projects had detailed schedules. However, it was found impractical to use detailed plans for reporting to the steering committee executives, where they were usually interested in whether or not the project was on target, and they could not see this in the mass of detailed activities. This requirement was sometimes difficult to obtain as project managers tried to hide the potential delays from the top management. It is however of the utmost importance to timeously provide the sponsors of the projects with detailed information outlining problems when experienced in order for informed decisions to be taken at the highest level. If not, these problems will be identified later with inevitable consequences.
In general, the process followed in the projects was more of planning the project activity-by-activity. The assumption was that as soon as the sub-projects were started, more information will become available about the other activities. We call this ‘management-by-luck’. So not surprisingly, the project managers were often sucked into a spiral of planning and re-planning, as they ceased to manage their projects and the plans lost credibility.
Our close observations of the projects show us that the project management teams did a good job in the upfront planning process, but then were not able to manage the projects effectively from that point on. This included problems managing scope change, resolving issues, communicating proactively and managing project risks. One explanation for this setback lies in the fact that even though in many projects the roles and responsibilities of the project teams were clear, their decision authority was often limited due to the level of influence and decision power of the project members.
There was also a tendency in the projects to focus on deadlines and perform target-led planning approaches. Too much attention was given to such dates. This affected the performance of the project members. By being concerned only about a point that lies far into the future, the project members felt that there were plenty of time to do the work. Consequently, the project activities were delayed and took longer than anticipated.
Project Managers should realise, at an early stage already, if a project was underestimated, has too many built-in security features, when inadequate system provision was made by the vendor as a result of the underestimation of the project or deadlines cannot and will not be met. They should then advise the management and the sponsor of the project accordingly and also advise on a possible new strategy. By not doing so, the eventual problems are just postponed with tremendous costs and frustration.
For instance, when the steering committees or the top management were presented with the project schedule, they presumed that it must have been possible to do the work more quickly and in shorter periods. One motive behind such slashing was to please the project sponsor and to inform him that the project was on track. To make the plan attractive, the project managers then reduced the estimates of work content and duration, and then convinced themselves and the steering committees that the new estimates can be achieved. Tragically, it did not, and the projects generated plans that were the unbending evidence to prove the failure of such planning practices. This has had serious implications for some of the projects.
There was also tendency to plan the projects as if the outside world did not exist. The project schedule lacked any slack or a contingency timeframe. Many of the project activities were extended due to the unavailability of staff for reasons such as holidays, sick leaves, training courses and seminars and of course those skilled staff members that were appointed very late.
As it is the case in any project, project progress was dependent on certain decisions being made within the organisation. It was common not to give proper attention to the political factors underlying the decision coming from the top, and to underestimate the time required to study and implement such actions. The result was that insufficient time and resources were given for many tasks. Sufficient time was not allowed for some important activities, which later impacted negatively on the schedule. Critical tasks were done inadequately and were redone. All these identified factors affected the planned activities, and required another re-planning activity to happen. As indicated earlier, project managers were sucked in re-planning their plans repeatedly.