Select Page


International Rescue – Recovering a Failing Project
By Allen Ruddock

As an experienced project manager you often get dropped in to a failing project to turn it around. How should you go about this. I’d like to describe a situation I found myself in earlier in my career and draw some lessons from the experience.

Setting the scene

The project in question was to implement a new back office system to settle trades in fixed income securities – corporate and government bonds (or loan instruments). A package solution had been chosen and the company was working with the supplier to enhance the package to meet their needs (there’s the first hint of problems!).

At the same time as implementing the system for corporate bonds, it was also decided to enhance the system for a new trading mechanism for UK government bonds, otherwise known as gilts (so called gilt edged securities because they were backed by the UK government). This was called CGO2. In addition, the new head of the function had decided to take the company into the Japanese Government Bonds (JGBs) market and the new system was being used for this.

So we had three separate projects all on the same system and the original project manager had left. I was asked to act as caretaker for the project whilst a permanent replacement was found. Up to this point, only minor European currencies had been migrated onto the system and the weekend before I took over it was the turn of UK corporate bonds. The cutover weekend failed. Not because of the system, but because they ran out of time to clean up the data on the legacy systems. Lessons were learned and we’d try again in a few weeks – or so we thought.

The problems emerged

The failed UK migration happened just before the end of September. So the next event was quarter end. This was the first quarter end for the new system, and more importantly, the first time a full reconciliation had been performed. This threw up several major problems.

  • The system was not accruing interest on bonds properly.
  • The accounting entries generated were wrong.

  • The traders P&L statements and trading ladders (reports that show details of trades including trading dates, interest payments etc.) were wrong.

All hell broke loose!

The actions

First and foremost we had to keep the show on the road. Trades still had to be settled and we had to start tracking the accounting manually to keep the books straight. We also had a fixed deadline for the CGO2 project so that could not be compromised. Next we had to get to the root causes of the problems so I set up small targeted teams to address each area.

Some of the interest accrual issues had been identified and fixed in the standard package so we tried applying the software supplier’s fixes. But because we had tailored the package, the code didn’t work. Our code bases had diverged. So we got the supplier in to work side by side with our coders to apply the same principles to our own code base. That addressed the accrual engine problems.

The accounting entries had to be re-worked from first principles. This was a long and arduous task with long nights and weekends of work. The lady that lead that work showed a lot of dedication and sacrifice. To ensure she knew that was both recognized and appreciated, at the end of the task I told her to take her husband out to dinner at their favorite restaurant. She hadn’t been the only one to suffer and the recognition of her husband’s sacrifice meant the world to her.

The trader’s ladders and P&L were another issue. Here I brought in specialist external help to assist my own team to crack the problem. So action plans and teams were in place, it was now just a question of time.

It never rains, but oh how it pours

In the securities business nothing ever stands still. So in the midst of all these issues a new trading team were hired and they wanted their favorite trading system implemented and integrated with the settlements system. Yet another strain on a hard pressed team. The answer was to again turn to specialist help from outside the company. This time we hired a niche consultancy that specialism in systems integration. I appointed one of my project managers as the link between the two teams and a plan was established.

Six months down the line, and we were ready to restart migrations. The first release of code for the CGO2 project went live and the accounting and P&L issues were fixed. It was time to hand over to the permanent replacement.

Key lessons

  • If you buy a software package wherever possible change the way you run the business to fit the package, not the other way round. If it is an established product it will have many clients using it so why is your business so different that you need to change it?
  • When you encounter problems, identify the root cause and go back to first principles if necessary. Do this first rather than treat the systems. It will be quicker and cheaper in the long term.

  • Draw on external expertise to help. It won’t be cheap, but again, it will be cheaper in the long run than trying to do it all yourself.

  • I used dedicated teams to fix each problem. Whilst this worked well, I had a small number of key resources that could have been better deployed solving problems across several of the teams. Have focus, but don’t be too rigid.

  • Communication is key – across the team, with the external suppliers engaged to assist and most importantly with the impacted business areas. We had some tense conference calls throughout the recovery but being able to demonstrate a risk based plan to address the issues and clearly communicating it, and progress against it, bought us the time we need to succeed.

  • Celebrate successes and recognize exceptional effort. As with the accounting team lead, positive strokes go a long way to keeping the team motivated in times of stress.

Allen Ruddock is currently a Delivery Assurance Manager at Royal Bank of Scotland and is the Director at ARRA Management Ltd.