Project Portfolio Management – Predicting Project Success

Project Portfolio Management – Predicting Project Success
By George F. Huhn

Original Title: Project Portfolio Management – Predicting Project Success the Way a Meteorologist Predicts the Rain

Estimating the probabilities of success for your projects is necessary for calculating the expected value of a project and is an essential part of project portfolio management (PPM). Unfortunately, most project managers and project management offices (PMO) don’t do this very well. They could learn to do it better by looking at how meteorologists predict the probability of rain.

So, just what does it mean when a meteorologist says “the chance of rain today is 60%?”

Each day in the United States, a massive amount of data is collected from weather stations, satellites, and weather balloons from around the world and sent to the National Meteorological Center near Washington, D.C. The data is processed to give a multi-dimensional picture of global atmospheric conditions, and then it is analyzed using various algorithms to develop local weather forecasts and predictions.

But this isn’t how they make the “percent chance of precipitation” predictions. Even with the massive amount of data and super computer speed, their predictive algorithms alone just aren’t good enough. So they use comparisons to historical data.

Basically, they take the current atmospheric conditions and compare them with days in the past that had very similar conditions. So when they say that “the chance of rain today is 60%,” it means that it rained on 60% of the days in the comparison set.

And guess what? Assuming the data was entered properly, these predictions are 100% reliable all the time. Why? Because they are only predictions of probability – they aren’t “wrong” on a particular day, whether it rains or not. But whether they are accurate or not in the long term is an entirely different question.

The only way to determine if the predictions are accurate is to collect the data and plot the actual versus the predicted conditions over time to learn the margin of error. If it only rained on 30% of the days that the prediction was 60%, then there is a problem with the data or the data processing.

You can do the same type of probability prediction and testing with your business projects, too. The more accurate your estimates, the more confidence you will have in your overall project-value ranking in your project portfolios.

Developing more accurate project risk estimates requires 4 basic activities:

1. Identifying the key drivers of cost, time, and resource risks in completing project tasks.

2. Preparing a database of these tasks that includes the corresponding cost, time, and resource estimates assigned to each project and the basis for those estimates at the beginning of the project.

3. Tracking the actual costs, times, and resources used performing the task as each task is completed.

4. Comparing the actual costs, times, and resources with the starting estimates.

After you have maintained this database for your project portfolio for a period of time, you will be able to plot the actual versus the predicted results. This plot will show you the accuracy of your cost, time, and resource estimates as well as revealing the distribution of the actual results. (You will probably learn that your cost estimates were too low, your time estimates were too short, and your resource estimates were for too few. And that is a good thing to learn.) Eventually, you will be able to use the actual results data as a basis for future probability predictions because patterns will emerge. The data will also give you an understanding the uncertainty in those estimates.

I saw the data of one major pharmaceutical company who did this for their project “percent probability of success” estimates over a number of years. The data between 20 and 85% was surprisingly linear; for example, about 50% of the projects that had “percent probability of success estimates” of 50% were ultimately successful. It also showed that all projects that had an estimated “percent probability of success” of 85% or greater succeeded and all that had an estimate of 20% or less failed.

If you’re involved in project portfolio management and you’re looking for ways to improve your project planning, compiling and analyzing your historical data is a great way to test and improve your future estimates.

George F. Huhn, President of Data Machines, Inc, founded the company in 2000. Data Machines offers business applications and consulting to help businesses improve their performance through superior project portfolio valuation and optimization. George has authored or co-authored numerous papers and articles in publications ranging from The Journal of Organic Chemistry to Newsweek, and has delivered seminars and keynote addresses at events across the country. He also holds several U.S. patents, and has been written about in Chemical and Engineering News.

He holds an Executive Masters of Science degree in the Management of Technology from the Wharton School and the University of Pennsylvania. He is also a Moore Fellow in Technology Management at the University of Pennsylvania’s School of Engineering and Applied Science, and holds a B.S. degree in chemistry from Drexel University.

DataMachines.com offers a project portfolio management software tool called Optsee® for calculating project and project portfolio value even with uncertain data. By automatically analyzing your project portfolio in thousands of scenarios using easy-to-run Monte Carlo simulations and then optimizing against multiple constraints such as limited funding and resources, Optsee® quickly shows you your best, worst, and most-likely returns from an optimal portfolio.

Data Machines also offer a spreadsheet workbook for easily calculating the return on investment (ROI) for any project portfolio management tool.