How to Choose a Project Portfolio Management Tool?
By Miley W. Merkhofer

Tools are good. Tools for project portfolio management can help organizations improve the selection and management of their project portfolios. This can and will allow organizations to increase value even while cutting costs. Tools promote a more deliberate, careful, and consistent evaluation of project alternatives. They force the generation of more and generally better project data. Many of the available tools provide excellent data management and reporting capabilities, making it much easier for managers and executives throughout the organization to understand the work that is being conducted.

All tools, however, are not equally good.

Which Tool Is Best?

The best tool is one that actually helps your organization make better project choices. Outputs that support the management of individual projects are nice, but the greatest opportunity is the ability to use analysis to identify the best project choices. For a tool to do this, it must make the right recommendations, and this requires that it be based on a decision model that is accurate, logically sound, and complete. Furthermore, the tool must be practical, effective, and acceptable, otherwise it won’t be used or its use won’t actually influence decisions. Although some tools are clearly better than others, no one approach is best (or even adequate) for all circumstances. The choice of a PPM tool needs to be made differently for different organizations and applications.

For a tool to be the best approach, it must compute project priorities based on the value that alternative project portfolios would provide. An acceptable tool must be based on sound project valuation theory, one that captures best understanding of how projects actually create value for the enterprise. Most tools aren’t based on sound project valuation theory and, therefore, fail to provide accurate estimates of project portfolio value. As a result, they do not offer much help for making tough project choices. The quality of the model used to quantify project and portfolio value, in my opinion, ought to be the critical discriminator for choosing a PPM tool.

Recommendations for Choosing a PPM Tool

More than 80 tools are currently being marketed for project portfolio management

• Do not assume that available products are similar: they differ in ways likely to be of critical importance to buyers. The more expensive products don’t always offer more capability, but inexpensive products are invariably low cost for good reason.
• Be skeptical of marketing claims.
• Clearly understand your business, functional, and technical needs before reviewing and choosing tools.
• Understand the underlying architecture of candidate tools. This includes not just software, hardware, and integration issues, but also knowing how the tool does what it does and how much flexibility really exists.
• Consider obtaining help for tool selection from knowledgeable and independent advisers. They can help you to avoid costly mistakes by suggesting evaluation questions and providing objective reviews of candidate tools.
• Learn a little about the current theories for valuing projects (especially multi-attribute utility analysis and real options). Understand how these theories apply to your organization. Use them to develop a model for how your projects create value. Determine whether the tool offers sufficient flexibility to enable it to represent your organization’s value model now, and in the future as your understanding and needs grow.
• Reject tools that fail to include appropriate algorithms for valuing projects and optimizing project portfolios. Ask to see an independent peer review, or arrange for your own technical review through a local university. Buying a tool that can’t optimize the project portfolio is like buying a hand-held calculator that can’t add. The tool should enable you to make better decisions than you could make on your own, not just provide you with pretty bubble charts.
• Don’t use tools that select projects based on balance, strategic alignment, goal maximization, or any other logic not directly tied to value maximization. Use tools aimed at maximizing the total (risk-adjusted) value of the project portfolio. CEO’s and CFO’s want to know how much value will be created by their project portfolio. They want to be assured that projects are chosen so as to maximize this value. Metrics based on goal achievement, portfolio balance, strategic alignment, or “points” are not surrogates for value, and won’t (or shouldn’t) be of primary interest
• Avoid tools that define project value solely in terms of the tradition financial metrics, such as net present value (NPV), return on investment (ROI), or payback period. Financial metrics are important, but they fail to capture the non-financial benefits of projects. Typically, such tools grossly undervalue certain types of projects as well as the project portfolio.
• Make sure the tool can capture all considerations critical to your decisions. Commonly ignored considerations include various soft project benefits, investment urgency (as opposed to investment value), project sequencing and other types of project interdependencies, and risk (especially market risks and other “correlated risks” that similarly impact multiple projects).
• When comparing tool costs, take a total cost of ownership (TCO) approach and cumulative costs over at least 3 years, including costs for software support and maintenance,software customization, tool implementation, and training.
• Assess tool risks, including tool adoption and tool productivity risks.
• If you purchase a tool from a vendor, make sure that you use all available flexibility to tailor it to suit your needs. It is particularly important that criteria and weights be defined such that the underlying model prioritizes projects based on the value added per unit of cost. If the vendor can’t provide the expertise, get help from an expert.
• If you engage consultants to help you build a tool, be sure that they are experts, not just experts in software development, but experts in the theory and algorithms for valuing projects and optimizing project portfolios. The field is highly specialized, so check references carefully.
• Plan on the need for education and training. Having a thorough understanding of concepts is as important as knowing how to use the tool.
• Unless your organization is very small, use a phased approach to implementation. Start with one department, conduct a post- implementation lessons-learned review, and make changes before you move to the next level.
• Make sure that all stakeholders have necessary buy-in and confidence in the tool. Otherwise, insufficient effort will be devoted to generating inputs and/or model recommendations won’t change decisions.
• If the answer seems wrong, then either your intuition is wrong or there is a flaw in the model. Check the model logic. A good model will have a compelling logic. If you still don’t agree, don’t trust the model (replace it). Remember, a tool is an aid to, not a substitute, for sound decision making.

Custom-Designed vs. Configurable Tools

A significant difference among tool providers is one of philosophy. As noted previously, the approach favored by most software vendors and many consultants involves creating a tool based on a “configurable” decision model — a model that is hard-coded within the software but includes parameters and options that can be set to help fit the tool to a range of different applications or situations. Frequently, marketing materials describe such tools as being “fully customizable,” but the truth is they can be adjusted only within a narrow range allowed by model parameters.

The alternative approach, favored by some consultants, is one that provides flexibility for creating custom decision models. These tools are created on modeling platforms — high-level programming languages designed to facilitate general purpose modeling and analysis. These platforms include Excel, Visual Basic, and DPL Portfolio, as well as web-based, project portfolio management tools that allow one of these or a similar modeling platform to be accessed via a web portal.

Each approach has its own advantages and disadvantages. Tools with configurable models are convenient. They can be implemented quickly. They do not require effort on your part to design a custom model—that work has already been done. They key question is whether the configurable model is adequate for the application. Potentially relevant questions include: Does the tool provide capability to handle all types of projects and project portfolios that might eventually need to be analyzed? Does it account for all types of project benefits, including dynamic, time-varying project impacts? Does it allow for rigorous multi-attribute utility valuation of projects? Does it allow for true portfolio optimization (not just ranking)? Does it allow for risk valuation as well as risk characterization?

Configurable models are typically programmed in software languages that are not friendly to changes. If the configurable model doesn’t provide some capability (either a capability currently desired or one that might be desired in the future), beware that it may be difficult or impossible to obtain this capability. The code may be so complicated that only the original programmers are capable of making changes. Changes that affect structure often produce ripple effects that require extensive rewriting of source code. For example, if a tool expects cost savings resulting from a project to be entered as an annual average value, changing the model to allow entering year-by-year estimates can be difficult.

Custom tools built on modeling platforms are much more flexible than tools based on pre-set, but configurable models. (Compare the ease of changing an Excel spreadsheet with the difficulty of getting a software vendor to make a model change to its tool.) Custom tools implemented on modeling platforms can more easily “grow” as the user organization gains experience and understanding. However, developing customized, quality tools for project portfolio management using Excel or another general-purpose modeling platform can be labor intensive. It requires the client organization to participate in the design process by making choices and requires a consultant skilled in modeling and portfolio analysis to implement the custom model on the modeling platform. The process is faster if the consultant has compiled a library of sub-models for use as building blocks, and if the tool automates some of the labor-intensive programming steps (such as creating user input templates for the custom model). However, tools with flexible modeling platforms can be large and more costly to develop, so they may be more expensive to acquire than configurable tools.

If a configurable tool fits the need and contains a defensible logic for valuing projects and optimizing project portfolios, then such a tool can probably be implemented more quickly and with less cost than a custom tool. However, these are big “if’s.” My experience to date has been that customers who want a tool that correctly identifies value-maximizing project portfolios (in situations where project value is more than discounted project cash flow) must either use a custom tool built on a general modeling platform or must link a vendor tool (i.e., tool providing the desired data management and reporting capability) to external models that correctly compute non-financial components of project value. However, tool capabilities are advancing rapidly, and new options will become available that reduce the costs and difficulty of obtaining a tool that meets all of the needs of the organization.

Miley W. (Lee) Merkhofer, Ph.D., is an author and practitioner in the field of decision analysis who specializes in assisting organizations in implementing project portfolio management. He has served on advisory panels for several government agencies and has received grants and research awards for work in the area. Lee is an editor of the journal Decision Analysis.

Prior to becoming an independent consultant, Lee was a Partner of PriceWaterhouseCoopers, where he founded that organization’s capital allocation and project prioritization business practice. Lee is a founding partner of Folio Technologies LLC, a provider of web-based, project portfolio management software.

Lee received his Ph.D. in engineering economic systems from Stanford University. He is the author of the book Decision Science and Social Risk Management and co-author of the book Risk Assessment Methods..

Additional papers on project portfolio management can be found on Lee’s website, www.prioritysystem.com. E-mail: lmerkhofer@prioritysystem.com.