A Minimalist’s Approach to Project Metrics
By Owen Head
What project metrics do you keep? A simple question, right? Perhaps, but there are so many ways to get wound around the axle in the age of information where every question has twenty equally compelling and comprehensive answers.
I personally prefer a minimalist when it comes to metrics management (or anything for that matter, methods, templates, …, anything). The issue for most PM’s (and other managers) when presented with the intellectual overkill found in so many advanced methodologies, models, templates, worksheets, etc., is in identifying what to keep and what to toss. Instead, many feel compelled to make use of every process step, template, and metric.
The trouble is, projects come in all shapes and sizes, and the one size fits all standards don’t usually fit all that well. Very few projects will require a lion’s share of the metrics included in any comprehensive off-the-shelf standard/methodology (PMBoK, PRINCE II, …). Couple that with the fact that it would take a small army to collect, evaluate, and react to them all, and it becomes obvious that PM’s in the real world have to prioritize their metrics collection and response efforts.
No one needs metrics – what they need are answers. As a result, it often makes the most sense to begin the process of identifying metrics, by first identifying the questions that will need ready answers as the project moves forward. In most cases, those questions will be answered in status reports using a combination of metrics and summary text, so it follows that metrics exist primarily in support of project status reporting.
As a result, a reverse engineering (RE) approach that works backwards from reporting requirements to metrics requirements is often the best way to go. The RE approach begins with an up-front design of key project status reports (periodic status, milestone, project close, …) that will be used to inform stakeholders and call team members to action. That report design effort will result in the identification of component metrics required to fill in the blanks.
To produce the best possible report, contents should not be limited based on perceptions regarding the availability of data, but should instead represent the best “imaginable” report given the intended audience. As a PM, a well designed report can also guide daily/weekly tracking and control activities, making those efforts more automatic. Once report design has passed review, the process for data gathering must be developed based on report format, distribution method, and timing.
When establishing the processes to be used in gathering metrics , it’s important to prioritize the list in order of importance, and avoid processing any that aren’t truly needed. No metric should be included, for example, unless the team will actually be able to take the time to react to it. If the team won’t be able to take action in response to an undesirable metric, then it would be a waste of time to gather it. It’s always possible to increase the number of metrics as the project moves forward, but attempting to take on too much at the beginning can steal time and attention from other critical project activities.
So what basic questions need to be answered? Let’s start by recognizing that everything in management can be cast in terms of short term impact (tactical) and long term impact (strategic). Understanding that most long term metrics are not critical to the execution of the current project, it then makes sense to shift the responsibility for those metrics away from the project team to functional teams like QA, design, or test. Of course, where the long term metrics target improvements in project performance, the project team may be required to maintain ownership.
Some Basic Project Questions
Project – Tactical (short term impact)
- Is the project on-time?
- Is the project on budget?
- Is the project delivering the required high quality?
- What issues/risks threaten scope, schedule, or quality?
- Are project control activities (change, risks, issues, actions, escalations, …) being dealt with in an effective and timely manner?
- Does the project still meet project selection criteria?
PM Office – Strategic (long term impact)
- How responsive is the project team (issues, actions, changes, …)?
- How effective is process (intervals & quality)?
- What project lessons have been learned?
- How accurate are requirements, estimates, plans?
Metrics, of course, are dependent upon a competent baseline. Effective projects get one chance at producing a competent plan, and once it’s been approved, it is the baseline. If the plan/baseline must change, then you really have a new project on your hands. Otherwise, there exists no basis for any of the questions above and metrics become fruitless – you can’t usefully measure an object undergoing transformation. The baseline provides the fixed values for time, scope, and costs (resources) from which all project metrics are measured. As a result, plan vs. forecast vs. actual values provide almost everything you need to measure the status surrounding scope, schedule, and cost. The project is on time if planned critical path dates are equal to their forecast dates, on budget if planned costs are equal to forecast costs, etc… Of course all measurements must be cumulative from the bottom of the WBS to the top to ensure accuracy.
In terms of metrics measuring performance of unplanned work, benchmarks can be used in place of baseline plans. Take action items management as an example. How can you tell that delegated action items are being actively worked and not being starved for attention? The best method involves breaking action management into a series of steps (decomposing it) that can each be benchmarked. New actions, for example, may originate in “Pending” status and require that the PM admin screen the action to ensure it is valid prior to moving it to “Open” status. In this case, it would be easy to establish a reasonable benchmark of 24 hours as a maximum time in “Pending” status. Then all actions could be measured in terms of the benchmark. It’s more difficult to benchmark the length of time an action should remain “Open” status since actions vary in terms of the work to be done. It will make more sense to reference a considered and agreed due date as the baseline for that time-in-status metric.
As stated earlier, the metrics themselves are only part of the story. It’s the expert interpretation of what each measurement means and the explanation of response activities that complete the picture. It’s the formation of the right questions to be asked and answered that set the context. Finally, it’s making the right priority calls surrounding those questions that ensures that you have the information you need to manage effectively and not so much information that it diverts resources from more critical PM tasks.
Owen Head has over nineteen years of technical Program and Project Management experience in ISO/TL 9000 compliant organizations. He has built a number of PM Office practices from ground up, and managed more than 70 technical, business process, and change management programs and projects, in all areas of IT and Telecom development and support. He holds a B.Sc. in Computer Science Engineering from the University of Texas at Arlington and has been certified as a Project Management Professional (PMP) by the Project Management Institute (PMI).
Owen is the Managing Director of PMOSoft, LLC, a company dedicated to PM Office performance maturity, through fast and affordable maturity management system (MMS) solutions. For more information, please visit http://www.pmosoft.com or our blog http://pmosoft.wordpress.com.