By using several measurements around the project pyramid, you can measure project completion. Project completion is a function of how accurate your original estimate was and how much progress you’ve made. But measuring only the schedule progress is not good enough. The only accurate way to measure progress for a software project is to measure how many features the project team has completed, how good those features are, and how many features are left to implement.
I once assessed a project in an organization where the developers had met every single date in the project schedules, but the testers were consistently late. Seems suspicious, doesn’t it? The developers hadn’t actually met any milestones at all—they checked in stubs and fixed the code when the testers reported defects. But because the project managers only looked at the dates—and never measured anything but the dates—the developers could say they had met the deadlines—without actually meeting them.
The only accurate way to measure progress for a software project is to measure how many features the project team has completed, how good those features are, and how many features are left to implement. As shown in the velocity chart in Figure 2, the number of features grew over the course of the project. The project team started with fifty features but released sixty-five features. If the team hadn’t tracked their progress including the number of features, they would not have been able to explain to their management why things took “so long.”
If I’m not implementing by feature, I like to use progress toward release criteria as a project completion measurement. Table 1 is an example of how I use release criteria to track project completion—or the lack thereof:
|Performance of Scenario 1 under 10 seconds||5/1, build 57: Performance < 30 seconds||5/8, build 70: Performance < 15 seconds||5/15, build 78: Performance < 12 seconds||5/22, build 85: Performance < 8 seconds|
|Number of defects found decreasing for at least 4 weeks||5/1, build 57: 22 defects found, same as last week||5/8, build 70: 15 defects found||5/15, build 78: 5 defects found||5/22, build 85: 2 defects found|
Release criteria are a late-in-the-project measurement. Even if you start assessing release criteria progress at the beginning of a project, most often, the release criteria data are available close to the end of the project.
No matter what lifecycle model you’ve selected for your project, to determine how good your initial estimate was, you can use Estimation Quality Factor (EQF), originally described by Tom DeMarco in Controlling Software Projects. At periodic intervals during the project, the project team answers the question “When do you think we’ll be done?” Each data point is the consensus agreement on when the project team believes the project will be finished. At the end of the project, draw a line backward from the release date to the beginning of the project. The area between the line you drew and the when-will-we-be-finished line is how far off your estimation was. This is a great technique for people to use as feedback on their individual estimates. But even if you don’t use it for feedback, it’s a great technique for the project manager to see what’s going on.
Figure 3 is a chart of an EQF for a project that was originally supposed to be nine months long. For the first couple of months, when the project manager asked when people thought they’d finish, they said “September 1.” And for a couple of months, they were optimistic, thinking that they might finish early. But during the fifth month, team members realized they didn’t know enough about some of the requirements. What they discovered changed the architecture and pushed out the date. For the next few months, they still weren’t sure of the date. They realized in the last three months of the project that, because of the changing architecture, they were encountering many defects they hadn’t anticipated. So, evaluating EQF, a qualitative metric, was helpful to the project manager and the project team as a check against the progress charts.
Schedule estimates are just guesses, so anything you can do to show and then explain why your schedule varies from the initial plan will be helpful to anyone who wants to know “where are we?”
This original article can be found at: http://www.jrothman.com/Papers/are-we-there-yet.html
Johanna Rothman consults, speaks, and writes on managing high-technology product development. Johanna is the author of Manage It!’Your Guide to Modern Pragmatic Project Management’. She is the coauthor of the pragmatic Behind Closed Doors, Secrets of Great Management, and author of the highly acclaimed Hiring the Best Knowledge Workers, Techies & Nerds: The Secrets and Science of Hiring Technical People. And, Johanna is a host and session leader at the Amplifying Your Effectiveness (AYE) conference (http://www.ayeconference.com). You can see Johanna’s other writings at http://www.jrothman.com.