Estimating and Forecasting Biases in Projects – Part I (#2 in the series Choosing the Wrong Portfolio of Projects)
By Miley W. Merkhofer

People are notoriously poor at estimating and forecasting. They interpret statistical correlation as implying cause-and-effect. They tend to naively extrapolate trends that they perceive in charts. They draw inferences from samples that are too small or unrepresentative. They routinely overestimate their abilities and underestimate the effort required to complete difficult tasks. Estimating and forecasting biases are a special class of biases important to project-selection decision making.

Misestimating Likelihoods

Uncertain situations are particularly troublesome. Studies show that people make systematic errors when estimating how likely uncertain events are. As shown in the figure below, likely outcomes are typically estimated to be less probable than they really are. And, outcomes that are quite unlikely are typically estimated to be more probable than they are. Furthermore, people often behave as if extremely unlikely, but still possible, outcomes have no chance whatsoever of occurring.

Overconfidence is another powerful bias. We believe we are more accurate at making estimates than we are. I’ve often repeated a well-known demonstration to illustrate what I call the “2/50 rule.” People are asked to provide confidence intervals within which they are “98% sure” that various uncertain quantities lie. The quantities for the questions are selected from an Almanac, for example, “What’s the elevation of the highest mountain in Texas?” “Give me low and high values within which you are 98% sure that the actual value falls.” When the true value is checked, up to 50% of the time it falls outside of the specified confidence intervals. If people were not overconfident, values outside their 98% confidence ranges would occur only 2% of the time.

Overconfidence is also demonstrated by the many examples of people expressing confidence about things that are subsequently proved wrong. For example, British Mathematician Lord Kelvin said, “Heavier-than-air flying machines are impossible.” Thomas Watson, Chairman of IBM, reportedly said, “I think there is a world market for about five computers.”¹ Similarly, surveys show that most drivers report that they are better than average, and the most companies believe their brands to be of “above-average” value.

¹PM Hut: There is little evidence supporting this (although Watson is well known for making this statement).

Miley W. (Lee) Merkhofer, Ph.D., is an author and practitioner in the field of decision analysis who specializes in assisting organizations in implementing project portfolio management. He has served on advisory panels for several government agencies and has received grants and research awards for work in the area. Lee is an editor of the journal Decision Analysis.

Prior to becoming an independent consultant, Lee was a Partner of PriceWaterhouseCoopers, where he founded that organization’s capital allocation and project prioritization business practice. Lee is a founding partner of Folio Technologies LLC, a provider of web-based, project portfolio management software.

Lee received his Ph.D. in engineering economic systems from Stanford University. He is the author of the book Decision Science and Social Risk Management and co-author of the book Risk Assessment Methods..

Additional papers on project portfolio management can be found on Lee’s website, www.prioritysystem.com. E-mail: lmerkhofer@prioritysystem.com.