Estimating tasks in hours is local optimization. Stop doing it!

This weekend I was listening to Eliyahu M. Goldratt's "Beyond the Goal" where he expands on the Theory of Constraints. In chapter three I listened to him explain how estimating in hours is in fact local optimization and therefore "idiotic". Here is a portion of that chapter loosely transcribed:

"The way to ensure that the project will finish on time is to ensure that every task will complete on time. This is the local optima policy that leads to a huge conflict. Placing a person on such a project jeopardizes the most important thing for them - their own image.

Suppose you are this person and you are asked how much time will this task take? You are extremely reluctant to give any number. And when you give that number the Project Manager will try to squeeze it. Why are you reluctant? Because you know that the number you give is an estimate but that it will be converted to a commitment - because the project needs to finish on time.

So what situation does this place you in? This common procedure of turning estimates into commitments puts you into a huge dilemna. Why? The objective of every person is to be regarded as a reliable person by your boss, by your peers, by people. But what is the meaning of being reliable? For example, if you give a commitment and you don't meet it - once is ok. But if you give a commitment and many times don't meet them do you expect to be regarded as reliable? Of course not. So one of the conditions of being a reliable person is to meet your commitments.

Also - if you are fighting for something (an estimate) and then it turns out that you have exaggerated by a mile? If it happens once, fine. If it happens frequently, then you will get the name of someone who exaggerates wildly - so kiss goodbye the image as someone who is reliable. In other words, another consideration for being a reliable person is to not be considered as someone who exaggerates. Now look what is happening when I come and ask you how long it will take to finish this task?

Now look - whatever you do, you are doomed. This is a huge personal conflict. How do we in reality deal with this conflict? Let's face it - this conflict is so important we must find a way to break it. We want to not only be reliable, but to be considered reliable. How do we do it? We cheat.

So we finish our estimates on the nose and sometimes a little more. They become self fulfilling prophecies. If there is a danger of finishing too quickly we add some more tests, do a more thorough job, we add another function in, we check things a little more thoroughly, and we finish on the nose. Do you see what's happening?

Due to the result that our local optimization turns any estimate into a commitment, we have developed the habit of giving estimates which includes in it the safety. If murphy (i.e. Murphy's Law) doesn't hit, we waste the time rather than giving it to the next link in the chain or project. If we do give it to the next link then we are declared as exaggerating and next time they won't give us the extra time. So look what a horrible situation this is. And this... is what kills the project. This is totally idiotic"

When you read this story the first time you will likely notice there are several examples in there of how not to run a project, of poor leadership, and his concluding statement is a little harsh. As many commenters have pointed out, this style of project management is not similar to how we typically run agile projects. However, at the core of the story is the desire of each person to be reliable by meeting hourly estimates whether they are created by the team, by yourself, or by someone else. When I first listened to this argument I was fascinated by that part of it. So, if that part of his argument is correct, then what alternatives do we have if we need to estimate? (This paragraph has been updated based on the comments - thank you)

If your project requires an estimate:

• Stop estimating tasks or user stories in hours - hopefully the argument above has already given you enough reason to at least consider this 'radical' change.
• Don't estimate user stories in hours or convert relative points to hours. OK - I do understand that it can be valuable to look historically at hours vs. points, but don't use this conversion for guessing how long a task or user story will take.
• There is some evidence that due to variability, counting the number of stories completed in an iteration is of equal value to counting the number of points completed in an iteration (velocity). This would help keep the focus off of hours per user story or task.
• Where possible, avoid asking how many hours are left for a task or user story. I understand why and how this information can be valuable, but understand the risk you take when you ask this question. 
• If possible, estimate at higher levels (epics) rather than lower levels (user story).
• Give broad project estimates with ranges like 1-2 months, 3-6 months, 1-2 years instead of doing detailed estimates. Most projects don't need a detailed estimate in order to determine if they should be completed or not. For many projects on our list we already have a good enough guess about their size to understand their value compared to the cost. (Unfortunately, as a consultant that usually doesn't include the projects I work on).
• This logic also suggests that relative estimating might be dangerous. I still have to think about that one as there are significant benefits to exercises like planning poker even if estimates are not created as output.

If your project does not require an estimate:

• Do a little song and dance!  

In either case, continue to break down your project into smaller projects and small deliverable pieces of value and working code. Build the high priority pieces first in an iterative  fashion. Make a decision about whether the project is done after each completed story or each iteration to avoid the problem described above - the temptation to finish on your estimated completion date. Continue measuring your progress using completed stories but consider using cycle time (average time to completion for all stories) vs. velocity (average number of points completed in a given time frame) to avoid some of the local optimization that could occur for each user story.

For further reading on the Theory of Constraints, check out any of Goldratt's books. I highly recommend starting with The Goal which is also available on iTunes Canada as an audiobook for $2.95.

Subscribe to this blog by Email