I’ve given up on estimating software development tasks well. Yes, you can do interval estimates, as How to Measure Anything recommends. Yes, you can track your estimates and improve them over time. But it’s slow and few project management applications support it. (OmniPlan is the only one that works on Mac and gives you Monte Carlo simulations based on your interval estimates. But getting information on how well your estimates matched reality is still hard.)
So I’ve settled on the 80⁄20 solution, Evidence Based Scheduling. It’s implemented in FogBugz, which forecasts milestone completion using a Monte Carlo simulation based on your past estimates and how long it actually took. Which means that you make quick and dirty estimates, and out comes a probability distribution over completion dates that automatically takes into account how good you are at estimating.
They changed their pricing recently, but it should still be free for up to two users. You might have to ask the sales team.
All that said, if you have actionable guidance on how to estimate, how to get milestone completion forecasts based on the estimates, then how to judge and improve your estimation accuracy – if you have that and all in a convenient way, I’d be happy to know and adopt it.
Thanks, I’ll try to write up that post in the next couple of weeks.
In my old software dev team we got very good at estimating the time it would take to complete a single work-package (item on the backlog) but those were at most a couple of days long. What we were not very good at is the estimation of longer term progress, in that case we were in a start up and I think that was unknowable due to the speed at which we would change plans based upon feedback.
Did you get very good at estimating, because you had tracked the time on similar pieces of work before? Ie. were you doing reference class forecasting? If yes, that’s a good reminder for me. I’m familiar with the concept, but it has slipped from my mind recently.
Also, how much effort would the estimating itself take? For example, how many seconds or minutes would you be thinking about a three-hour work item?
Yes we tracked time, but only in an aggregate way. Our list of work-tasks had a very rough estimate (XS, S, M, L, XL—each being about twice the size of the previous, and XL being just more than we could complete in a 2 week period). When we came to plan our 2 weeks of work we estimated hours using ‘planning poker’ (which is a bit like the delphi method—blind estimates by each member of the team, followed by a brief discussion of the reasons for the differences, followed by one more round of blind estimates, then I [as team lead] had the final decision). At the end of the 2 weeks we would talk about each item, this sometimes involved a discussion of the amount of work relative to the estimate (either the initial e.g ‘S’ or the hours e.g. 4). In our discussions for the tasks people would regularly refer back to previous tasks as reference. We would always talk about our productivity (i.e. the size of the tasks we completed, where XS=1, S=2, M=4 …) but this was a balancing act, it would be easy to mess up incentives here.
We spent 4h every 2 weeks planning tasks, but that might involve a small discussion/argument over what should be part of each task, not just the estimation. We would also spend 2h and the end of the 2 weeks reflecting on 1) what we had made and how it impacts the development roadmap 2) things to increase MPH (motivation, productivity, happiness—I was far too pleased with myself for coming up with that acronym :P ).
Individually, I thought a lot about how to increase development speed and accuracy in estimates. But that was at least a third of my role in the latter stages, the rest being split with planning the development roadmap and doing actual development.
For a 3h task, most of the time we would spend ~2min listing to one person describe what it is. Then <1min for everyone to show their card with their estimate of the number of hours. Often that was in enough agreement that we wouldn’t do anything extra. We did have a few where one person guessed 3h and another guessed 20h, that often resulted in a 10min discussion, as there was clearly a disagreement on how to do that task ‘properly’).
I’ve given up on estimating software development tasks well. Yes, you can do interval estimates, as How to Measure Anything recommends. Yes, you can track your estimates and improve them over time. But it’s slow and few project management applications support it. (OmniPlan is the only one that works on Mac and gives you Monte Carlo simulations based on your interval estimates. But getting information on how well your estimates matched reality is still hard.)
So I’ve settled on the 80⁄20 solution, Evidence Based Scheduling. It’s implemented in FogBugz, which forecasts milestone completion using a Monte Carlo simulation based on your past estimates and how long it actually took. Which means that you make quick and dirty estimates, and out comes a probability distribution over completion dates that automatically takes into account how good you are at estimating.
They changed their pricing recently, but it should still be free for up to two users. You might have to ask the sales team.
All that said, if you have actionable guidance on how to estimate, how to get milestone completion forecasts based on the estimates, then how to judge and improve your estimation accuracy – if you have that and all in a convenient way, I’d be happy to know and adopt it.
Thanks, I’ll try to write up that post in the next couple of weeks.
In my old software dev team we got very good at estimating the time it would take to complete a single work-package (item on the backlog) but those were at most a couple of days long. What we were not very good at is the estimation of longer term progress, in that case we were in a start up and I think that was unknowable due to the speed at which we would change plans based upon feedback.
Did you get very good at estimating, because you had tracked the time on similar pieces of work before? Ie. were you doing reference class forecasting? If yes, that’s a good reminder for me. I’m familiar with the concept, but it has slipped from my mind recently.
Also, how much effort would the estimating itself take? For example, how many seconds or minutes would you be thinking about a three-hour work item?
Yes we tracked time, but only in an aggregate way. Our list of work-tasks had a very rough estimate (XS, S, M, L, XL—each being about twice the size of the previous, and XL being just more than we could complete in a 2 week period). When we came to plan our 2 weeks of work we estimated hours using ‘planning poker’ (which is a bit like the delphi method—blind estimates by each member of the team, followed by a brief discussion of the reasons for the differences, followed by one more round of blind estimates, then I [as team lead] had the final decision). At the end of the 2 weeks we would talk about each item, this sometimes involved a discussion of the amount of work relative to the estimate (either the initial e.g ‘S’ or the hours e.g. 4). In our discussions for the tasks people would regularly refer back to previous tasks as reference. We would always talk about our productivity (i.e. the size of the tasks we completed, where XS=1, S=2, M=4 …) but this was a balancing act, it would be easy to mess up incentives here.
We spent 4h every 2 weeks planning tasks, but that might involve a small discussion/argument over what should be part of each task, not just the estimation. We would also spend 2h and the end of the 2 weeks reflecting on 1) what we had made and how it impacts the development roadmap 2) things to increase MPH (motivation, productivity, happiness—I was far too pleased with myself for coming up with that acronym :P ).
Individually, I thought a lot about how to increase development speed and accuracy in estimates. But that was at least a third of my role in the latter stages, the rest being split with planning the development roadmap and doing actual development.
For a 3h task, most of the time we would spend ~2min listing to one person describe what it is. Then <1min for everyone to show their card with their estimate of the number of hours. Often that was in enough agreement that we wouldn’t do anything extra. We did have a few where one person guessed 3h and another guessed 20h, that often resulted in a 10min discussion, as there was clearly a disagreement on how to do that task ‘properly’).
Thanks! I especially like how differences of understanding were exposed when estimates diverged.