The Doom in Estimates (with alternative)

A grim bureaucratic-political fate awaits many individual contributors in organizations. Here is the form of it. Management sits you down and asks,

"So, what's your estimate for this task?"

If you're a callow youth, you give your offhand take. If you're not, you start wriggling like a worm caught in a bird's mouth. Perhaps you give a suitably considered and viciously well-padded prognosis. Perhaps you give a gaze blank and pitiless as the sun. Or perhaps you just shrug.

Either way, that's not the doom that comes for you.

"7 weeks."

"So... 7 weeks. Why couldn't it be 5?"

Doom!

Doom that comes for you, no matter if the estimation is story points or t-shirt sizes or work-breakdown or analogy estimates. Doom that comes for you, no matter how little or how much actual data you have. Doom for both the manager and the individual contributor. Doom that comes for you because of a fundamental mismatch in worldviews between the manager and the IC.

The managerial job is one of negotiation. Managers are often trained both in negotiating and in having a negotiating worldview whether by some MBA or by the school of hard knocks. It seems natural for the manager to negotiate budgets, whether monetary or temporal. Monetary budgets are often surprisingly arbitrary, so throwing their weight around a bit can lead to better results for them and theirs.

That's not an invalid or illegitimate point of view. If you're sitting with a manager and a budget in an organization, you're in some kind of organization that wields some legitimate power. Whether a corporation or a government or nonprofit or what-have-you, that sort of negotiating is just the meat of the job of managing.

However, the individual contributor often has budgets both monetary and temporal forced upon them. Negotiation is possible, especially for the senior, but still a risky business. And the inexperienced are often not cognizant of the fact that the situation is a negotiable one. Doom is inherent in the nature of all negotiations where one party does not know they are negotiating.

And the time budgets themselves are chancy. Monetary budgets can be scrimped and saved for. Cutting scope and taking shortcuts is how one saves and scrimps for a temporal budget. But any seasoned expert remembers the dread times when they were forced to cut time budget but not scope, because that is itself a point to haggle over.

There is another unpleasant surprise, no surprise to those who've lived it. Temporal budgets have significant internal structure. If a widget costs ten dollars in money, buying two widgets costs twenty. To buy 10⁵ widgets costs less per widget, but that cost curve is known and can also be quoted by the widget manufacturer. Not so for time budgets.

The U-corp handler has to be extended so that it can handle L-corp interfaces. That task is not a widget to buy. It is a compounded quantity. Does the new L-corp solution conflict with the previous M-corp solution? What about the W-ray emissions? We have a deadlock over whether we should go with the M-corp Standard or the U-corp Standard for handlers that some of the senior IC's have been fighting about for two years which will affect the handling. And we have to deal with the Noodle Phenomenon that the regulators are talking about.

Doom!

The intermediate structure of things, to make things worse, is always new: your screws are always the same, the wished-for product may be Yet Another Electric Drill or Yet Another Customer Relationship Management App, but the screw-linkage-to-the-armature, the U-corp-handler-extension - those are always new.

Doom!

Such temporal costs do not add, but multiply. We know this because of the many orders of temporal magnitude that projects span, from minutes to centuries.

Doom!

This essay is one instance of a genre. There are a thousand essays decrying estimation in software and a relatively equal amount decrying estimation in other fields. The reason why the senior IC wriggles like a worm caught in the trap when asked for an estimate is because of that intermediate structure. That structure slouches towards the IC's dreams to be born as innovation and uncertainty.

Unlike basically every other instance of that genre, though, we do have something to suggest to you that is not some hortation to organize yourself differently or to spend more time thinking about what-have-you.

Mertonon is a new kind of budgeting tool. Mertonon is designed for resources with politics, internal structure and intermediate structure, which we claim can substantively replace the underlying organizational purpose of estimates. Instead of estimating, using the estimates to determine costs and then allocating resources based upon the costs, Mertonon gives you some suggestions on where to allocate resources directly. It is intended for organizational budgeting first and foremost.

Vobbs: So, how's the role-based access control coming along?

Tobbs: Wait, what's this?

Vobbs: Mertonon. The rectangles are groupings of the intermediate tasks. What's the news from D-corp, Robbs?

Robbs: We got news back from Dobbs at D-corp... he only likes the UX for permissions, not the rest of it, so I put the weights in to reflect that fact.

Tobbs: Huh? What are these weight thingies?

Robbs: High weight from X to Y means that Y depends a lot on X. Low weight means that Y depends a little bit on X. We put in original values, and Mertonon suggests some weight adjustments to conform to outside info, in this case the feedback from Dobbs. Or other possible adjustment criteria that the Mertonon devs haven't added yet.

Vobbs: OK. What's that feedback from Dobbs look like for the Mertonon adjustment?

Robbs: Don't forget to kick off the gradient calculation for the Dobbs feedback.

Vobbs: Oh yeah, can do.

Vobbs: Right, the Mertonon adjustment for Dobbs feedback.

Robbs: Mertonon is saying to reallocate effort toward UX for the 400's and not the role displays.

Tobbs: Wait, where's that number from? I don't get how this thing works, what's the inputs and outputs?

Robbs: The inputs and outputs are politically determined. In this case, the inputs correspond to each of the workers and are just 1-0, where they're 1 if present or 0 if not, and the output also just correspond to whether there's been feedback from client corps or not.

Vobbs: But that whole input-output structure is just because we agreed upon it when we started using Mertonon. There's a whole menagerie of possible representations you could do.

Robbs: Neural net backpropagation is basically the chain rule and caching. The thing you cache is called the delta, and it's the change in the overall goal when there's change in the activation value of one node in the neural network. This is usually used to figure out weight changes then thrown away, but Mertonon uses it to suggest changes to those nodes, too.

Tobbs: How do I know it's not garbage in garbage out, that sort of thing? Sounds political.

Robbs: Well, it's a political budget. They're putting down their own view on the politics of the org. If people say things that aren't true, they're doing so to us. And there's no underlying stick we're whacking them with like with estimates, so there's no disincentive to tell the truth.

Vobbs: Sounds good. Let's get Zobbs to go work this 400's task a bit. Mertonon suggestion says Pobbs but she's busy this week.

Robbs: Says here Hobbs is more related to the 400's.

Vobbs: Really? Hobbs it is, then.

Robbs:"Done. I'll go tell Hobbs.

Vobbs: Anyhow, next week?

Tobbs: Cool. Sounds good to me.

Robbs: Sounds good to me.

If you're piqued by this little incomplete vignette and want some more details and information about Mertonon, contact us directly at howon@mertonon.com or browse the other docs here. Here are docs for setting up Mertonon and here are docs for using Mertonon.

Q&A

What about stakeholders?

When they ask for an estimate, show them your Mertonon instance instead. This is the advantage Mertonon has over iron-hard willing yourself to not have deadlines: you can show people something.

We already don't do estimates, though.

Some organizations substantively led by individual contributors, like id Software under Carmack, manage to not give estimates by sheer dint of leadership will and fury.

Mertonon is for people and organizations who may or may not have such grim ideological hardening. If you already don't do estimates, Mertonon can be something to show your political situation to stakeholders inside and outside the organization.

What about truly hard deadlines, like planetary conjunctions or something?

This approach is not suitable for absolute deadlines imposed by physical reality or by the government or something, but such deadlines are not actually ordinary in most ordinary work. Most supposedly-hard deadlines are basically a negotiation tactic, to increase urgency.

But Agile solves this problem in software in some way (that I will now explain).

Did it? It's been 22 years. Agile is old enough to drink, even in America. And yet individual contributors still get asked for estimates daily, and those estimates still get ground down by negotiation.

Agile as a manifesto-driven movement is so vague that it cannot fail, it can only be failed. Therefore, you can't actually expect results out of it - it's the culmination of, a result of, good corporate politics, not a path towards undoing bad corporate politics. Woe betide the doomed fool who adopts Scaled Agile Framework, a framework from which I have seen exactly no decent software come out of!

How's this different from just Kanban?

You can stick people and relative weightings of factors in Mertonon in a way that doesn't make sense in Kanban. Mertonon also tells you which things to do and how to reweight relative weightings, given a goal and numerical journal entries with respect to the goal.

We'll put in a way to do a weighted-global-semi-topological sorting of Mertonon nodes (which would induce an overall global prioritization list with respect to the goal and be compatible with plain export to Kanban boards) when Mertonon tells us to do it.

If you actually want to see our Mertonon instance, contact us, because we haven't made the open public view options and security measures yet.

No, what I'm really here for is to have something to blame my failures on. Can Mertonon do this?

We fully intend that Mertonon's suggestions also work for the very specific quotidian purpose of looking good and blaming someone else for your failures. A guide for that and countermeasures and counter-countermeasures and counter-counter-countermeasures is forthcoming whenever Mertonon allocates effort to it. And of course, you can blame Mertonon for your failures directly.

Again, if you want some more details and information about Mertonon, contact us directly at howon@mertonon.com or browse the other docs here. Here are docs for setting up Mertonon and here are docs for using Mertonon.

Thanks to FH along with the other 219 peeps, and JB, for reading and comments.