Conviction is Good, Optionality is Better
Or how to not dread ambiguous problems
You’re an engineer 8 months into a project and all you’ve done is write docs and emails. There are still a dozen different design ideas that have potential, so you're hesitant to build anything yet. You feel stuck, frustrated that you can't make progress. You want to move forward, build something, ship something.
Thinking in options might help.
Desire paths
Imagine an urban planner itching to slap pavement on the ground after months of swimming through email hell. Her team can't agree on the right path to take, but she knows the answer and takes the initiative to hire a construction company. Every day the road goes a little further and she becomes more convinced that it’s moving in the right direction. At some point, the road is long enough that there’s really no choice but to finish it. 3 months later she has a beautiful paved road to stand on proudly. Then a couple days later she sees this:
It's easy to get stuck when we think that our choices are "I'm building something" and "I'm stuck building nothing". That's definitely the case if we weigh our value by how much pavement we put in a straight line. But what if it’s not just the final road that is valuable? What if we also value the work that improves our options along the way? We’re not really “stuck”, we can:
Expand our options - get approval to build bike lanes just in case we discover that bikes are the best solution.
Enhance all options - invest in a road-building machine that improves road-building time by 50%.
Prune our options - plant some grass.
Number 3 is well illustrated by a classic example about college footpaths. Campus planners plant grass to learn what the desired commuting paths are.
We are builders, and want to stand proudly on what we have built and declare "I made this". Along the way, we should also proudly find opportunities to improve the project’s optionality.
Optionality
A lot of ideas develop naturally as an accumulation of experience. These kinds of ideas are evidence that you've earned valuable perspectives through growth.
Every now and then, you read something that slams you head on like a truck, pressurizing years of unlinked experiences into a single, cohesive diamond.
“Tidy First?” by Kent Beck did this to me with its explanation of the value of optionality. Briefly — builders often believe that their value comes from what they make. An urban planner stands on a road and says "I made this" and is fulfilled by the value she provided to society.
What Kent proposes is this: often the value we provide is not what we build, but what we enable to be built in the future. If you've ever felt stuck in an ambiguous problem space, you may be familiar with the itch to do something useful — to finally ship something to users instead of writing investigations, spinning up proof of concepts, and ferrying messages between teams. What Kent is saying is that your value is not just what you build at the end, it's also all of the doors you open along the way.
What does building this way look like?
In my case, I'm about a year into navigating an ambiguous multi-year project that will probably deliver something next year. During this last year I've been faced with many decisions, but there's one in particular that I'm quite proud of. Last year, we knew we wouldn't get serious design direction for 6-12 months given the size of the project and its ambiguous nature. So two primary options stood out to us:
Option 1: Wait for designs - Focus on advocating for getting designs sooner, and use a waterfall-ish development cycle to build from scratch after we get designs.
In this option, only the end result is seen as having engineering value. It asks: why start building now if we’re not sure what will be shipped to users?
Option 2: Build to enable any design direction - Invest in upgrading our existing infrastructure to unblock building any of the features we think we may need, regardless of design specifics. As the designs get clearer, we build more specifics and apply more constraints.
In this option, the work along the way is seen as having value, even if it doesn’t actively contribute to the final pixels sent to the user. Optionality-increasing work could include broad infrastructure upgrades, prototyping, and identification of constraints. Constraint-applying work could include pixel-perfect UI and inter-team integrations.
After much deliberation, we chose option 2. Truthfully, it’ll take at least 6 months until we find out if we made the right choice, but at the very least, there have been plenty of times in the last year where I've been really glad we didn't pick option 1. Option 2 is not a silver bullet — it requires more net engineering time. This made sense for our project because we knew designs needed time to float around the ambiguous product space.
Unstructured ideas
The essay is over, so you're welcome to leave if you want! There are a couple unconnected ideas that I've found quite interesting from the lens of optionality. A better writer would have found a way to connect them into a cohesive essay, but here we are.
Optionality reduces the need for alignment — I plan to write more about alignment, but something interesting that I've discovered is that when you reduce your options, you need to make sure all of your cross functional partners agree that the reduction is safe. When you expand or enhance your options, you only need to align on the resources spent, not the outcome itself.
Infrastructure — all infrastructure investments are investments in enabling future functionality. The best kinds of infrastructure investments apply constraints in select places to enable optionality in the right areas. If you are building infrastructure, your job is to consistently provide productive optionality. That could be enabling scale, enabling new features, freeing up engineering resources, etc.
Multiple value of software — Software moves fast and has zero marginal cost. That means the things that can be accomplished with software are massive — i.e. the optionality of software is massive. It's interesting to see a software engineering concept supported by the financial market — the value of a software company as a multiple of revenue is vastly higher than any other kind of company specifically because software represents the potential to do almost anything for almost no cost.
Go read Tidy First? — I did my best to reinterpret what Kent wrote, but there is absolutely no substitute for his explanation. He ties value of software to money and uses the options market to illustrate how engineers can value optionality. A quote:
Thinking of software design in terms of optionality turned my head backwards. When I focused on balancing creating options & changing behavior, what used to scare me now excited me.
This example is a great accompaniment to Beck’s work. Thanks for sharing!