Make better decisions by running small experiments.
(5 Min Read) Leverage small experiments to make better and inclusive decisions faster.
Do you ever think, "What is going to be the impact of the decision I am about to make?" for questions like:
Will this productivity tip work for me?
Should I share my views with others?
Are people going to adopt this new process?
Should I learn this new technology?
Will I like my team/colleagues in the new company?
Should we have this new meeting?
And, many more?
We all struggle with these (or similar) questions because they are hard, and the answer is not straightforward. Sometimes, this struggle is big enough to cause a mild case of decision paralysis, and as a result, the decision never gets made. We must realize
Making no decision is a decision in itself.
If we decide to make no decision, then it should be a conscious and deliberate choice instead of an unconscious one.
And, for all the decisions that we do make, some of them will have unfavorable outcomes. There could be multiple reasons why the outcome went south, but often it's because of wrong/invalid assumptions. We can't prevent all bad outcomes, but we can significantly reduce them by making decisions based on data.
Decisions need data
Data-driven decisions are a norm in Tech industries. Every company has (or should have) a data division with the sole mission to collect, process, and make data available such that the company can make better business decisions.
We can use the same data-driven technique to make better decisions at the personal, team, or project level. But, surprisingly, we don't.
Maybe we can't because the human brain cannot collect, process, and map the large amount of data required to make such decisions. Or perhaps we don't because the data collection exercise seems too daunting to even try at a personal level.
Even though the former is a valid reason but later is what happens in practice. And it doesn't have to be.
Collect data via experiments
The best way to collect data is to convert every decision into a small experiment. We should
In reality, we don't need a massive amount of data to make a personal, team or project-level decisions. All we need is signals on two things:
🤢 Bad outcomes: Things we don't want to happen.
😊 Good outcomes: Things we want to happen.
We could add more dimensions, but that will lead to an elaborate experiment. In my experience, the KISS (Keep it simple stupid) principle is the way to go when doing small daily experiments. (Yes, I said stupid, we are all grown-ups here.)
What to experiment with?
If you decide to stop reading beyond this section, that's fine. But do read this section twice. Yes, twice.
The most common mistake that we must avoid at all costs.
Don’t waste time/money/effort experimenting when the outcome is obvious.
I know that this advice is obvious. But so many experiments are performed just to confirm the expected outcome. If you know the outcome with high enough certainty, take the decision and move on. We only want to experiment when the outcome is not apparent.
This obvious thing that can
Prevent you from spending too much time in the decision valley, and
Significantly increase the decision velocity
(More on these below)
How to experiment?
As we plan to experiment a lot, the experimentation process must be simple, effective, and low effort. Here is a lightweight way to set up experiments:
Step 1 - Setup: Figure out the signal for a good and bad outcome.
Step 2 - Measure: Start with a small sample set.
Step 3 - Decide: After each iteration, look at the signal.
If a good outcome needs more data, repeat step 2 with a larger sample set. If it doesn't, stop the experiment and take the decision. If you see a bad signal, ABORT.
If you repeat step 2 several times, it should look like a spiral staircase. The higher you go, the bigger the sample set gets and the more data you collect.
Now, let's look at examples of how to experiment with the questions we highlighted in the first section.
Will this productivity tip work for me?
Experiment: Identify if the productivity tip, such as this one (experimentation), or increase productivity by stopping context switching, or getting more work time per week, will work for you.
Step 1:
Good outcome - productivity increases when you follow the tip.
Bad outcome - tip doesn't work.
Step 2: Try to follow the steps in the post for 1-2 weeks.
Step 3: Measure, evaluate and take the decision. Repeat step 2 if necessary.
Are people going to adopt this new process?
Experiment: Apply the process in small, low-cost settings.
Step 1:
Good outcome - people find the process simple and effective to use.
Bad outcome - people stop doing it after one iteration.
Step 2: Start with 1-3 people.
Step 3: Measure, evaluate and take the decision. Repeat step 2 if necessary.
Should we have this new meeting?
Experiment: Figure out meeting agenda and required people.
Step 1:
Good outcome - the meeting is useful, and everyone was engaged.
Bad outcome - The meeting is redundant.
Step 2: Start with 1-2 meetings.
Step 3: Measure, evaluate and take the decision. Repeat step 2 if necessary.
Will I like team/colleagues in the new company?
Experiment: Simulate a working/casual environment with future colleagues.
Step 1:
Good outcome - You like hanging out with your colleagues and discuss work and other things.
Bad outcome - Any red flag such as team is burned out or doesn't enjoy each other's company.
Step 2: Have a happy hour/lunch with future colleagues.
Step 3: Measure, evaluate and take the decision. Repeat step 2 if necessary.
The cost of experimentation is LOW when compared to bad outcomes.
For example, the cost of subscribing to this free newsletter is low. Experiment for 2-3 weeks and see if you find it useful. Continue if you do, unsubscribe if you don’t.
Rate of experiments vs. Decision velocity
I don't have exact metrics, but if I did, I suspect that the graph between the number of experiments vs. decisions made would look something like this.
Dark line - number of decisions made.
Bright-line - number of experiments conducted.
As you introduce experiments in daily life, the number of decisions made (dark line) drops (gaps) initially as the number of experiments conducted (bright-line) increases.
That's expected because some decisions have to be delayed until the experiment is complete. But over time, experimentation should not reduce the number of decisions made. Rather the number of decisions should stay roughly the same with improved outcomes.
I like to call the gap, drop in the number of decisions due to experimentation, decision valley.
Crossing the Decision Valley
Crossing the decision valley shouldn’t take long. And once you cross it, the views are amazing. Every decision is:
Supported by data. 🙌
Have significantly reduced probability of bad outcomes. 💰
If you continue to experience reduced decision velocity after the initial accommodation period, i.e., you are spending too much time in the decision valley, don't abandon experimentation. Take a step back and check for the following things:
Are you experimenting with obvious decisions? (a most common mistake)
Are you experimenting beyond the inflection point i.e. an increase in the sample set does not result in better data? (due to lack of experience with experimentation)
These are usually the most common pitfalls people encounter.
Crossing the decision valley for the first time may seem challenging. That’s because we don’t know how many experiments we can run simultaneously and how long we should do them. But this is a one-time cost. Once you cross the decision valley for the first few decisions, you find your rhythm of experimentation. Then it’s just smooth sailing ahead.
Communicate your experiments
Do you want to be part of an experiment without knowing that you in one?
When the experiment involves other people, communicate and involve them. Like when trying out a new team meeting format.
Involving others provide valuable data and insights. Use this as a signal to make better and inclusive decisions. Not asking will be an antipattern.
If you are still skeptical, consider this post as an experiment in itself. Try it out for 1-2 weeks and check if the decisions you experimented with had better outcomes with reduced anxiety and increased confidence.
Please share your experiments in the comments. I would love to learn about them and the outcomes.
Also, don’t forget to share this post with a colleague who is about to make a tough decision without data. It may help them make better decisions.
Adios!!!