How Microsoft Taught Me to Test Things and Try Things to Learn Better

“I have not failed. I’ve just found 10,000 ways that won’t work.” — Thomas Edison

When I started in Developer Support at Microsoft, I learned so many amazing things.

But the most important thing I learned was to test things and try things.

By embracing the philosophy of testing and trying, I learned the power of experimentation for deeper, faster, and more impactful learning.

I was surrounded by super smart people that seemed to always know the answer to everything.

But there were two types of smart people:

1. Those that “knew” the answer because they knew it intellectually.
2. Those that “knew” the answer because they tried and tested things.

The difference wasn’t obvious at first. After all, “everybody” seemed to know “answers” all the time.

But what I found over time was that those who tested and tried things, learned faster, further, and deeper.

And they outpaced everyone over time.

Did You Create a Repro?

In Microsoft Developer Support, we helped each other out.

But when somebody was stuck on a tough issue, the first thing someone would ask was:

“Did you create a repro?”

In other words, did you create a small experiment to “reproduce” the problem.

The goal in creating the repro was to boil the problem down to the smallest possible example.

Once you could repro the problem, you gained a lot of insight.

You learned how things work.

You learned what happens when certain vairables change.

You could invite other smart people to help you analyze what the problem might be.

The repro turned an abstract issue into a concrete challenge you could learn from.

Creating Small Experiments Created Unlimited Learning

Creating these small experiments, or repros, taught everyone how to learn faster.

Speed matters when you’re dealing with complexity and volume.

And the ones who did the best, were really fast at creating experiments and repros.

I made it a goal to get super fast at creating repros, so I studied everyone I could.

Learning from everyone was a mindblowing experience.

Everyone had special strengths that helped them create fast repros.

I wanted to learn all the strengths and apply my own.

I learned my strength was synthesizing from everyone.

I could distill and share best practices so everyone could do better.

And I liked changing the game.

It’s the Doing that Builds Your Empathy

Empathy comes from the doing. I found that those that tested also had deep customer empathy.

Those that didn’t, thought they had empathy.

And the engineers that had empathy could understand the customer challenges in a much deeper way.

In fact, it was this empathy that often leaded to leadership.

The people that were promoted had more empathy, so they could manage, shape, and lead the business better.

They were invited to be the Product Liaisons because they could represent the Voice-of-the-Customer.

They could build better readiness guides because they had deep empathy for the customer pains, needs, and desired outcomes.

And they had deeper empathy with colleagues because they could relate to the struggles of learning and adopting new technology.

Intellectual, Emotional, Physical

Eventually I learned that you learn deeper when you learn something at 3 levels.

The first level is intellectual.

You can learn the facts, the figures, the concepts. You can regurgitate them.

The next level is when you know something emotionally.

The concepts mean something to you. You relate to them in a deeper way.

You have emotions related to the knowledge.

The last level is physical. It’s burned into your basal ganglia and muscle memory.

It’s why you can do some things without even trying or without even looking.

Your body knows the knowledge.

So, by testing and trying things, it helped me learn deeper.

My body learned how to use certain tools very fast.

And by constantly breaking things down into smaller problems, it made it easier to tackle the tough stuff.

Nobody Can Learn for You

As one of my mentors puts it, “Nobody can learn for you.”

I realized that when somebody gives you an answer or advice or a piece of information or even deep knowledge, it’s not the same as learning it.

Knowing how important empathy and practice are to really know something, I made it a habit to keep testing and trying things throughout my journey at Microsoft.

It’s now always obvious what breakthroughs or hops or jumps it will lead to.

But it’s obvious that it’s a very different level of knowledge when you try to apply what you learn.

It’s this nuance that is important.

Testing Use Case, User Stories and Scenarios

As I grew up in Microsoft, I found that language varied a lot around how people talked about what users were trying to accomplish.

Some people would always say use cases.  Other people would use user stories.

Most people would just say scenarios:

“What’s the scenario?”

But in all cases it was conceptually a variation of “who is trying to do what to achieve what goal or purpose?”

Technically speaking the relationship between use cases and scenarios comes down to this:

In the software industry, there’s three common usages of scenario:

1. The same as a use case.
2. A path through a use case.
3. An instance of a use case.

Usually, the most helpful one is “an instance of a use case.”

Why? Because if a scenario is an instance of a use case, then it’s testable with concrete data.

And I found that using a simple format for user stories worked wonders:

As a <role>
I want <goal>
so that <benefit>

For example
As a Team Leader, I want to implement an AI-driven analytics tool, so that I can gain deeper insights into team performance and identify areas for improvement more effectively. 

List of Business Experiments

I found that leaders at the highest levels, test and experiment in different ways.

Some use deep questions that cut to the core of capabilities.

Others simply focus on experiments they can try with customers.

One leader I know is fairly famous for always asking for lists of experiments he can try with customers.

He uses the simple list of experiments at CEO and C-suite level to get the company excited with possibilities and to quickly turn ideas into action and results, or at least deep learning.

For example, here’s a list of AI experiments that a leader can conduct to test the value and impact of AI for their organization:

1. Chatbot for Customer Service: Implement an AI chatbot on the company’s website or customer service platform to handle routine queries. Measure its impact on response time and customer satisfaction.
2. AI-Enhanced Recruitment Process: Use AI tools to screen resumes and shortlist candidates. Evaluate the efficiency in the recruitment process and the quality of candidates shortlisted.
3. Sales Forecasting Model: Develop an AI model to predict future sales based on historical data. Compare its accuracy and effectiveness against traditional forecasting methods.
4. Email Sorting and Prioritization: Implement an AI system to automatically sort and prioritize incoming emails for employees. Assess time saved and employee productivity.
5. Automated Social Media Analysis: Use AI to analyze social media trends and customer sentiments about your brand. Evaluate the insights for marketing strategy adjustments.
6. Predictive Maintenance in Operations: Integrate AI into machinery and equipment to predict maintenance needs. Measure the reduction in downtime and maintenance costs.
7. Personalized Marketing Campaigns: Utilize AI to analyze customer data and create personalized marketing content. Monitor engagement rates and conversion improvements.
8. AI for Inventory Management: Implement an AI system for predicting inventory needs and optimizing stock levels. Assess the impact on inventory costs and availability.
9. Employee Onboarding Process: Use an AI-driven onboarding system for new employees. Evaluate the speed and effectiveness of the onboarding process.
10. Fraud Detection System: Deploy AI to monitor transactions and detect fraudulent activities. Measure the efficiency in identifying fraud and the reduction in losses.
11. AI-Assisted Project Management: Utilize AI tools for project planning and resource allocation. Monitor improvements in project delivery times and resource utilization.
12. Automated Document Analysis: Implement AI for analyzing and summarizing large documents or contracts. Assess the time saved and accuracy of the analysis.
13. AI-Driven Customer Insights: Use AI to gather and analyze customer feedback from various channels. Evaluate how these insights improve product development or customer service.
14. Employee Performance Analysis: Deploy AI to analyze employee performance data. Use insights for better talent management and development plans.
15. Energy Consumption Optimization: Use AI to analyze and optimize energy usage in company facilities. Measure the reduction in energy costs and environmental impact.

Those are high level and pretty big tests.   Usually what happens from there is a smart Solution Architect with a Business Architect will translate those bigger tests, into smaller tests to prove and evaluate the path.

But the key in all cases is to learn your way forward by testing and doing and getting faster feedback.

How I Test Myself and Keep Learning

If I catch myself “thinking” my way through too much, but not testing enough, I stop.

I ask myself, “How can I practice this?”

And I break the high value information down into some atomic tests I can practice.

I just want to figure out very small tests that help me learn the information deeper and relate to it better.

It’s how I turn insight into action.

In fact, my process actually starts a little bit earlier than this.

As I read information, I cut through it faster using one simple questions:

“How can I use this?”

So, when I go from “How can I use this?” to “How can I practice this?” I’m basically whittling down using information into the smallest, or most atomic actions.

If it involves changing thought habits, then I try to figure out a simple one-liner reminder or mantra.

Or, sometimes I turn it into a question that helps me practice it or think a different or more dynamic way.

How To Shift from a Know-It-All to a Learn-It-All

A big limit in life can be what you know, or what you think you know.

As Mark Twain put it:

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.”

There’s brilliant TED talks on this idea, and even books all on the same idea:

“Don’t believe everything you think.”

One of the most surprising and insightful books is The Half-Life of Facts.  It reminds us that “facts” aren’t always facts, and that often they are based on what we “know” at the time, and people reinforce.

A saying a few Microsoft colleagues use is: “Untested assumptions lead to false facts.”

The way to shift from a know-it-all to learn-it-all is to change your language.

I used to say, “Customers want X” or “Customers want Y.”

As my mentored taught me, I simply need to change that to:

“I have a hypothesis customers want X” or “I have an assumption customers want Y.”

And that little shift changes everything.

Suddenly, it’s OK to experiment and it’s OK to be wrong.

The problem with stating things like facts or making strong assertions is it makes it shuts people out or locks you into a position of needing to be right.

I made this shift several years ago and when I changed my language, it changed how I thought about things, too.

But more importantly, it got me questioning assumptions in everything.

I’ve learned more and more how easy it is to fall into the pitfall of false facts and bad assumptions.

Industries have been built on false assumptions, too, which is why Blue Ocean strategy is a great way to disrupt.

But by focusing on assumptions, it also helps you “unlearn” things and stay open to what would otherwise be completely conflicting concepts or ideas.

It’s a way to be more inclusive and diverse.

I find it incredibly helpful when somebody tells me something like a fact, to simply wonder, “What assumptions is that based on?”  Or, “What assumptions would need to be true, for that to be true?”

Talk about learning on fire.

The Problem and the Pitfall of Learning with AI

Most people won’t notice this at first.  Learning with AI is easy.

It’s easy to ask questions and get answers.

And the answers will often look good.  Sometimes, too good.

The answer from AI is not the problem.

The problem is when you don’t learn what’s behind it, or when you don’t understand or test the assumptions.

Or when you don’t actually practice or learn the knowledge at a deeper level.

The irony is AI can actually help you do that.

But you have to get over the human hump of taking the easy path, which is just take the answer and run.

It’s a slippery slope.

Absolutely, speed up what you “know” using AI.

But also speed up how you practice it and test it, so that you can learn at deeper levels where it counts.

And you can use AI to help you with that, too.

Sometimes it’s as simple as asking ChatGPT, “How can I practice that?”

What I’ve learned so far is that while ChatGPT will get me in the right direction, I still have to dig deeper to create more profound ways to practice.

A simple mantra I use is:

“Learn by teaching, know by practicing, and evolve by challenging your assumptions.”

And keep reminding yourself, don’t believe everything you think.

Experiment in Work and Life

My journey at Microsoft taught me the invaluable lesson of embracing a ‘learn-it-all’ mindset over a ‘know-it-all’ attitude.

The real magic lies not in just knowing things intellectually but in putting knowledge into action through experimentation and repros.

This approach not only accelerates learning but also cultivates deep empathy and a profound understanding of customer needs and challenges.

It’s this hands-on experience and relentless curiosity that truly transforms knowledge into wisdom, driving innovation and leadership.

As we find our way forward through the ever-evolving landscapes of technology and business, remember that the most powerful tool at our disposal is our ability to question, test, and learn from every experience.

So, the next time you face a challenge, ask yourself not just what you know, but how you can test and apply that knowledge to learn and grow.

Embrace the journey of continuous learning.

It’s in the doing, experimenting, and empathizing that we find the path to true mastery and transformative leadership.

You Might Also Like

Agile Innovation Framework
10 Favorite Frameworks I Used at Microsoft
How To Balance Your Vision with Customer Needs
How To Innovate Better with Simple Stories of the Future
How User Stories Help Leaders Learn the Cloud
What I Learned About Strategic Content at Microsoft
Why Master Pains, Needs, and Desired Outcomes