In an ever-changing work environment, there is increasing demand for innovation and new ways of working—whether it’s pressure from your customers who expect the best and the latest products, or simply a desire to increase collaboration within your immediate team. Having a system in place to share and experiment with new ideas can help your organization progress.
The concept of a “test-and-learn factory” may fit the bill: an environment where you can share ideas, experiment, and evaluate findings to reflect and adjust or move on to try something new. The test-and-learn factory is a framework you can use to incubate creative ideas, document experimentation, and uncover new best practices that are ready to be shared across the enterprise. Frequently, those practices, when applied to new use cases, lead to improvement insights and even more ideas for experimentation.
It’s important to note and appreciate that innovation comes in all shapes and sizes. Not all innovations will revolutionize an industry. It’s just as important to explore smaller ideas that solve a specific problem. These little victories often serve as fertile ground for the curious mind, leading to a host of additional ideas.
The primary role of the test-and-learn factory is to generate a pipeline of creative ideas to test and accelerate your organization’s rate of innovation. So, let’s get into the details of this framework. It’s comprised of five basic stages (see our infographic below):
- Analyze
- Ideate
- Prioritize
- Test
- Reflect and adjust
Stage 1: Analyze
Opportunities for improvement will present themselves in different ways to different groups. The key is to be aware that these openings exist and to look for ways to document them so you have data to evaluate. Sometimes opportunities for improvement are glaring, like a significant drop in sales; other times, they’re harder to spot. Either way, look for data sources related to the problem so you can perform an analysis.
Generally, it’s a good idea to follow these steps in analyzing an opportunity:
- Ask questions.
- Collect data.
- Clean the data.
- Apply analytics.
- Interpret the results.
Your goal is to have a clear understanding of the situation in a measurable way. If the team agrees the opportunity is worth pursuing, you can start kicking around ideas.
Stage 2: Ideate
There are many good ways to gather ideas, but here are a few methods to consider:
- Use a virtual idea box: create a space, accessible to all stakeholders, where people can submit ideas and catalog them to apply against the opportunities.
- Conduct a brainstorming session: Give participants an overview of the opportunity and time to think of ways to take on the challenge. Then, meet as a team to present, challenge, and expand on those ideas. Catalog the ideas.
- Conduct benchmarking research: Explore industry research, trends, and thought leadership in the marketplace. Identify promising ideas and don’t be afraid to dig deeper.
- Explore lessons learned: Review previous efforts to solve for a similar challenge. These can be initiatives that took place inside your organization or external case studies. Note what others tried and how it went. Create a list of ideas to try.
No matter how you generate and capture ideas, it’s important to keep your audience or end user in mind to provide direction for approaching the opportunity. At the end of this stage, you should have a catalog of ideas for possible experimentation.
Stage 3: Prioritize
Prioritize your ideas in any way that works for the team, but one recommendation is to use the ICE model and rate the ideas based on three factors—each scored as high, medium, or low.
- Impact: How much of an impact will this idea have on your team or business?
- Confidence: How confident are you that this idea will be successful?
- Ease: How easy will it be to implement this idea?
Note: Scoring of the above should be a team activity and will depend on several factors within your organization, such as the size of your team, the resources available, sponsorship support, competing priorities, the corporate culture, dependencies, etc.
Consider the ideas with the best ICE scores for testing. Those that rate highest will likely bring you the largest return on your investment. Review all scores and identify the top experiment to try first.
Stage 4: Test
Begin by setting up the experiment. This includes determining who needs to be part of the test. Larger initiatives may necessitate a cross-functional team, while you may be able to run smaller experiments by yourself.
Documenting the specifics of the testing is crucial for the benefit of others within your organization who may need to repeat or adapt the experiment to a new use case. It’s also helpful to establish parameters at the outset so you know when the goal is met or when the test is complete. Be sure you’re capturing all relevant data and comments, but at the very least, you should include:
- Name and description (what you’re testing)
- Hypothesis (what you expect to happen)
- Duration (how long you expect the experiment to run)
- Metrics (a rubric for success—specific, measurable, relevant)
- Notes (how you can replicate and understand the experiment results, areas of focus for the next iteration)
Give thought to the optimal amount of documentation rigor. Developing a standardized experiment template is a common best practice. Consider the following benefits when evaluating the level of rigor your organization needs. Good documentation allows you to:
- Debrief the experiment to determine whether your hypothesis was accurate.
- Rapidly scale (“lift and shift”) your learnings to other parts of the enterprise.
- Create a record in support of continuous improvement.
Based on our experience, here are a few tips and tricks to keep in mind during the testing stage:
- Assign someone as an experiment lead.
- If you have the resources, it’s okay to conduct multiple experiments at a time.
- Document best practices quickly and build on proven results.
- Compare experiments against one another to help prioritize the sequence of experiments that others may try.
Stage 5: Reflect and Adjust
Conduct a retrospective (a team discussion—often used in Agile practices—that creates common understanding of what went well, what didn’t, and what could be improved) to decide what actions you want to take going forward. Was the experiment successful? Did it meet your success metrics? Are there parts of the experiment that you want to continue implementing? What challenges did the team face? Is there anything that didn’t go well that you could change to improve the process or develop a new way of working?
Maybe you want to keep running the experiment with a few adjustments. Or maybe it’s time to move to the next prioritized idea on your list. It’s okay to say the experiment was a failure if it was. The important thing is to understand your lessons learned.
Reflect as a team on how the test-and-learn factory process is working for you. Take some time to pause at appropriate intervals (for example, after your first experiment or after a predefined amount of time) to evaluate the process and make any necessary changes to optimize the process for your team.
Test-and-Learn Factory in Action
“The best way to have a great idea is to have more ideas.”
—Linus Pauling
The test-and-learn factory approach offers a structured way to surface and capture great ideas sooner and more consistently. It can empower everyone in your organization to seek opportunities for improvement and innovation. The test-and-learn factory harnesses a teams’ innovative thinking by providing a designated safe space to try new things, no matter how big or small the innovation may be. It also:
- Helps retain and motivate talent with an openness to try new things and to drive toward positive changes
- Promotes risk-taking in a structured, safe way where experimentation is “just the way we work”
- Provides cover for creative thinking rather than doing things “the way we’ve always done them”
- Integrates easily with Agile methodology, where approved experiments can become part of a sprint backlog prioritization exercise and related meeting ceremonies
- Facilitates propagation of best practices across the enterprise through the documentation process
The test-and-learn factory can be used as a way to address challenges and new ways of working, or as an intake model when you’re tasked with providing solutions to a customer or internal clients. It’s also a great framework for companies who want to stay ahead of the competition by generating and sustaining an innovation pipeline. It can be applied in a variety of settings to address any number of challenges you may face.
Do you spot an opportunity for improvement in your team or organization? Start brainstorming ideas with your team and put the test-and-learn factory into action! What new idea do you want to test out first?