8 Steps to Run Data-Driven Experiments
“We should try this… How about that… What if we…?” How many conversations in the organization start with some variation of one of those phrases? Someone comes up with an idea for marketing or sales or a new product line, the boss likes it, and then it is off to the races. Suddenly, an individual, a team, maybe even an entire department is running headlong toward the “success” of some new goal or initiative - all before stopping to see if it is actually a good idea.
The truth is, most decisions are gut-based. We all have existing knowledge and experience which influences how we see the world and how we tackle problems. This can be problematic because every situation is different. Just because a Facebook ad campaign had a great ROI for a certain brand of baby diapers, doesn’t mean Facebook ads will be as effective for sports apparel. Maybe Instagram or influencer marketing would be better, or perhaps Adwords, or even, sponsored sporting events – which is why we start with small tests.
The problem, when it comes to solving problems, is that there are always a million possibilities. And while brainstorming new and innovative ideas is usually a valuable exercise, deciding which initiatives to undertake comes down to vetting the ideas with data. Otherwise, how do you know if things are actually working? You may just be spinning your wheels.
They say, “You cannot manage what you cannot measure.”
And when it comes to the success or failure of the organization and of individual decisions, nothing could be more true. Good ole’ dumb luck only lasts so long before systems and processes become necessary to ensure success. That is where data-driven innovation comes in.
Running Experiments to Make Data-Driven Decisions
Step 1. Brainstorm potential solutions or hypotheses
The first step in any decision or new initiative is laying all ideas on the table in as non-judgmental and unbiased a way as possible. Anything less automatically restricts creativity and innovation, and thus, the potential upside.
Step 2. Consider how to test each potential hypothesis
Data-driven decision-making is all about running simple, low-budget, small-scale tests to determine the best course of action. Agile means minimizing the cost and time commitment of each idea before knowing (thanks to verifiable test results), that specific strategies or initiatives will work.
For a more personal example: Would you try out an extreme diet if you hadn’t seen others have amazing results? Of course not. And, as with any diet (focused on weight loss, strength gains, body composition), your new strategy should have clear, measurable goals like # of new email subscribers or website visitors, # of meetings booked, or new revenue generated.
Step 3. Set up data collection procedures
Before testing possible ideas, it is critically important to have data processes in place to collect and store relevant information. There is nothing worse than investing the time, energy, and resources on a promising experiment or test only to find out that you didn’t record the results correctly. That’s like heading for the North Pole with a compass that doesn’t point north.
Step 4. Establish a baseline and test one thing at a time
Another important question to ask is: how does a company know if a new drug, an ad campaign, or a sales strategy actually work if you don’t have something to compare the results to? As such, it is critically important to know your numbers and the metrics you seek to improve, be that new leads per month, daily sales, or hours of customer service calls per month to be able to evaluate the test results.
It is equally important to keep things as consistent as possible and not test too many variables at once. If, for instance, a company tried a new pricing model while also altering the sales script, it becomes impossible to know which changes led to which results.
Step 5. Rank the most promising solutions to test
Because it often isn’t possible to run multiple non-conflicting tests simultaneously, it is usually best to stack order the most promising ideas and test them in series. This is all about leveraging the individual and/or team’s experience in evaluating and prioritizing the best ideas – just be prepared to be proven wrong.
Step 6. Systematically test the best ideas
Data-based decisions are all about testing to discover which strategies actually achieve the best results, whatever the objectives may be. Here it is critical to think through not only what needs to be tested, but also the best method to evaluate the results because understanding the big picture leads to better experiment design, and thus, more reliable, applicable results to scale into the real world.
Step 7. Analyze the outcome
Once all tests have been run, it is time to analyze the results and choose the winner(s). This comes down to the primary goals of the initiative. For an example, let’s go back to our baby diaper example:
If for instance total sales and profits were your main objective and the Reddit ads you ran ended up having the lowest CAC (cost of acquisition) and thus, highest profit margins, but were much less scalable than those on Facebook, Google, and Youtube, maybe you’d prioritize Facebook, Google, and Youtube.
Or, if you found that the ads across most platforms were profitable, it might be worth setting up multiple campaigns, depending on the time commitment and opportunity cost. It all comes down to which KPIs matter most for your business. Which is why you need to know what you are optimizing for before testing your hypotheses.
Step 8. Iterating on the process and scaling up the tests
The last step after making a decision or rolling out a new initiative is continued testing and iteration because your first ad campaign or pricepoint won’t be your last. The true measure of success of companies like Amazon, Facebook, and Google is not only their commitment to data and data-driven decisions, but also their relentless optimization.
Alec Baldwin once famously said, “Always be closing!”
Well, when it comes to data, especially as it relates to websites and online commerce, a better line may in fact be: “Always be testing!”
A culture of split-testing and data-driven decision making is what defines the most successful companies of our time, because there is no crystal ball. There is no easy way to just make the right decision.
Even Steve Jobs wanted to restrict the iPhone to Apple-only apps in the beginning. And if Jobs himself can be so wrong, it pays for us mere mortals to leverage data whenever possible for our most critical decisions.
If you are interested in building a data-first company culture built on the fundamentals of testing, iteration, and data-driven insights and belief that great ideas can come from anywhere, consider giving Veezoo a try.
Sign up today for our 14-day free trial to empower your employees to make independent, real-time, data-driven decisions to further the goals of your business.
It’s simple: connect your database, ask questions, and start discovering actionable, company-altering insights in seconds – no coding or technical skills required.
Not sure if Veezoo is for you? Set up a call with a member of our customer success team to learn exactly how Veezoo’s industry-leading, ultra-intuitive Q&A chat system can be tailored to your organization’s specific data and business intelligence needs.
Answers to Your Most Pressing Questions
Empower your employees to make the real-time, data-driven decisions needed to move your organization forward.
Sign up, connect your data, and start asking questions in seconds – literally!
