I've been working in the eCommerce analytics and business intelligence space for almost 10 years. And there's one lesson that has held true throughout the years…
Analyzing data is not enough.
It leaves too much room for error.
All too often, analysts identify correlations that are wrong, or worse, decision-makers draw wild conclusions from what they see in insights reports.
The truth is, the lens we use to analyze data and draw conclusions is clouded by our experience, beliefs, assumptions, bias, knowledge etc…
What you know, or want to believe, may not be reality.
The only way to uncover the truth is to run an experiment and let the results confirm or deny any conclusions you/your analyst/your partner/your dog drew from the data.
Remove ego from the decision.
You may believe strongly that you're right. And you may be. But your patience and effort will pay dividends in the long run. Run the experiment and let the results make that decision for you.
Not only will you have more confidence in your decision, but you will also gain insights that you would have otherwise missed by analyzing the data alone.
Here's a great cartoon by Marketoonist that I really feel hits home the message: https://images.app.goo.gl/rPL2NfxndEVFS3y27