I had a company reach out with an interesting problem on Monday.
They had migrated from a Legacy platform to a new Fabric headless e-Commerce solution a few months ago.
Conversion rates on the new site dropped about 40% with (from what I understand) no material changes to design, copy, layout, or flow of the website. It was pretty much a copy of their Legacy site.
THANKFULLY they had kept the Legacy site and started running a split test between the two variations (a super rare practice btw so kudos to them for not fully migrating before doing this)
But after several months the company hadn't been able to figure out what's caused the drop in CR. After our discovery call, I offered a free analysis (out of goodwill and hopes they become a client) to identify the problem.
In a few hours my team and I were able to identify a potential issue with Chrome.
No need for testing. Just good data.
But thank god it was available.
Here are some important lessons to keep in mind:
1) Always keep a backup of your old site when doing a full-site redesign or migration.
2) Always split test your new design/platform for at least 30 days to see if there are any material differences in your key performance metrics (site speed, bounce rate, conversions, etc..). This will allow you to compare data between the two sites and identify where friction is happening.
3) Never redesign an entire website from scratch and just launch. From my experience, 9/10 full-website redesigns perform WORSE than the original. This is primarily because what designers coin as "good UX" is based on their opinion, some assumptions, and best practices that don't necessarily apply to you and your business. Test your designs iteratively. Everything from branding, to layout, to copy can dramatically impact conversions. This process takes much longer, but the outcome is more aligned to what (I hope) you're looking for… a better performing website sitewide.
4) Always dig in the data. Your topline metrics are AVERAGES. They are made up of 100/1000s of user segments. Dig into what's working, what's not, and for who. And then use those insights to progressively optimize and iterate your site.
5) Always be testing. Because there is always room for improvement.