Three key concepts to keep in mind when testing solutions for your e-commerce business

Third-party software solutions have become an essential component of e-commerce strategy.  There are now options available to improve any and every aspect of your retail business, from driving traffic to your site to improving AOV.  Each solution claims to enhance your business in some way – but how do you know if the software is actually doing what it’s supposed to?

Let’s say you invest in a solution called “ConversionPro”, which claims it can increase conversions on every product in your store.  How do you know if the software is significantly affecting your conversion rate? And how can you be sure it’s positively impacting your bottom line overall?  Simply put: test, test, and test some more. In order to stay competitive in the constantly evolving e-commerce landscape, it is imperative that retail leaders understand how to test & measure third party software on their website.

Here are three important things to remember before running your next test – or even before reviewing the results of a recent one.

Source: Optimizley

A/B Testing is one of the most common methods of testing a solution’s value

Define precisely what you're measuring.

When trying to measure a solution’s value, it can be all too easy to get lost in a sea of metrics.  There are hundreds of ways to analyze how a customer interacts with your business. Additionally, differing definitions for a single metric are often used.  In order to test successfully, your team needs to be in complete agreement about what you are looking for and how exactly you plan to measure it. 

To test ConversionPro for example, it is imperative to have a precise definition of conversion rate.  You should know exactly where your base level is and how you measured it. Additionally, you should have a clear idea of the effect you are looking to see after implementing the solution, measuring the impact using the same precise definition.  For example, conversion rate can be defined as the total number of sales transactions divided by the total number of visitors to the website, measured the same way before and after implementing the solution.

Judging where a solution falls within the customer journey funnel can provide a cheat sheet of the most pertinent metrics to use when measuring the solution’s value.  Here are some of the most important ones to keep in mind.

Of course, this list is nowhere near exhaustive.  It does, however, give a basic view of what to look for, especially if you’re unsure of where to start. 

Understand how other factors could influence testing.

Many solutions can be difficult to measure accurately, especially those that engage with your website’s complex software ecosystem.  These on-site solutions, such as our ConversionPro example, typically interact with many variables, including website load time, design changes, and sales promotions.  These variables all have the potential to alter the user experience.  

User Generated Content solutions such as Olapic can provide value, but the numerous complications of the on-site software ecosystem often make them tricky to test accurately.

Source: Rebecca Minkoff

Because of this, it is important to maintain awareness of other factors surrounding a test.  An unmonitored variable, such as a website load-time increase, can ruin the results of a test if not accounted for.  For example, while testing ConversionPro, you might also launch a week-long sales promotion. This decrease in price will likely lead to more conversions regardless of the solution’s impact, potentially leading to testing error if not recognized.  Do not let these factors scare you away from testing on-site solutions, however. Testing a solution warrants proper attention and energy, just make sure to stay cognizant of variables that might affect it.

Implement a test with a clear plan & follow-up.

Whether the person running the test is you, an in-house tech lead, or a third-party testing service, communication is paramount throughout the process.  Before testing begins, all parties involved should be absolutely clear about how the test is being implemented, including what metrics are being measured and over what duration.

Behind every test is a person running it and people make mistakes.  However, people can also answer questions and be held accountable. Don’t treat testing as a set it and forget it situation.  Be sure to stay on top of any changes or problems facing the website. The more you communicate with whoever is operating the test, the more likely the test is to deliver the end goal: improving your website.