Multivariate Testing: A Rebuttal

In his October 9th blog post “Multivariate Testing – Thoughts from X Change” Gary Angel of Semphonic, argues that multivariate testing (MVT) is “useless” and “routinely encourages exactly the wrong sort of testing by making it extremely difficult to build a creative hypothesis.”

This is true only if you aren’t interested in huge uplifts in conversion and revenue, while saving money by avoiding subjective content decisions.

Multivariate testing is the first step to providing customer insights for marketers to build a creative foundation that will positively impact the marketing cycle. I’ve seen my own customers painfully pour thousands of dollars and hours into website re-designs—without testing first—only to watch their conversion rates plummet. But by using live visitor data, MVT lets creative teams in on which design and content directions are or aren’t going to yield the best conversion results—and swiftly avoid high cost and high risk situations.

However, I agree with Gary when he tells readers: “Don’t even think about doing multivariate testing till you’ve created a real testing plan…” I strongly encourage every client to adopt a testing culture within their organization to help them create a plan and roadmap designed with their brand and creative in mind.

But the strategy shouldn’t stop at multivariate testing. Today, most organizations operate a “vanilla” site—it’s not dynamic, segmented or targeted, it simply “is”. But, it is a great starting point to understand the customer, build conversion uplift and improve revenues through testing, segmentation and personalization techniques.

Segmentation targeting takes testing to the next level by enabling different varieties of content to be tested with in a certain visitor attribute groups, such as demographics, marketing source, geographical data, offline activities, etc. By displaying more relevant, engaging content for all visitors to a site, segmentation can generate even greater conversion uplift than just MVT alone.

Similarly, personalization solutions can also drive uplift in conversion rates, but again, are best utilized when coupled with a website optimized by MVT programming. Without testing and personalization in place, high volume web pages and conversion funnels are missing out on their maximum potential and revenues.

With so many top brands employing testing, segmentation and personalization to achieve website conversion improvements, produce a better online experience and stay ahead of the competition, I don’t see how any marketer could deem this technology as “useless.” After all, the data doesn’t lie.

SHARETweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Pin on PinterestDigg thisShare on RedditShare on TumblrShare on StumbleUponEmail this to someone

A career long evangelist in online, Mark Simpson is the founder of Maxymiser where he is focused on increasing conversions and Web site engagement for clients through all verticals including Asda, Santander, Virgin and Cancer Research. Prior to Maxymiser Mark headed up online marketing and business development for Travelport and focussed in particular on the acquisition of ebookers, Octopus Travel, Hotel Club and RatesToGo. Prior to Travelport Mark was part of the team taking Hitwise to market at on its start-up in the UK.