The Data's in the Details: Sophie and Sparse Data Sets

In a world where there are seemingly infinite numbers of companies that advertise, it may seem strange that there was no company focusing on the small business problem before the founding of Lexity. Why might this be? After all, automation software already exists for advertising and extensive research has been done into how to optimize campaigns for large budgets.

As it turns out, what works for large budgets doesn't necessarily work for a small business. What throws a wrench in the mix is not that smaller companies tend to sell different kinds of products, might have more restrictions on services such as shipping, or don't have a brick-and-mortar storefront. Automation software often uses something called machine learning, which means that the computer needs to examine what has worked and not worked for a store in the past and make intelligent decisions for the merchant based on this data. A larger merchant may have a lot more money to spend on ads daily; this not only translates to wider distribution of these ads, but to more data that the algorithm (or campaign manager, if a person is managing the campaign) can use to make decisions. When we built Sophie here at Lexity, we knew that not only did she have to take into account the obvious information – the number of ad clicks, the number of times an ad is shown, and how much money a keyword makes for a store – but data that was not immediately available through a service like Google AdWords or Analytics. Sophie was specifically crafted to find this extra data and use it to “learn” faster than other ad managers. Therefore, she can spend less of a merchant's money to figure out what works and doesn't work and avoid wasting valuable ad clicks on ads or keyword that aren't profitable.

For example, an ad manager might want to make decisions based on the sales a search term (or “keyword”) makes for a store. But crunching the numbers for this strategy is revealing: assuming a 2% conversion rate (which is industry standard for online e-commerce) and a reasonable $0.50 per click, a keyword would only be expected make a single sale for every 50 clicks, or $25 worth of marketing! This is, of course, a lot of money to use to see the behavior of a keyword that may not even convert for the store at all, and only provides a very unreliable data point. And this is only for a single keyword – even small campaigns contain dozens, if not hundreds, of keywords. With an ad budget as low as $10/day, merely looking at the sales data is not enough to make a decision about the profitability of a keyword.

It's clear that more information needs to come from another source – and looking at visitor behavior is a good next step to take. Many merchants already use Google Analytics and know that this gives a good overview of the amount of time that shoppers are spending on their sites and the rate at which they “bounce,” (leave the site immediately). But for an ad manager like Sophie who has to be careful about every penny that she spends, this also does not provide a full-enough picture. To illustrate one reason why this is true, let's drill-down into what a “bounce” really means in Google Analytics. Google Analytics considers a bounce to be any shopper who leaves the website after only viewing a single page. Suppose a shopper clicks on an ad and is brought to a specific product page on your site because it relates very strongly to the search term she used on Google. Because this is exactly the product the shopper was interested in learning more about, she does not need to click around the site to find it. She looks at the page for a while, decides that she's interested in the product but that it is not exactly what she's looking for, and leaves after a few minutes. Analytics will count that as a bounce. A user who goes to the homepage, clicks around for a while, and leaves quickly without looking at any product for more than a few seconds will not count as a bounce. When a store has a large enough number of visitors to a site and advertising dollars to spend, edge-cases like these are not as important and the bounce rate averages out to be a relatively reliable number. But when a keyword only get a precious few clicks a day, every visitor's intent is important for Sophie to consider.

So how can Sophie tell the difference between an interested shopper who happens not to make a purchase and a casual browser with no intent to buy? One of the coolest sources is actually from a feature that appears to have nothing to do with Sophie – Live View (read about it here). Data from Live View will tell Sophie exactly what path the user took through the store, and she can figure out how many and which products the shopper looked at and how long she spent examining each one. Sophie might decide that a shopper did, in fact, “bounce” if she spent very little time on a single product or on very few products, and store that information to use later.

A Snapshot of Google Analytics Data

Live View Data, Streaming in Realtime

Because Sophie knows these exact paths of shoppers as they navigate through your store and what kinds of actions have led to sales in the past for both your store and other stores like yours, she knows what a genuinely interested customer looks like. She can then try to get more of that kind of customer in the future by characterizing the customer based on variables like the site where she saw the ad, the search term that she used, or the wording of the ad that caused her to click. Thus, you don't need to wait around for several shoppers to look at your product and decide whether or not to buy – Sophie can make guesses about the success of a new product, keyword, or ad based on a smaller sample of visitors. And because Live View data is recorded in realtime, she doesn't need to wait three hours for a new Google Analytics report to be available – she can use the most up-to-date information every time she tunes bids or updates keywords. Sophie also has all of this data recorded for every store that has ever used Live View with Lexity, so every shopper that has ever landed on a Lexity store provides data that helps Sophie learn about the kind of customer your store wants the most. And, like an elephant, but unlike most mortal ad managers, she'll never forget about the data a single visitor provides.

This is just a snippet of what Sophie uses to make decisions in the world of online advertising, but she is constantly on the lookout for new ways to get data that bigger companies don't need or want to think about. Sophie was born with a mission: to ensure that your store will never have a data set too sparse to let your online store find its online audience.

-Ilana