Editor’s Note: Few would question the fact that data-focused digital marketing has given rise to a new type of executive. But George Sadler stands out even among his new-age peers. How many top marketers do you know who have been a staff sergeant in the U.S. Marine Corps, an infantry captain in the U.S. Army and – yes, and – a special agent in the FBI; who speak Arabic; who hold two patents; and who have engineered major change at two massive stalwarts of modern technology?
George has. And while all that certainly makes him a very fun conversation, it also makes him a foremost practitioner of data-oriented decisioning in driving revenue growth. After managing Dell’s Analytics and Information Management software portfolio and helping to found the company’s global marketing sciences organization, he is now the global leader for CRM and Loyalty Analytics at eBay. His team is part of a new group at the online auction giant that brings together data scientists and business analysts to keenly understand consumer behavior via A/B tests, econometric/multivariate models and other techniques.
Over a two-post series here on “Perspectives,” we have asked George – he far prefers if you call him by his first name, by the way – to detail how his team applies those learnings to create marketing-driven growth via new buying and selling activity on the site, and how the sales-like way they “bank” the results of this work creates a new model for marketing management. This first post focuses on the methodologies they use and team skills they hire for, and next week we will post part 2 on the compensation model – and cultural vibe – that drives their results. Enjoy! And thank you, George! – Brad Young, Global Content Marketing Strategy Leader, Dun & Bradstreet
* * *
Our charge within eBay is very likely similar to yours: Understand the customer. We obsess over it the same way you do. What do we know about them – and how can that information drive relevance and personalization? You ask yourself and your team these same questions. How we get to the answers at eBay – and how those insights shape our overall approach – is where I think we’re a bit different.
That uniqueness stems from one other key question we went on to ask ourselves here: What if we could hold marketing to the same standards as sales, demanding that marketers retire a quota as real and measurable as a purchase order? It is a shift in mindset that has, in turn, evolved our definitions of accountability, our processes and the skill sets we look for in our team, all in ways that have driven the tangible growth our management demands.
Here is a breakdown of the process we employ and the team skills we seek. My hope is that it sparks actionable ideas that let you create direct connections between marketing activities and new revenue for your company.
Process: A Methodology of Causation, Not Correlation
Many marketers – especially B2B marketers – use attribution modeling to measure the ROI of their activities. And that can get fairly sophisticated, with econometric models and regression models and partial attribution models. What our work is showing us, though, is that correlation does not mean causation, and there is often no connection between attribution points and incremental revenue. That bears repeating: No matter what model we pick, and whether we are measuring first click or last click or something else, we rarely see a connection of any kind to those attribution points and what is really driving incremental revenue for eBay according to the strict testing approach we adhere to.
Members of the CRM and Loyalty Analytics team blazing new trails at eBay
The only way to really know whether you’re causing something to happen is to set up a strict experiment following the scientific method. Through randomization you set up controls for all activities except the treatment – the marketing stimulus – you want to measure. When everything else is randomized, you can claim with some certainty that the resulting customer behavior was caused by the non-randomized treatment you put in place.
We achieve this through exhaustive A/B testing. My team primarily focuses on email, notifications (such as mobile app push notifications) and loyalty programs. Given the scale of our global business, we can set up tests and controls that are much larger than most researchers are used to. By turning on and off who receives a certain message, we can know with relative certainty that the message drove the activity. We can show that we are increasing relevance and personalized experiences. We don’t necessarily care that you are likely to buy shoes (a straight predictive model). We care whether or not weknow you will buy shoes if we send you a certain email (a conditional probability model).
At eBay, our marketing teams are goaled against delivering incremental sales volume. And the methods we employ are the best way to create the tests that prove you’re going to achieve sales volume growth. What we are learning is influencing marketing activity across the company. Outside of email, as an e-commerce company we have of course always done A/B testing on the site, but teams are also employing this approach to everything from search keywords to highway billboards.
Two critical points to keep in mind as you explore this approach. First, get complete management buy-in before you embark. We are fortunate that our team senior leader sponsorship. Without that, your management will likely need more hand-holding as you approach this more scientifically rigorous approach. And second, use common methodologies globally. As you collect and collate the results of these tests, you want to make sure you’re using the same terminology and definitions of success across projects and continents. Executives go bonkers when one number conflicts with another. Don’t let it happen.
Skill Sets: Powerful Pods
The people I work with at eBay make up the most technically skilled analytical team I have ever been around. PhDs with science-based research skills team with marketing partners and product leaders to develop ideas and drive incremental sales volume. We have a particle physicist. We have PhD statisticians and classically trained economists. We hire from pharmaceutical companies because of the testing rigor they must embrace. Then we deploy them with marketing and product in a pod-based structure where dedicated groups tackle, say, new customer analysis or seller behavior.
The diversity within the pods creates a terrific dynamic where individuals add clear value. I have been in organizations where we had someone in analytics who knows a little something about marketing or someone in marketing who aspires to do analytics. With us, people stay in their lanes, and the partnership that results is one where we have very few of the “pull numbers for me so I can make a decision” requests and more of the “tell me what you think we should do” conversations. The structure of the pods also lets us move quickly from a test to scale and to fail fast. It’s a continuous flywheel of iteration, using the customer insights we have to improve the testing techniques for the next time.
The groups share in their goals. Together pods want to “bank” and retire more quota. And for how we have gotten a bunch of marketers and PhDs to embrace such salesy metrics of success, come back for Part 2 next week.