Data Analysis in Marketing; What Google and the Flu Can Teach Us

I saw an interesting tool the other day from Google, that analyzed raw data on searches related to the flu in order to predict the severity and timing of flu outbreaks. Available at http://www.google.org/flutrends/, this tool is an interesting example of approaching data in a unique way in order to understand a problem.

Whereas an individual person searching for a flu-related topic is clearly not a diagnosed flu victim, the correlation between those searches and an actual outbreak is quite good, and more importantly, the data is available much faster than the medical diagnoses precisely because it is not relying on a properly diagnosed flu victim as the base data point it works from.

As we implement lead scoring algorithms, and other predictive approaches within our marketing automation systems, we face a similar challenge. The best way to look at whether we have scored the leads the right way is to look at whether the leads we passed to sales became qualified opportunities and eventually closed revenue. However, similar to the medical diagnoses in the flu example, this can take a significant amount of time. However, the raw data can show us some very interesting trends and give us immediate insights.

When you have built out an initial algorithm, incorporating the best practices for lead scoring, the simplest thing to do is to pass your entire dataset through the algorithm to see how they would score. This “bottoms-up” look at the data gives a very quick view of the potential results. This technique is best used when looking at the explicit score (the “who” that indicates the right buyer in the right organization) rather than the implicit score (the “how interested” that indicates the level of current buying interest. The reason for this is that the “who” is not likely to change over time, while the “how interested” will obviously vary significantly over time.

With your entire database scored through your new algorithm, the results will tell you some very interesting things.

- Were the final numbers what you expected? If you scored 100,000 contacts on the explicit dimension of lead scoring, and only 0.1% ended up as A leads, was this what you would have anticipated? If you were expecting significantly more, it could easily be a data problem. For example, if your scoring algorithm looks for a key title, such as “Vice President of Marketing”, and you have not cleansed your data, you may miss most, if not all, of the contacts you are looking for. In our own experience, that search returned only 300 results before cleansing, and over 17,000 results after data cleansing to standardize all the other ways of writing “VP. Marketing”, “Vice Pres Mktg”, etc.



- Does the sales team like what they see? If you show your sales team a sample of the leads who scored well in your new algorithm, do they think that these are the right set of leads they should be speaking with? If you show them leads that did not score well, do they agree that these are not leads they would like to speak with? Remember, of course, that you are only looking at explicit information, not buying activity in this example, so it is assumed you would only be passing your sales team these leads when buying activity was detected. Balancing your sales team’s intuition with your objective lead scoring algorithms is as useful here as it is in highlighting flaws in the process through sales “cherry-picking” of leads.


- Does history agree with your hypothesis? When Google looked at their flu data, they compared it carefully with CDC data on actual flu outbreaks to ensure that there were minimal false negatives and false positives. Similarly, your lead scoring results need to match history accurately. If you look at contacts who were scored highly, and who have also been around for a long time, is there a higher number of them who have become customers? It not, what is missing?

Marketing data can be a true gold mine of insight if you use it carefully, much in the same way that search data shows extremely interesting predictive insights when looked at in certain ways.

What insights have you found in your marketing data that surprised you?

The Buying Process; Auditing your Content Assets

Shifting your thinking from a selling process to a buying process means understanding how buyers buy and what stages each type of buyer goes through. When you have mapped out your buyer roles and the stages in their buying process, the next step is to determine what content is needed or available at each step in order to facilitate their buying process.

A very useful exercise to undertake is an audit of your existing marketing content in order to understand what content you currently have available and what new content may be required. This audit leads to a content matrix that forms a very useful basis for defining both your nurture marketing approaches and your content development strategy.

Creating the matrix is relatively straightforward. On one axis, list the key buyer roles you encounter; perhaps these are the standard technical buyers, user buyers, and economic buyers that are common definitions across many industries, or perhaps you have a more specific set of buyers you engage with.

On the second axis, list the stages of the buying process for your customers. Generally, this goes through three stages:

- Education and Awareness, where buyers learn about what is possible or required in a given industry
- Vendor Discovery, where buyers determine what vendors are available to solve a particular business pain
- Solution Validation, where buyers short list and ultimately select a particular vendor to assist them with a specific challenge

However, the phases of a buying process for your business will be quite specific.

With this matrix complete, you can now remove any cells that are not relevant, perhaps if user buyers are only brought in late in your buying process. Each cell in your content matrix now represents a combination of a buyer and the phase in the buying process that they are going through at a certain point in time. Each of your marketing assets should map to one of these cells (or perhaps a small number of cells in the case of certain broadly applicable assets).

By filling in the content matrix with your available marketing assets, and perhaps color-coding with red, yellow, and green based on quality, you get an overall view of where in the buyers’ buying process your content assets exist, and where they are strong. Some obvious gaps may become immediately apparent.

This content matrix can then be compared against the marketing challenge you face. As you plan your lead nurturing strategies, understanding where in the marketing funnel you need to focus on and where in the buying process your content exists is key.

Medical Symptoms and B2B Marketing Processes

The medical world is very used to the fact that symptoms such as pain, headaches, and nausea are often the body's way of indicating that something has gone wrong. When a treatment is applied to the symptoms alone, it can mask an underlying problem. For example, regular unsupervised use of antacids for heartburn or reflux disease might mask the symptoms of more serious things such as barrett's esophagus, a stomach ulcer, or stomach cancer.

I was reminded of a similar challenge when a lively discussion ensued on LinkedIn about whether marketing should allow sales to “cherry pick” leads. I had written a post talking about the challenge to the overall flow of the funnel when sales is allowed to cherry pick leads. The natural flow of scoring leads, qualifying them, and then handing them off to sales is disrupted when a sales team is allowed to grab the best leads from the funnel regardless of where the scoring system has placed them.

The group discussing the issue agreed, in general, that the disruption of having a salesperson cherry pick leads was not ideal, however, it had the critical benefit of pointing out flaws in the scoring process. A salesperson cherry picking leads that the lead scoring algorithm has not deemed ready for sales may be an indicator of a deeper underlying issue. Perhaps the scoring algorithm misses certain key activities as criteria, perhaps there are opportunities in verticals outside of where the lead scoring algorithm focuses.

In the same way as medical symptoms are uncomfortable for a reason, to get your attention to a problem, the disruption of the smooth flow of leads caused by sales cherry picking can be seen as a symptom of an underlying problem. Whereas I still feel that sales cherry picking is an undesirable outcome, I do believe that if analyzed as a symptom, it can provide interesting and crucial insights into where a lead scoring process is missing the mark.

Sometimes, allowing a few "breaks" in a marketing automation process is a great way to identify opportunities to improve the overall process. When looking at your marketing team's interaction with sales, look for where you see breaks in the process, and look for processes that would completely prevent or totally mask breaks in the process. Removing a symptom of a broken process, much like removing a medical symptom, may only be hiding a deeper underlying problem.

Marketing Automation - What does it mean?

Consistently, the term “marketing automation” is applied to our industry. I find it a term that is less than ideal when it comes to describing what the industry does, but nonetheless, the term has stuck. I often get the question of “can marketing really be automated” and I think that there’s an opportunity to clarify what aspects of marketing truly can be automated, and what aspects cannot. Also, given that the industry’s growth is surging, there’s an opportunity to look at the dynamics that are driving this desire to automate marketing.

If asked for a definition, I would say:

Marketing Automation is the art and science of automatically managing the targeting, timing, and content of your outbound marketing messages.

Let's look at why.

The Changing Buyer

First, let’s look at our buyers. Over the last decade, their ability to self-educate and manage their own buying processes has increased astronomically. Whether it is through vendor websites, analyst websites, social media, or peer reviews, buyers can acquire the information that they need in order to move towards a purchase decision.

However this transition in our buyers has meant that we as marketing organizations need to work with our buyers differently than we did previously. If we hope to facilitate their buying process, and guide them to consider, prefer, and select the products and services we are offering, we need to provide them with the precise information they need as part of their buying process.

The most critical factor is relevance. If we are able to deliver information that is relevant to the buyer’s role in the buying process, their stage as a buyer, their level of interest, and the areas of decision making of interest to them, we will establish a connection, and our message will get through. If not, our message will be lost in the clutter.

So where does marketing automation come in?

If you look at what is required, we need to first understand each of our prospective buyers individually, then we need to provide a message to them that has ideal timing and content based on their interests and stage in the buying process. This level of precision on targeting, timing, and content is nearly impossible without having a solid underlying platform to work from. The art of marketing, when it comes to creating persuasive, compelling copy, great offers, and elegant positioning cannot be automated, and likely won’t be in our lifetimes. However delivering the right selection of those messages to the right person at the right time is something that can no longer be done without automation.

Marketing Automation and Timing

Marketing automation gives us the ability to work on the exact timeframe of the buyer. This is best understood in the context of marketing initiatives like downloadable free trials. In downloading a free trial, the buyer has indicated that, at that exact moment in time, they are at the stage of their buying process where spending time with a trial is their most appropriate use of time. Your communications, as a marketer, need to reflect this buyer timing in order to best connect with this buyer.

Without automation, if a marketer is to attempt this with batch communications, the more closely one tries to align with the buyer’s timing, the smaller and smaller your batches must become, and the larger your workload as a marketer grows. Only through automation can a marketer effectively deliver a message on day 1, day 15, day 30, and day 90 to each individual prospective buyer.

Marketing Automation and Personalization

As one communicates with prospective buyers, each communication should ideally contain content that is precisely in line with their interests. The best way to do this is with dynamic content that automatically matches content to their interests. However, most prospective buyers will not explicitly declare their interests. If they do, the data is likely to be inaccurate. Marketing automation, by letting you tie web activity into buyer insight allows you to understand buyer interests based on what they do, not what they say.

On top of this foundation of buyer understanding, marketing automation gives you a platform from which to have that insight automatically personalize outbound content. Manual processes to personalize the content would quickly prove impossible, and the impact of not personalizing the content is a significant decrease in its relevance to the buyer.

Personalization and the Sender

However, the content itself is not the only aspect of personalization that impacts the buyer’s likelihood of engaging. Who it comes from is equally important. Recipients are 30% more likely, in most cases, to interact with content if it comes from a known person, rather than from a company. A marketing automation system can automatically have each communications come from the appropriate member of your sales team, building the relationship while increasing the response rate. To send your communications on behalf of 5 or 10 sales people might be possible if done manually, but to send on behalf of 50 or 100 requires automation.

Marketing Automation and Sales

As buyer progress through their own buying processes, they eventually reach the point when they would be willing to talk with someone in your sales team about pricing, contracts, or other elements of the purchase process. Knowing when they have reached this stage involves understanding their digital body language. Signs of buying activity can be seen and with the appropriate lead scoring algorithm, sales ready leads can be identified and passed to sales.

Whereas this analysis of leads has in the past been done manually or with spreadsheets, the need to identify and follow up with leads when they are most ready means that it must be done quickly. Using automation allows marketing teams to objectively and automatically score the leads in their marketing funnel in real time, identifying those that are sales ready and those that are not. Those that are not yet ready can be kept warm over time through lead nurturing, again a process that automation greatly facilitates.


The New Importance of Data

What we’re seeing in the above discussion is a shift away from batch communications that are not highly differentiated to true one-to-one personalization. However, as we do this, and we have marketing automation platforms, rather than people looking at the data to make decisions, the importance of data quality takes on a new priority.

Data, in order to be used by rules for personalization of content, segmentation of lists, and scoring and routing of leads, needs to be clean and consistent. Marketing teams are being tasked with ensuring this consistency is maintained at all times, even though marketing data may be touched by web forms, list uploads, CRM synchronizations, or sales input. The only way to consistently and constantly maintain a clean and standardized set of data is to use automation to manage marketing data quality in-line within the marketing database.


What Can’t be Automated?

Marketing remains as much art as it ever has been, even as the new buyer requires elements of science in automating how the targeting, timing, and content of a message is delivered. Compelling offers, captivating visuals, great positioning, and elegant copy are as difficult as ever to create. Likewise, the understanding of market segments, buyer journeys, stages of a buying process, and what moves a buyer along their buying process still differentiate excellent marketers from merely good marketers.

However, as today’s marketers shift from outbound messaging to understanding a buying process and facilitating it, they can only do so if enabled with a platform that automates the conversations, timing, and personalization needed. That is where marketing automation comes in.

A/B Testing; What Result are you Testing Against?

Most B2B marketers today understand the importance of testing. A quick change in a subject line, some copy, or an offer can dramatically alter the performance of a campaign. At Eloqua, our own marketing team does A/B testing on almost every piece that is sent out. However, the question we all face as marketers, is what result we should be testing against. In B2C marketing, it is often significantly easier, as campaigns are often designed to drive explicit purchase behavior, and success can be measured against revenue results.



In B2B marketing, however, the revenue result is often significantly further away. Ideally in testing a campaign, one would be able to determine which drove more actual buying behavior, but with most buying processes stretching out over months, it would be impossible to efficiently test and launch a campaign in this manner. Inversely, testing against the common results of opens, or click-throughs only tells a small fraction of the story, as there is only a very loose tie between an email open or clickthrough, and the final result of a purchase.

Luckily, our options for what results we test against, in fact, span a spectrum. A careful selection of the end result to test against guides exactly what our tests will show. Looked at along two dimensions, we have the following options to test against:



Email Opens: The easiest to test against, but the least accurate by far. This is mainly an indicator of the quality of your subject line. Whereas this can be tested very quickly, it suffers from technological differences in email platforms, as well as a very limited testing scope.


Email Click-throughs: Slightly more difficult to test against as it requires tracking of links clicked, but this is common in most email platforms today. However, this also suffers from a very limited accuracy, as it does not indicate much about the recipient’s interest in purchasing


Inquiries: Tracking of which email drove more inquiries (landing page form submissions) is a significant bump in test accuracy. This now tests whether the subject line was compelling enough to lead to the email being read, the content and offer was interesting and drove a click-through, and the landing page was optimized to maximize form submission rates. This is a very comprehensive test of your campaign, and usually sees results within one or two days of running an A/B test campaign


Qualified Inquiries: Even higher in terms of testing accuracy is testing against qualified inquiries. If, in the A/B test, email A drove 100 inquiries, and email B drove 80, but most of B’s inquiries are the right target executive, while most of A’s inquiries are students and more junior staff members, clearly email B is the best option. Note that the dimension of lead scoring we are talking about here is explicit scoring, as we are just looking to see whether the right executives are the ones inquiring.


Opportunities: We do see an increase in accuracy as we move to Opportunities as a result to test against, but this also increases our difficulty significantly. There are often many more factors involved in qualifying an opportunity as being ready for sales than just one campaign, so this leads to a significant increase in complexity of the situation to analyze.


Revenue: This is clearly the highest accuracy to test against, but the length of a sales cycle means that it is prohibitively difficult to work with, and the ideal timing to run the campaign in question may have long passed by the time that the test results are available. The way we need to think about B2B marketing analysis means that, in general, it is nearly impossible to test the effectiveness of a single campaign on revenue in a meaningful way.



Each of these options has benefits and drawbacks, so it is important to consider what you are testing against when defining your A/B test. In my experience, in a B2B marketing situation, testing to determine which option produced more qualified inquiries often provides the optimal balance between ease of testing and accuracy of results.