Analyzing Big Data: Handling the Avalanche

advanced google analytics

The world of digital marketing is divided into 2 camps: Those drowning in the massive amount of data available to them with no skill or plan for effectively analyzing big data,  and those who wish they had some data they could use to drive strategic decisions. And, both are failing in their efforts to garner insights that can help optimize their digital marketing strategies so they can grow their business.

analyzing big data
Image courtesy of ZDNet

Above you can see the massive growth in real-time data over the last decade and projections for 2024 and 2025 that show this growth in data is likely to continue. And, that’s just real-time data that requires instant analysis if the firm has any hope of using it to drive decisions effectively. Add to this the massive amounts of data available from Google Analytics, analytics on various social media platforms, metrics coming from your email and SMS marketing programs, and results from your online advertising. Now, you face massive data streams and must bring them into a single dashboard to compare performance and make informed strategy decisions. Now consider the data coming in from IoT devices, AI, and other technologies that help you manage your business more effectively. The result, in many cases, is analysis paralysis where you feel overwhelmed by the amount of data available to the point where you simply ignore it.

Analyzing big data

First, let’s reach some agreement about what big data is. According to Lisa Arthur, at Forbes:

Big data is a collection of data from traditional and digital sources inside and outside your company that represents a source for ongoing discovery and analysis.

Lisa goes on to underscore the challenges of understanding or agreeing on big data:

Big data is new and “ginormous” and scary –very, very scary. No, wait. Big data is just another name for the same old data marketers have always used, and it’s not all that big, and it’s something we should be embracing, not fearing. No, hold on. That’s not it, either. What I meant to say is that big data is as powerful as a tsunami, but it’s a deluge that can be controlled . . . in a positive way, to provide business insights and value.

I think she did a great job of underscoring the problem of trying to get organizational traction on analyzing big data. And, most businesses are drowning because they either lack a plan for deriving insights from all the data sources available to them or lack the skills necessary to glean these insights from data or both.

marketing data management
Image courtesy of Adobe

The 5 Vs

Big data suffers from the 3 Vs of high velocity, high volume, and high variety. These have been around since 2001. GA 4 (the new Google Analytics platform) helps with the variety by combining data across multiple platforms such as the Google Play Store and YouTube to provide a more holistic view of your performance. Event tracking and UTM tracking also allow you to build a more nuanced view of performance for actions you take offsite, such as in email marketing campaigns and social media posts. Using these tools, you can build an understanding of not only which platforms performed best but also which links drove positive user actions.

We can now add 2 more Vs to our understanding of big data:

  • Veracity – is the metric a true reflection of what’s happening? For instance, my Google Analytics report sometimes doesn’t update as expected so a simple metric such as the number of visits yesterday is sometimes different if I access the report in the morning versus later in the day. Meanwhile, MonsterInsights installed as a plugin on my website provides yet another number to express the visits to my website.
  • Value – not all metrics offer value worth interpreting. For instance, number of followers is termed a vanity metric since it doesn’t correlate with indicators of success such as conversion. In contrast, the engagement of your followers does track to these key performance indicators (KPIs). Below are some KPIs to consider adding to your analysis plan.

    event tracking
    Image courtesy of Matthew Woodward

The big data avalanche

In a recent report from Forbes, 91% of business leaders said the growing number of data sources has limited the success of their organizations and 72% admitted that the overwhelming volume of data has stopped them from making decisions at all. They’re drowning in data and have given up trying to gather insights to help them make better decisions and resort to suboptimal decision-making tools such as defaulting to the way they’ve always done things. While other sources cite different numbers, the overwhelming conclusion is that a large number of firms are drowning in big data rather than analyzing big data.

In some organizations, the amount of data not only defies analysis but also strains resources just to STORE all that information, although prices have been coming down since AWS was introduced. I live in Northern Virginia where the massive growth in server farms attests to the desire of firms to store more data with the hope of analyzing it.

No big data to analyze

Other organizations struggle with antiquated ideas of WHO owns corporate data and some marketers find it difficult to get ANYTHING from their organization’s IT department. And, what they do get provides few insights because reports reflect what IT THINKS marketers want, not what they really need.

Somewhere, functional areas within the firm got the idea that protecting their data meant sequestering it away from internal users. Certainly, the more people who can access your data the more vulnerable you are to outside sources but a firm must balance protecting data with sharing it to make better decisions. Luckily, there are tools that allow firms to share some data with users without exposing the rest of their data. For instance, you can set up Google Analytics so that managers can access data related to their areas of responsibility, such as a division or product line, without sharing all your website data (BTW, much of this data IS available to outsiders through tools like SEMRush and Alexa, so you’re only hurting your decision making by not sharing data with internal users).

Analyzing big data: Building a plan

Marketers need a plan to analyze big data, which might involve struggling with IT to share relevant data. But, what goes into building a plan to analyze big data — to corral the mess so it makes some sense? I’ve actually shared my ideas about this in several recent posts. In one, I share my process of building a data analysis plan using goals and KPIs. I also shared my 4-factor model for assessing social media performance. I don’t want to rehash that content, so please visit these resources to learn more.

You can also find other sources with recommendations on what to measure. Here’s one from Avinash Kaushik, the guru from Google Analytics. He built his measurement strategy around the process consumers go through in making buying decisions and evaluations after the purchase.

metrics for digital marketing greatness


Today, I’d like to expand on what I covered in earlier posts with some salient issues for analyzing big data.

Focus on insights

Certainly, using KPIs to help determine what to analyze is valuable. But, other data points might also help optimize your digital marketing strategy. It’s important, however, to ensure you’re only collecting metrics that lead to operational insights — identifying things you CONTROL and which impact your market performance.

One of the biggest problems I see when firms analyze big data is they focus on developing access to certain metrics and then don’t use them to help make better decisions. If knowing something doesn’t impact how you manage your business, why invest time or money in analyzing it? For instance, knowing the sources of traffic to your website is really helpful in determining where to put your marketing efforts. But, knowing how many fans you have on a particular platform is really a nonsense number with NO implications for management.

Many insights come only when analyzed over time rather than at a specific point in time. Hence, while your follower count is mostly meaningless, mapping the growth in your follower count can point to actions that reach more followers. For instance, if I pay for a social media ad that increases my follower count, I can infer that the action was valuable.

Dig deep

Don’t limit your insights to metrics contained in standard reporting tools. Dig deeper for insights. For example, knowing that a particular Facebook status update did much better than average is nice. But, understanding WHY it got better lift is critical for improving your market performance. Hence grouping those posts that outperformed other posts and looking for similarities in the message, the image used, or when you posted it might provide more insights that help you craft new posts that outperform your average

To further investigate why a particular Facebook status update did better, record all the variables related to the post: time of day, day of the week, topic, headline, image, links, etc. You’ll have to score qualitative factors, such as images, to adequately express them. Monitor the amount of engagement the post received and who engaged with the post. Now, TEST each of these variables (A/B testing is best) to determine which factor or factors contributed to the relative success of this update. Knowing which factors are salient gives you the information needed to create MORE updates that exceed average ROI.

I really like building regression models using the testing results because it’s likely several factors contributed to lift. Regression provides insights into the factors that affected lift the MOST. Often, a combination of these factors will drive the highest return.

Build nuanced insights

You can also dig deeper into your data using tools like Reports in GA 4 to determine which demographic or geographic group performs best when it comes to conversion or average order value. You can see which groups are most likely to leave a product in their shopping cart rather than complete a transaction. Below is a custom report I created to view the impact of age and gender on the number of pages viewed per session but you can build similar insights for any variable you think is worth tracking.

universal analytics
Demographics based on PageViews/ visit

Don’t stop with the numbers

We know most data available in social media is qualitative — unstructured — data, such as words and images. Scoring is a nice way to analyze this data (and is the fundamental behind how most SAAS products handle unstructured data), but you lose SO MUCH information when you reduce rich, qualitative data to a number — like in sentiment analysis.

Don’t stop with analyzing numeric data! Try to understand the beliefs, emotions, and norms expressed in the utterances of consumers on social platforms. Such utterances are guides for unmet needs, problems encountered with your product, who influences buying behavior, and a host of other insights.

For instance, I analyzed a Disney forum and discovered that multiple folks mentioned the advantage of staying near a tram stop at a Disney resort. Disney used this information to decrease the average distance from each room to the nearest tram stop. Without such analysis, would Disney have ever realized that tram stops were an important decision variable for resort visitors? Probably not.

Don’t feel that every number has some meaning

Just look at Google Analytics. Here’s just a partial list of the analytics available:

metrics in google analyticsAnd, with each metric, there are multiple data points such as the percentage of the total. Plus, you can always add filters, sources, mediums, dimensions, etc. That’s an incredible amount of data and Google Analytics is but one of the myriad of tools available. Likely MANY of these numbers have little meaning and others are beyond your control. Trying to assign meaning to everything is like looking for a needle in a haystack — your vision is clouded by a bunch of stuff that doesn’t matter.

Again, KPIs help guide your exploration within your data, but it’s dangerous to be too narrow in your analysis as you might overlook valuable insights. For instance, if you’re focused too closely on metrics that impact your market performance directly, you might miss factors with an indirect impact on performance, such as complaints.

I once taught a bunch of MBA students from big companies like P&G, GE, and Toyota. Their data-driven organizations taught them to pay close attention to data. However, they only paid attention to behavioral data — how much, when, and how often. They believed I was the stupidest Ph.D. on the planet because I advocated for monitoring consumers’ emotions relative to the brand and understanding factors BEHIND the numbers — the why of consumer behavior.

Unfortunately, too many organizations — big and small- fall into the trap of over-analyzing big data and missing its subtle insights. Don’t get so focused on analyzing the minutia that you overlook the big picture.

Need marketing help to support business growth?

We welcome the opportunity to show you how our data-driven, results-oriented marketing strategies can make your marketing SIZZLE.  Sign up for our FREE newsletter, get our FREE guide to creating an awesome website, or contact us for more information on hiring us.

Hausman and Associates, the publisher of MKT Maven, is a full-service marketing agency operating at the intersection of marketing and digital media. Check out our full range of services.