Why Feedback From Customers Still Matters in CX Research

by Mike Deinlein

Don’t leave them out

The proliferation of customer data has made it easier than ever to create more engaging customer experiences and effective marketing tactics. But as companies become awash in customer data, some are considering eliminating traditional CX and brand-tracking survey systems in favor of purely database solutions. Why ask customers what they think when you can collect data on customer interactions and use that to predict churn, customer lifetime value, next-best actions and more?

The promise is alluring and companies should continue to invest in predictive customer analytics to deliver personalized and micro-moment interactions with customers. However, it’s clear that traditional survey systems still play a critical role in telling the whole story by filling the empathy gaps that database solutions alone often struggle to address.

Let’s explore some of the evidence that supports the use of surveys to reinforce the depth and relevance of insights derived from customer data.

A Lasting Advantage

Companies have recognized that real-time marketing and customer experience engines are core drivers of business value and are quickly becoming imperative to establishing a lasting market advantage. The business case is well-documented.

• Eighty-eight percent of customers believe that the service they receive is as or more important than the products a company offers.¹
• Customers are 3.5x more likely to purchase from a business after a positive customer experience.²
• Businesses that invested in customer experience boosted revenue by 70%.³
• Nearly 10% of company revenues are at risk due to bad customer experiences.²

Recognizing the impact that customer data could have on creating more engaging, more personalized customer experiences, researchers and marketers invested heavily in tools, technologies and systems that enabled a deeper understanding of customer needs and motivations. Today, these tools produce, collate, analyze and operate on copious and varied sources of data to support the creation of more personalized, more relevant customer experiences. But before companies can craft personalized marketing and CX solutions, they need to identify the different data sources at their disposal and define what each brings to the table. The typical data stack has five primary components:

• DIRECT FEEDBACK – solicited feedback with surveys or direct customer contacts;
• INDIRECT FEEDBACK – what customers say about you in chat logs and transcripts, social media and service interactions;
• TRACKED BEHAVIOR – omnichannel behavioral tracking from digital browsing, IoT, visitation, transaction history and service records;
• OPERATIONAL – organizational data on customer outreach, promotional information and marketing campaigns;
• FINANCIAL – timing, frequency and volume of purchase, cost-to-serve and profitability.

Traditionally, direct feedback and survey-based measurement systems were the primary drivers for uncovering customer needs. By engaging customers and understanding their pain points, researchers could then envision solutions to address their frictions. And while many companies had other data that could help uncover greater insight around customer needs, they did not have the resources or tools to make sense of it all.

However, the accelerated pace of digital transformation and the introduction of technologies have made it easier to capture, analyze and understand indirect feedback, tracked behaviors and operational data. Consequently, these database solutions have allowed companies to leverage such data in forecasting churn, retention, customer lifetime value, next-best actions and offers, while also informing the deployment of personalized marketing content in the most effective channels. Natural language processing has made it easy to understand massive quantities of unstructured data in ways that seemed impossible just a few years ago.

While this proliferation of sources provides more opportunity than ever to discover and act on customer needs, it also presents challenges in knowing what data to use, when and for what purpose.

CX Surveying Will Effectively Be Dead?

At a recent customer experience conference we attended, one of the keynote speakers made a provocative claim: By 2030, only 1% of customer data will come from surveys and the customer experience survey industry will effectively be dead. While the facts of this statement are sound, the conclusion is misguided.

The exponential growth of customer data isn’t slowing. Direct feedback will inevitably become a smaller and smaller sliver of the overall data stack – perhaps only 1% of all customer data, as the claim suggests. But this relative reduction in size does not necessitate a corresponding decline in importance.

In fact, direct feedback provides the opportunity for a level of empathy that the other data sources often lack. It allows you to understand the why and the how of customer behaviors by engaging with them, listening to their pain points, understanding their workarounds and then crafting solutions that address those problems. We must acknowledge the strengths of direct feedback, while at the same time recognizing opportunities to augment and improve traditional measurement systems with behavioral database solutions.

As customer data grows more prevalent, solutions more predictive and business functions more automated, we believe that the need for management to maintain a direct source of feedback will become even more important. Our reasoning is both practical and philosophical.

A key feature of survey research that cannot be easily replicated via database or transactional data is what people say about who they are and why they do the things that they do. Consumer psychology tells us that direct responses collected through survey research are subject to various biases inherent in human cognition. They sometimes do not reflect what consumers truly think about a topic and, consequently, don’t accurately predict future behavior. Yet, despite all that, surveys provide vital approximation of how consumers reason and rationalize in the course of their decision-making process. Their responses are analogues to System 2 – slow and deliberate thinking that constitutes the stories customers tell themselves to understand their own thinking, mind-set and behaviors. Yes, relying solely on what people say inevitably leads to bias because people often say one thing and do another. But if we only observe what people do without hearing their rationale, we miss invaluable context on their underlying motivations, predispositions and personal aspirations.

Imagine a tale as old as time itself…

Let’s say a customer was asked in a survey whether they are likely to purchase a box of Oreo cookies during their next trip to the grocery store. They say, “No way. I am trying to watch my weight and live a healthier life.” But upon seeing those delicious cookies in the aisle, they put them in their cart – and promptly polish them off as soon as they get home. Proponents of predictive data analytics that rely exclusively on behavioral data may look at this as proof of the fallibility of survey research. In their view, a company would be better served by observing this customer’s past behaviors, recognizing patterns in those behaviors and predicting that they are a suitable candidate for habit-loop consumption of tasty treats.

But a more nuanced view – built around the reinforcing insights from predictive and traditional methods – reveals something truly invaluable in this “conflict of self-interest.” This potential customer may tell themself that they want to eat healthy but enjoy the taste of cream-filled chocolate cookies too much to pass them up; that means something.

So, armed with both sets of data, Nabisco can justify a healthy, low-calorie innovation that offers the same great taste its customers love. This individual feels happy that they are taking steps that align with their healthy-living narrative while still exhibiting purchasing behaviors that are beneficial to Nabisco. And so was born the Oreo Thins line of cream-filled cookies.

This vignette shows that to appeal to customers on a deep or emotional level, we need to consider the aspirational rationales that people have about their lives and life choices. And the best way to understand these aspirational rationales is to ask about, and listen to, the stories consumers tell.

A More Holistic Picture

Consequently, it is by effectively marrying self-reported survey metrics and behavior metrics that we get a more holistic picture of consumers, one that integrates both System 1 inputs (what consumers ultimately do) and System 2 thinking (why they think they did what they did). When we’re implementing classification or predictive models at Burke, we often see notable accuracy improvements when both behavior and self-reported metrics are used in models.

One recent example involved a Fortune 500 utility provider. Our client needed help predicting the likelihood that customers would sign up for its green-energy programs. It had a wealth of internal behavioral and operational data, along with lifestyle enrichment data from a third-party provider. However, by layering on direct feedback from customers, we were able to create a model that could best predict program enrollment and delineate between the next-best green product for targeting its customers. Certainly, this provided an empirical benefit, but anecdotally, self-reported metrics also bring context to why people behave the way they do. Why is someone likely to sign up for paperless billing but less willing to sign up for clean-energy access programs? Layering on direct feedback provides a deeper level of insight to understand these nuances and inform business strategy.

Additionally, we must use survey metrics not only to mitigate risk but to validate and protect consumers from algorithmic bias. For example, behavior data can help us extrapolate more efficient pricing strategies through dynamic pricing. However, to ensure marketing tactics do not cause undue negativity among customers or treat customers in a way that is detrimental to certain subgroups, there needs to be a constant stream of direct consumer feedback to ensure marketing activities are working as intended. Unwittingly exaggerating an existing bias in marketing or using behavior levers that favor some subgroups without understanding how consumers feel can bring trouble in terms of both business outcomes and ethics.⁶

One of the more notorious examples of perpetuating bias is the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) algorithm, which is used to predict the likelihood of recidivism and act as a guide for when criminals are sentenced. An audit of the software found that Black defendants were almost twice as likely to be misclassified with a higher risk of reoffending (45%) in comparison to their white counterparts (23%).⁷ While successful experience management is a stratospherically less serious topic, the lesson remains that companies need to be aware of their biases, recognize the burdens these biases place on certain customers and then make every effort to alleviate those burdens. One of the surest ways to uncover these pitfalls is through listening to direct feedback to bring empathy to behavior-only models.

Detailed the Gaps

So how does this come together in practice? Let’s look at customer experience as an example. Many publications have detailed the purported gaps of survey experience management. The most common objections are that survey systems are:⁴

• CONSTRAINED – even the best survey-based management systems cannot reach every customer;
• REACTIVE – surveys measure past experiences with a brand, so proactivity is inherently limited;
• UNFOCUSED – survey scores are rarely linked back to financial impact, so it is difficult to justify investments based on expected business value.

While these are valid concerns, the proposed solution to focus solely on automated, database customer experience management disregards the benefits of hearing directly from customers, as described earlier. To illustrate this point, take a common refrain from database solution-only proponents: Why use a survey to ask customers about their experiences when data about customer interactions can be used to predict loyalty?⁴ The logical retort is “Define a loyal customer.” More pointedly, “Define an advocate.” Are you creating a remarkable enough customer experience that funnels prospects to your brand or an engaging enough experience that ensures your customers will ignore competitive interventions (like discounts or promotions) to stick with your brand? Or are you creating customers who only stay with you because they have not found alternatives? The easiest way to know what consumers actually feel is simply to ask them.

A customer can follow the ideal CX path. They can nail all the right touchpoints, move through the purchase cycle or service cycle in record time, hit all the behavioral interactions you were hoping they would … but when they think about their experience, they can still be left dissatisfied. Database solutions will say there is low risk of churn but their human experience begs to differ. Humans are emotional and emotion is still difficult to predict – despite the advances in artificial intelligence and technology solutions attempting to do just that. That is why companies should see these two CX management systems as complementary rather than as surrogates for each other.

Figure 1 presents the three gaps and see how a complementary CX management system addresses them.