Consumer confidence is at its lowest level since 2014, but spending increased in August. Kari Butcher, Burson’s president of intelligence and insights for the Americas, says business leaders make the same mistake with election polls: treating what people say as a prediction of what they will do.
Business leaders make consequential decisions based on a category error: treating what people say in a survey as a reliable prediction of what they will do next.
Consumer confidence plummets and a retailer cuts inventory. A majority supports a social issue and a CEO assumes that speaking out will build customer loyalty. A company’s reputation remains stable and its management team concludes that customers aren’t going anywhere. Either way, the poll may be accurate. The business conclusion may still be wrong.
Polls measure what people are saying at any given time. They alone cannot tell a leader how that sentiment will translate into purchases, loyalty, employee behavior, or investor response. Get this distinction wrong and a company may withdraw while demand remains strong, enter a debate its customers didn’t ask it to join, or mistake the friction of change for true loyalty.
Midterm exams make things more difficult. The economy is voters’ top concern: 84% of registered voters told Pew that it was very important to their vote. Polls are everywhere and executives want to know what the election will mean in terms of spending, investments and what customers and employees expect. A poll showing who is in the lead can help a company anticipate the political environment. He can’t tell them how consumers, employees or investors will react to it. The average of recent generic CNN polls gives Democrats a 52% to 42% lead, but that’s a measure of preference, not what a customer will do next.
As a former pollster, I don’t think the polls are flawed. The question I would ask of any poll is what decision does it give me enough evidence to make. This means knowing whose opinions the number reflects, what is behind those opinions, and whether people will act on those opinions.
Take the economy. The Conference Board’s consumer confidence index fell 6.7 points in September to 81.9, its lowest level since 2014. The next day, the government announced that spending rose 0.9% in August while personal income rose 0.2%. Households made up the difference by saving less: the savings rate fell to 4.1%. The sentiment indicates that people are deeply pessimistic. The spending indicates that it continues. Both are partly true, because people can feel one way and spend another. A retailer who only monitors the sentiment number would not notice this.
Elections show how quickly a stated preference can be confused with predicted behavior. About a week before Michigan’s Democratic Senate primary, a poll had Abdul El-Sayed ahead by 15 points, 54 percent to 39 percent, among likely primary voters. He won by less than a point. One possible factor is the assumption about who would actually vote. In early October, YouGov’s chief scientist found that the same survey could show a Democratic lead in the national House vote of between 6 and 16 points, depending on the likelihood of voter identification. He called the high end an exaggeration and said defensible picks yielded 8 to 14 points. The answers have not changed. Assumptions about behavior did.
This is why one number is never enough. The Washington Post made a similar point in late September: a single poll is a snapshot, and trends across polls tell us more.
Businesses run into some version of this, and I must reveal that the evidence here comes from Burson’s own research. In the Burson Pulse, a survey of 1,601 U.S. adults conducted in late April, 61% said economic anxiety was causing them to save more and pay more attention to what companies say and do. Four months later, national data showed that households were saving less. The two measurements are from different months, so they are not a direct test, but they do show the risk of treating a stated intention as a forecast. Burson’s Pulse study is designed to track changes in trust over time. During the first wave, 48% of Americans said they gradually lost trust in a company, without being able to name a single moment. 52% said one event was enough. Trust can erode slowly, long before it shows up in a headline or metric, so an overall score may not reveal a decline in trust until the damage is done.
Don’t throw away the surveys. Stop asking one source to do everything. Match what people say with what they search for, share, buy and do. Read what is being said online and continue to ask people directly. Before acting on a survey, leaders should ask themselves one question: What decision does this give us enough evidence to make? With the midterm elections just days away, this question is the most important for the polls everyone reads: They’ll tell you who’s leading, not what your customers will do next. If the answer is unclear, find the missing evidence before making a decision.
The opinions expressed in comments on Fortune.com are solely the opinions of the authors and do not necessarily reflect the opinions and beliefs of Fortune.
Gn bussni
