Too many CEOs don’t know what their workers need. Employee ‘engagement’ surveys can make the problem even worse

Illustration of a maze in the shape of a smiley-face.
Poorly designed employee surveys can convince companies that their employees are happier than they actually are.
Illustration by Nicolas Ortega

When Starbucks’ founder and three-time CEO Howard Schultz spoke before the U.S. Senate in March to address allegations that his company was engaging in egregious union-busting, he seemed almost hurt by the behavior of his employees. How could these workers be unhappy and organizing when the coffee-store chain had long lavished them with perks like college tuition assistance and comparatively generous health care coverage? “Is there a union that you’re aware of that has those benefits, sir?” Schultz pointedly asked one senator.

It turns out that for many Starbucks “partners,” as the company calls its employees, tuition help at an online university wasn’t that crucial. They’ve proven to be far more interested in prosaic matters such as flexible scheduling, work conditions, and more predictable hours—the kinds of issues that have a much greater short-term impact on their income and quality of life. 

Starbucks is hardly alone among companies that seem to be misreading or not taking sufficient heed of what employees are saying, nor is it the only place where worker dissatisfaction is on display. Indeed, an annual Gallup survey of 15,000 U.S. workers published in January found that employee engagement in the U.S. had fallen to its lowest levels since 2015. Among the worrisome findings was that the drop was most acute among people under 35. “Employees are feeling more disconnected from their employers,” wrote Gallup’s chief scientist for its workplace practice, Jim Harter.

At Starbucks, new CEO Laxman Narasimhan has struck a noticeably more conciliatory tone at Starbucks, and he has begun working one shift a month in a cafe to better understand what baristas face. A spokesman pointed Fortune to a 2022 corporate blog in which the company said it was tweaking policies around issues such as scheduling, in response to employee feedback. But employee discontent still simmers: To date, some 300 of Starbucks’ 9,300 U.S. stores have voted to unionize.

Coming out of the pandemic and emboldened by a very tight labor market, workers are more vocal about getting what they want from employers and pushing back. While that phenomenon itself may feel like old news in the post-COVID era, it is taking increasingly widespread and concrete form. New unionization efforts are racing through restaurants and retailers, ranging from REI, Chipotle Mexican Grill, and Trader Joe’s to Apple stores, Amazon, and, of course, Starbucks. And the phenomenon is not limited to the services sector. Some workers at Tesla, for one, have tried to unionize, while many white-collar workers at the big Wall Street firms initially resisted return-to-the-office diktats last year. 

What’s particularly striking is that so many of these companies are famed for—and loudly proud of—their better-than-average wages and perks. Their leaders frequently tout these benefits and offerings as both a cause and result of strong “employee engagement”—engagement often measured by wide-ranging employee surveys. That in turn leads to moments of Schultz-like surprise when employees turn against bosses who think they’re giving the employees exactly what they want.

Some of this surprise is strategic, of course. But some of it is genuine, and it raises important questions at a tense time for labor relations. Do C-suites know what employees are really thinking? And are they asking the wrong questions when they try to find out?

An epic shift in employee attitudes

Experts say such an estrangement points to a potential danger for companies. You don’t want to be clueless about what employees really want at a time when many can easily find new employment, nor do you want to be ill-equipped to adapt to how those wants are shifting. “There’s a blind spot in many organizations where they think they’re doing well but they’re not,” Gallup’s Harter tells Fortune

But getting an accurate read on worker mood has become ever more challenging since the pandemic, which reset many employees’ expectations. What’s more, many employers are struggling in the remote-work environment to stay on top of what workers are feeling or thinking and to maintain the person-to-person contact needed for strong work relationships. 

As Martine Ferland, CEO of Mercer, the human resources consulting giant, describes it, we have moved through multiple shifts in attitudes about work in the last two decades. Through the 2000s, relationships between employers and workers were largely based on a contract of mutual loyalty. Then came an era anchored in the view that a less tightly bound but mutually beneficial relationship was good enough. In the wake of the pandemic, the dominant focus is on work-lifestyle balance, in which workers are much more in control and more comfortable questioning bosses. What’s more, that outlook—generally associated with Gens Y and Z—is infectious and now changing older workers’ attitudes. “Sentiments shift and change now a lot, and I’m not sure employers are used to that velocity,” says Ferland. 

In the service sector, it’s even more pronounced. For months on end, frontline workers like store employees or flight attendants dealt with abuse from the public and worked long hours because of the labor shortages; now their priorities have shifted, especially among younger workers. “This Gen Z group wants to know why you’re doing what you’re doing, and if you don’t share that with them, they’re going to complain loudly,” says Craig Rowley, a senior client partner at Korn Ferry.

The allure and the danger of employee surveys

Corporate America’s attempts to measure employee attitudes and motivation go back decades. Over the years, one method—the employee engagement survey—emerged as a durable tool. It originated during World War I when the U.S. military sought to measure troop morale and willingness to engage in battle, according to Peter Cappelli, the director of Wharton School’s Center for Human Resources. By the 1930s, such surveys had evolved and been adopted by companies: Sears, for one, used it in efforts to ward off unionization attempts. By the end of the 1950s, the surveys had gone mainstream, and companies used them to take the pulse of worker sentiment on things like benefits and job satisfaction.

By now, tens of millions of U.S. workers get multiple emails every year from employers pleading with them to fill out a survey. As anyone who does so knows, the questions can be pretty general. That’s by design, to ensure comparability over time and with other organizations. But it’s also a recipe for overly generic and frankly unclear questions—and, for managers, misleading results.

For instance, a poorly worded question might ask: “Do you feel management supports your professional development?” That addresses an important topic but doesn’t specify whether it refers to one’s immediate manager, one’s department leader, or top management—let alone the fact that “professional development” is a broad category. A better-phrased version of the question could be: “Do you feel your immediate manager provides you with access to classes or training that you need to learn the new skills you want?”

Multiply this kind of mushiness across a 50-question survey, and you’ve got a major source of leader-staff disconnect. “You could think there was no discontent among your employees because you didn’t see it and didn’t ask the questions that were going to tell you whether they were really happy,” says Cappelli.

Even when the questions are relevant, companies often don’t drill down far enough into the ample data they are sitting on to make the results useful, making surveys seem like a pro forma exercise more than a good faith effort to take the pulse. 

Mercer’s Ferland gives a theoretical example of a company analyzing responses to a classic employee engagement question: “Do you feel secure speaking up?” A typical, and good, result would be 75% of people saying yes. But employers can often look at that data from too high a level for it to be useful, she says. “If you slice it and dice it, you see that women are much lower than that average, and Latin American [people], for instance, even lower,” she says. “You’re missing the whole picture.”

Another practice that seems anodyne but can lead to bad results: the aggregation of responses as companies crunch the data. Gallup’s Harter says that often, to simplify report production and analysis, he will see companies combining the percentages of people answering a question with a 4 out of 5 and 5 out of 5 (lumping them together as, say, “better or much better than average”). That kind of blurring often makes the survey results look much better than they really are. 

Even when data is well collected and analyzed, companies often fail to make full use of it, making annual surveys seem like something HR does just for HR’s needs. To remedy that, Ferland says, companies should complement their big annual questionnaires by adding surveys throughout the year on specific topics and for specific sub-groups, with more messaging on social media to get the pulse. And above all, employers need to put to use the knowledge gleaned. “They need to communicate what they’ll do with the findings. If they are trying to boil the ocean, it’s not useful,” she says. Instead, she recommends that companies “select three priorities, say, and work on them so people understand you’re doing something.”

Of course, there’s always the old-fashioned tool of asking people directly about their needs and concerns. “You could have supervisors actually go talk to people. Employees are usually not shy about telling their direct boss what’s going on,” Wharton’s Cappelli says.

And that is key. If respondents feel that they are asked year after year for detailed feedback but nothing changes, they will stop taking part in “engagement surveys,” in part because they won’t truly be engaged. Indeed, Cappelli says that not taking heed of what employees are communicating increases their alienation. “Benefits can attract people, but it’s not going to keep them,” he says. And unless companies are in the mood for more employee rebellion, it’s in their leaders’ best interest to figure what will.

Subscribe to Well Adjusted, our newsletter full of simple strategies to work smarter and live better, from the Fortune Well team. Sign up today.