The skirmish began when The American Society for Quality, Journal for Quality and Participation published, “Understanding Employee Engagement and Trust; The New Math of Engagement Surveys” by Robert Gerst of Converge Consulting Group. It concluded:

“The dirty little secret of employee engagement surveys is that they’re largely junk science—placing the marketing objective of telling and selling a good story above the practical and ethical objective of telling the truth. Like the South Pacific cargo-cults that built airplane engines out of bamboo and radio headsets from coconuts, employee surveys are dressed to look like science but lack its substance.”

Understanding Employee Engagement and Trust detailed four ways in which employee feedback is corrupted: (i) using statistical significance to identify important findings, (ii) employing statistical models like regression to identify factors driving engagement, (iii) ranking results (i.e.; best companies to work for), and (iv) assuming complex concepts like engagement can be reduced to a single numerical index.

Positive reaction came from Quality professionals and statisticians, who have long known employee engagement survey analysis to be junk science. Even the popular press took notice. Maclean’s magazine asked the question: “Employee engagement surveys: useless or very useless?”

But the big fight began in reaction to Robert’s interview in HR Executive Magazine. In, “What Employee Engagement Surveys Really Tell Us“, employee engagement survey providers revealed an awe inspiring and humorous ignorance of basic performance measurement.

Mark Royal, senior principal at Philadelphia-based Hay Group Insight, stated that employee engagement wasn’t any more complex than “emotional intelligence, leadership effectiveness or personality”. Well, we certainly agree with him on that. And just how successful has the Hay Group been at reducing ‘personality’ to a single number? We’re guessing not so much.

Adam Zuckerman, global practice leader for employee surveys with New York-based Towers Watson, noted that while employee engagement is a complex concept, the idea that it can’t be measured and reduced to single number is “patently absurd”. Really? The folks at Towers Watson need to do some reading. We suggest “How to Lie With Statistics” by Darrell Huff.

Then again, science fiction might be more appropriate. Douglas Adams mocked Mr. Zuckerman’s and Mr. Royal’s view of the world in “The Hitchhikers Guide to the Galaxy“, in which the super computer ‘deep thought’ provides the answer to the ultimate question of, “What is the meaning of life, the universe and everything?” as “42“. Now that’s an absurdity. Anyone so gullible to believe that employee engagement (or the meaning of life or personality) can be reduced to a single number, understands very little about measurement, employee engagement, or life.

That’s because, as everyone outside of the employee engagement cargo cult understands, there are some rather substantial error terms in reducing complex concepts like the meaning of life or employee engagement to a number. In measurement science, this is called validity. Or, with employee engagement, the lack of it. There is so little validity in employee engagement indexes, that employee engagement models must remove the error terms just to get their numbers to add up. Without pulling this statistical fast one, no one would buy this CRAP. (The statistical term for employee engagement analysis–Correlation and Regression Analysis with P-values).

Meanwhile, the folks at Gallop also chimed in. Aon/Hewett has long provided one of the dumbest claims from the land of the employee engagement cargo-cult that, “20% of the organization’s employees create 80% of the value“. If true, we asked, why not fire the remaining 80% of employees? We meant it as a joke. Gallop took it seriously.

In “Five Steps to Firing a Manager With a Disengaged Workgroup“, the latest edition of the Gallop Business Journal recommends firing managers repeatedly scoring in the lowest quartile of the Gallop employee engagement metric. For example, applied to the executive team, each year would bring the obligatory firing of a senior vice-president for ‘bad engagement’. Knowing upon which side their bread is buttered, Gallop probably has an executive team exemption. We are not experts in employment law, but firing people because Gallop can’t do arithmetic strikes us as fertile ground for a lawsuit.

No sooner had the Gallop article appeared, when Leadership IQ released a study confirming the conclusions in, “Understanding Employee Engagement and Trust”–-that there’s no relationship between performance and engagement scores. The results were picked up by major media outlets such as The New York Times.

Rather than reaching the obvious conclusion that engagement scores have no validity, Leadership IQ concluded that this was true for all other employee engagement surveys, except their own. Gallop took exception to this, saying that it’s Leadership IQ’s engagement measures that are “less than optimum”. Leadership IQ fired back by calling Gallop’s criticism “despicable” and “shameful.”

It’s getting nasty out there. The only thing this sniping at one another proves is that the emperors of the employee engagement junk science cargo cult have no clothes.

Done right, engagement surveys are a good thing, producing valuable information that can, and should, be used to build better, more productive, workplaces. But that doesn’t happen when the information is corrupted by junk-science. People, and businesses, deserve better.

Voice of the Employee (VoE)TM by Converge, like real science, is open and transparent. No black boxes, proprietary methods, or secret decoder rings. We even publish a Do It Yourself Guide to conducting your own employee engagement survey. It includes some basic analytic procedures that will keep you from falling into the junk science cargo-cult quagmire of:

• using tests of statistical significance (p-values) to identify areas of interest or importance,
• using statistical models to identify engagement drivers contributing to engagement,
• using four, five or six point scales, especially when combined with calculating top-box scores,
• reducing something as complex as engagement down to a single number or index, and
• ranking engagement indexes (i.e.; best companies to work for).

Of course, we would like you to hire us to implement a VoE program. But if you can’t, the Do It Yourself Guide should help keep you on track to delivering ethical employee engagement survey research.

Let’s build a better workplace.