The Move to VoE

Posted on Jan 04 in Featured by admin

Organizations like Toyota, Ford, Hewlett-Packard and Toshiba are abandoning traditional employee survey research in favour of Voice of the Employee (VoE). What’s behind the move?

Part of the move is driven by criticism of traditional employee research from the scientific community, that points out that most employee research misuses scientific methods to lend an air of credibility to what is pure pseudo-science. Another reason for the move is the recognition by organizations that traditional employee research methods simply don’t seem to work all that well and that over time, creating  a gap between HR understanding of organizational issues and organizational realities. 

1. Criticism from the Scientific/Statistical Community

A big reason for the move to VoE is attributable to recent, well founded, attacks by the scientific/statistical community on the widespread misuse and overselling of traditional customer survey research. Traditional research based on enumerative methods (such as polling) simply can’t deliver what is promised. The enumerative emperor has no clothes.

Misuse of statistical significance as an indicator or measure of practical  importance.

Traditional customer research typically uses tests of statistical significance to highlight differences among departments, branches,  products, customer segments, time periods or geographical regions. Differences between customer segments, for example, are said to be important if the differences are statistically significant to some confidence  level such as 95% (the infamous 19 times out of 20 statistic). Management is expected to pay attention and to respond to these statistically significant results.

Of course, statistical significance has nothing to to do with importance and is of little economic or scientific relevance. Unfortunately, that hasn’t stopped some  from selling the fiction that it does. Recently, a detailed and brilliant attack from the scientific community has come from authors Stephen T. Ziliak and Deirdre N. McCloskey  in their highly recommended: The Cult of Statistical Significance: How the Standard Error Cost Us Jobs, Justice and Lives. The title says it all. Organizations that use statistical significance as a measure of importance are literally flaying at windmills and destroying their relationship with customers in the process.

Fortunately, the efforts in the scientific/statistical community are having an impact. In realizing the shortcomings of enumerative approaches, businesses are turning to analytic research methodologies such as Voice of the Customer with its sound scientific basis and focus on practical, real world or economic significance, to properly identify what is, and is not, important to customers.

Flagrant overreaching.

The scientific/statistical community is also taking direct aim at many of the claims made by those selling traditional customer research. Among the two most outrageous:

• that  comparisons can be made among companies to produce a ranking of performance (i.e.; 50 best or top 100),
• that statistical models can predict employee loyalty, satisfaction or engagement (i.e.; 12 things that predict loyalty).

Unfortunately, the simple truth is that both claims are well beyond the ability of the statistical sciences. What is being sold here is pure snake-oil.

Corrupt Comaprisons (i.e. 50 Best)

Creating corrupt comparisons, occurs when the results of one company are compared with the results of a group of companies (i.e; 50 best). For example, comparing your employee engagement score with the employee engagement scores for a comparison group. Conclusions are drawn that usually take the form of  a ranking, where one company is said to be higher or lower in the ranking than others. (We are in the highest quartile!). Such conclusions are examples of overreaching and are complete nonsense, but they sell.

Some of the snake oil involved is in having you believe that a single number, your employee engagement score in the current year for example, can be taken as an indicator for organizational performance generally. Comparing your measures to similar measures of comparative organizations compounds the error while promoting the delusion that scientific comparisons are being made. (Such comparisons also make use of statistical significance testing which adds a whole new layer of nonsense to the process.)  This is the scientific equivalent of deducing a trend from a single data point – a mathematical miracle of immaculate conception.

Basing employee and organizational development decisions on such fanciful conclusions is dangerous. It leads the organization to pursuing directions that have no potential for improving the employee relationship while ignoring others that do. The net result being wildly misdirected efforts. Small wonder research by the GAO in the United States has indicated that the least effective customer strategies are built on a foundation of organizational comparisons (see Benchmarking Customer Satisfaction Doesn’t Work).

Meaingless Models (i.e.; twelve factors that predict customer loyalty)

Meaningless models are those that use statistical correlations to predict employee engagement, loyalty or satisfaction. Examples are identifying the ten factors determining employee loyalty or the twelve drivers of employee engagement. If a statistics course was part of your education, there is a good chance you can recall one phrase your instructor pleaded with you to remember: correlation is not causation! It was good advice.

Customer research claiming to determine or predict levels of customer loyalty or satisfaction confuse statistical models of correlation (enumerative models) with models of causation (analytic models). Like shell game artists, they take advantage of the confusion to pretend one thing is another.  Such models of employee loyalty, engagement or satisfaction are as scientific as models linking declines in polar bear populations with decreases in the number of accordion players in Missouri. The two are correlated, but the relationship is a fantasy and the model, meaningless.

Edward Tufte, former Professor of Public Affairs at Princeton and now Professor Emeritus at Yale University, recently coined a word to describe such flagrant overreaching - economisting. (See Beautiful Evidence, Graphic Press LLC, 2006.) Derived from a German root, the word equates to the deliberate act or process of converting limited evidence into grand claims … see also the German meaning of mist. Such claims he notes, are evidence of mediocrity. We couldn’t agree more.

2. The Growth of Analytic Methods in Business

Another driver behind the move to Voice of the Employee research is the rise and acceptance of other analytic research methods in business, especially Lean, Six Sigma and Continuous Improvement. As companies gain experience with these improvement methodologies, their understanding of the differences between enumerative and analytic  methods has grown. With this comes the recognition that traditional employee research, which is enumerative, is simply incapable of answering the questions that are typically put to it.

For example, Lean Six Sigma improvement models call for improvement teams to gather the voice of the customer at some point in the improvement process – usually early on in the improvement model. Some companies equated this call with the traditional customer research being used. As experience with Lean Six Sigma grew, however, these companies began to realize that voice of the customer meant just that –  analytic research methods that properly capture customer requirements embedding them into product and process improvement activity.

At the same time, understanding of the distinction between enumerative and analytic methods has also grown. Organizations are simply doing a better job of appreciating the appropriate application and limitations of each research approach and doing a better job at aligning the research question with the research methodology used to answer it. (For more on the enumerative-analytic distinction in the research science see; VoE: Answering the Important Questions).

The Move to Voice of the Employee

The move to Voice of the Employee then has been prompted by three factors:

1. Strong criticism from the scientific/statistical community of the widespread misrepresentation and overreaching in traditional enumerative employee research. This is evidenced by the misuse of statistical significance and the making of corrupt and meaningless comparisons and models.

2. Growth and acceptance of other analytic methods through Lean, Six Sigma and Continuous Improvement. The use of these techniques has increased awareness of the distinction of enumerative and analytic research methods generally along with an increased appreciation for  the importance of aligning the research methodology with the research question at hand.

3. The continuing trend in moving away from traditional customer research to Voice of the Customer. With this has come the realization that the flaws in traditional customer research mirror those in traditional employee research. Both are enumerative research methodologies and cannot answer the really important questions the business needs answered.  You can’t separate the wheat from the chaff with enumerative methods or measures of statistical significance. It requires the use of real world economic significance and Voice of the Employee.