Data Fluency Series #3: Variation and Validity

This is the third episode in our series on Data Fluency. You can also watch the first episode or second episode.

If you are using a tool to measure people data, there are two things it has to do:

• It has to reveal actual variation in the real world.
• That variation has to validly relate to something else.

When it comes to performance reviews or surveys, most of the questions that are asked create data with absolutely no variation – everyone will give all fours and fives. If you ask a question like “I trust my coworkers.” You’ll get no variation in responses – everyone will answer “Agree” or “Strongly Agree.” Without variation, that question doesn’t validly measure anything (except maybe someone’s ability to use a computer).

The question must be worded so precisely that it creates range. Asking someone to rate the phrase “I trust my coworkers” won’t do this, but asking someone to rate the phrase “I know my teammates have my back” does. You can only learn this by asking question after question after question, and sorting out the ones that actually create variation in the responses.

Next, you have to learn if the variation you’ve created relates to something valid in the real world. Do the people who answer, “Strongly Agree” to the question “I know my teammates have my back” end up being more productive in the real world? (As it turns out, they do!)

This is called Criterion Related Validity. If a question can measure an increase in employee engagement, it should also be able to predict an increase somewhere else – something like productivity, or retention. Not only should it reliably and validly measure one thing, but we need that to predict something else to make the tool effective.

A good HR tool will yield valuable data that matters in the real world.

Watch the next Data Fluency Series episode for more information on how to become data fluent, and the impact of bad data on our businesses.

Note: The views expressed on this blog are those of the blog author(s), and not necessarily those of ADP. This blog does not provide legal, financial, accounting, or tax advice. The content on this blog is “as is” and carries no warranties. ADP does not warrant or guarantee the accuracy, reliability, and completeness of the content on this blog.

ADP, the ADP logo and the ADP Research Institute are trademarks of ADP, Inc. All other marks are the property of their respective owners. Copyright © 2020 ADP, Inc.