In academia, we are instructed to both lead and be led by the data that we are analyzing. This framework is sometimes touted as “art and science” in analytical domains. While this premise tends to work well in concept, we’ve seen several case studies in HR where getting the wrong balance can be sub-optimal or even debilitating. Why? There actually are quite a few factors that contribute to this dynamic in HR analytics, but we will focus on four of the most common ones here and the questions that influence just how much art is, and should be, applied:

Let’s explore each of these factors a bit further below:

Human behavior informs expected patterns

Factors are often not independent because there are both human and organizational architypes that dictate how some behaviors are interrelated at a minimum. A classic example here is the “competitive” organization, where individuals are pitted against each other and the “best” person theoretically achieves greater success. While some organizations are more transparent than others in using this workforce model, the underlying outcome is the same: Those who thrive on competition and traditional hierarchical and/or monetary achievement gravitate to these organizations if and until they either voluntarily or involuntarily get extracted from the competition. Given that there are several such human and organizational architypes, it is important to appreciate the underlying model or models at play when designing and reviewing analytics to validate both statistical and behavioral implications.

Key factors may be unavailable or excluded from the analysis

For many HR analytics teams, access to broader business, operational and market factors — or even relevant HR factors — influencing an analytical model may be problematic for a myriad of reasons, ranging from data access limitations to data quality and/or privacy concerns. Without these factors, the data may lead us to a dead end at best or, worse yet, a spurious result. As an example, we’ll share a personal story — with the acknowledgment that it is based on anecdotal evidence. When we ask for the top factors in vendor selection to support workforce analytics and planning efforts, the first response often cited is the “relationship” that a firm has with an individual or organization. Yet, organizations today debate whether to include relationship-oriented data and analytic methods (e.g., organizational network analysis) in evaluating workforce outcomes or to support longer-term workforce planning. It isn’t entirely surprising to see these data excluded because: (1) we often need to create a proxy variable or introduce another analytic method to quantify a measure of “relationship value”, (2) that proxy may incorporate data that others discourage or do not permit the HR analytics team to collect for various reasons and (3) the threshold for gaining acceptance for the methodology may be unduly stringent. These factors, and more, may limit or eliminate the efficacy of measuring this type of variable, albeit meaningful at its core. Given the potential value of specific variables in explaining an outcome, careful consideration should be given to currently available data and proxies, as well as to foundational work that can be done to prepare and add potential variables and methods to future analyses.

Too many factors are covered by the analysis

This pattern is almost the inverse of “excluded” data elements, whereby so much information is included that the results generate a potentially arcane outcome that isn’t fundamentally actionable (not to mention traditional concerns about adding factors that don’t materially improve the model itself). While we could debate the veracity of the outcome, the more likely response from these efforts tends to include words like “interesting” and “fascinating” (as we saw in Leonard’s case in our first blog post). While these statements may seem like positive acknowledgments, they need to be accompanied by corresponding action-oriented references — e.g., what will you do with the findings? Managing what variables are ultimately selected is often a balance between statistical power and “actionability” or relevance. For example, age, tenure and/or pay level are almost always found to be statistically meaningful variables when analyzing turnover. Yet, actionable variables tend to relate more to the organizational structure and the behavior of managers. Deciding what and how many factors are included in an analysis to address a specific question typically requires experience in working with these models across several organizations or at least subsets of an organization to see what patterns are systemic vs. potentially unique to a specific group or groups. While there is value in both types of variables in the ultimate model, experience tends to help those navigating these factors to deliver a more targeted and actionable result.

It takes too long

While “analysis paralysis” can be a real concern, we’ve often also seen an intolerance in, or with, HR when time is needed to manage data, the review process or even insight development and dissemination. In a world where more information is available at our fingertips, HR is expected to have the data and methods already in place to allow for efficient and effective analysis. Rather than bringing together data on an ad hoc basis to address a specific question, HR analytics teams are expected to have a pre-built, reliable analysis dataset that covers most requests. Since these data-management efforts can be daunting, anticipating and influencing which issues will be handled early in an HR analytics team’s evolution is essential, as is managing expectations related to the quantity and quality of data being utilized.

* * * * * * * *

Getting the right balance in both methodology and data management is important in any research effort. However, the unique dynamics of how talent operates within an organizational context requires additional considerations and creates “noise” when establishing and executing workforce analytics. If you’d like to learn more about Merit Analytics Group’s experience and approach to workforce analytics and planning or share other factors that you want to see added to this list, please contact us at info@meritanalyticsgroup.com .