The Bridge: Connecting Science and Practice: Leader Edge: Applying Science to Enhance Leader Performance and Diversity
Jeff W. Johnson and Sharon Arad
The Bridge: Connecting Science and Practice
“The Bridge: Connecting Science and Practice” is a TIP column that seeks to help facilitate additional learning and knowledge transfer to encourage sound, evidence-based practice. It can provide academics with an opportunity to discuss the potential and/or realized practical implications of their research as well as learn about cutting edge practice issues or questions that could inform new research programs or studies. For practitioners, it provides opportunities to learn about the latest research findings that could prompt new techniques, solutions, or services that would benefit the external client community. It also provides practitioners with an opportunity to highlight key practice issues, challenges, trends, and so forth that may benefit from additional research. In this issue, we explore how to apply science to enhance leader performance and diversity with Jeff W. Johnson and Sharon Arad of SHL.
Column Editors: Kimberly Acree Adams, Independent Consultant and Stephanie Zajac, Houston Methodist Hospital
Leader Edge: Applying Science to Enhance Leader Performance and Diversity
Jeff W. Johnson and Sharon Arad
SHL
Overview of SHL
We at SHL were honored to receive SIOP’s M. Scott Myers Award for Applied Research in the Workplace in 2018 for research supporting the development and validation of our Leader Edge selection, development, and succession management solution. Leader Edge helps companies articulate and prioritize the challenges leaders will likely face in their organization, and assesses a leader’s fit to role given those challenges.
SHL is the global leader in talent innovation, helping companies transform productivity through deeper people insight. Powering the future of business, our data and tools are proven to drive stronger, more sustainable business outcomes. Our assessment science, benchmark data, and analytics empower leaders and their teams to make confident, data-driven people decisions, when it matters most—transforming the way organizations recruit, manage, and develop talent. With 40 years of talent expertise, SHL is a trusted technology partner to more than 10,000 companies worldwide. We work with companies of all sizes in every industry across more than 150 countries, including 50% of the Fortune Global 500 and 80% of the FTSE 100. SHL employs over 300 I-O psychologists around the world.
Limitations of Existing Science for Matching Diverse Leader Profiles to Different Work Environments
Because of the increasing complexity of the work environment, demands on leaders have expanded in recent years. As the demands on leaders mount, leader performance suffers. Research indicates that nearly half of leaders who moved into new roles fail to meet their objectives and two-thirds are not adapting quickly enough to meet their business and strategic goals (Gartner, 2012; 2016). We observed that typical leadership programs tend to be predicated on a belief that a stable set of leadership competencies will enable leaders to become “agile” and perform effectively in any leadership role. Most organizations capture and communicate these competencies through leadership models, which identify the attributes that serve as the foundation for managing their leader talent. Our research suggested, however, that most successful leaders excel in a few specific areas rather than being effective across the board.
The variety of potentially successful leader attribute profiles is consistent with research showing that personality validity tends to be situationally specific, meaning the situation influences the relationship between personality traits and job performance. Meta-analyses show very wide credibility intervals around mean validity coefficients for personality traits (Tett & Christiansen, 2007) and major personality models recognize the importance of the situation as a moderator of the relationship between personality and job performance (e.g., Barrick, Mitchell, & Stewart, 2003; Johnson & Schneider, 2013; Tett & Burnett, 2003). These models agree that a personality trait will be most highly related to behavior when the situation is relevant to the trait’s expression and is not so strong that there is little opportunity for variance in behavior.
We can make general statements about moderators of personality–performance validity (e.g., autonomy, occupation), but it is very difficult to make specific predictions about how the many different aspects of a leader’s work environment may affect how his or her personality influences performance. This presents a problem for translating the knowledge we have about the situational specificity of personality validity into a practical application that enhances the predictive ability of personality assessments for leaders. Understanding how different work contexts affect personality–performance relationships requires both theory and extensive data to see the empirical relationships that exist in different contexts.
Context can be thought of as any relatively durable aspect of the work environment that could influence the occurrence of behavior in an organization or the relationships between variables, like certain job requirements, team composition, or organizational climate. To better understand the influence of context on the relationship between a leader’s individual characteristics and how he or she performs on the job, we conducted a large-scale study across dozens of organizations. We expected that the prediction of leader performance would be enhanced by incorporating context because different leader characteristics should be relevant to performance in different contexts. The goals of this research were to (a) measure a wide variety of specific contexts that have the potential to moderate personality–performance relationships, (b) demonstrate differential prediction of performance from personality scales based on context, (c) demonstrate improved prediction of performance and increased diversity among potential leaders in terms of leader attributes and demographic characteristics when taking context into account, and (d) create a leader selection and development product that allows clients to tailor solutions to the configuration of contexts that fits the work environment of a particular role.
Method and Results
Between 2014 and 2016, SHL conducted the largest validation study of its type to define a taxonomy of organizational context factors and investigate its usefulness in understanding leader performance. The study included nearly 8,700 leaders, 5,900 supervisors, and over 33,000 direct reports from 85 companies representing more than 25 industries globally. Data were collected from leaders at all levels of the organization—from front-line managers to chief executives—on their personalities, work experiences, opinions, and work priorities. To measure personality, leaders completed the Occupational Personality Questionnaire (OPQ), a 32-scale forced-choice measure that uses item response theory to estimate trait levels on each scale. Leader performance was measured with a multisource performance rating instrument completed by each leader’s supervisor and direct reports.
All participants also provided data that were used to define the leader’s broad work context. For example, leaders completed a job analysis questionnaire to identify the most important aspects of their unique roles. Supervisors completed an opinion survey measuring business priorities and different aspects of the organizational culture. Direct reports completed an opinion survey measuring team functioning and characteristics. We created numerous context variables from these data that describe the unique work environment for any particular leader at the role, team, and organization level. Role-level challenges include aspects of the leader’s job that often differ from role to role (e.g., the extent to which designing and driving new strategies is important to the job). Team-level challenges include the dynamics and makeup of the team, such as the need to transform a team with a high-conflict culture. Organization-level challenges include the business priorities and culture of the organization (e.g., the extent to which growing the business through innovation is a priority).
We used moderated multiple regression to identify context variables that moderate the relationship between OPQ scales and leader performance. For example, we found that the OPQ scale Independent Minded predicts overall performance in opposite directions depending on the level of importance placed on creating an environment that consistently yields creative and innovative ideas, products, or services from team members. When driving creativity is important, more independent-minded leaders tend to be seen as better performers. When driving creativity is less important, going along with the crowd tends to lead to perceptions of better performance.
For context variables that had significant moderating effects on multiple scales, we conducted further analyses by computing within-context correlations. For context variables that had multiple OPQ scales with high within-context correlations with performance compared to the overall sample and unique patterns of predictors compared to other challenges, we identified the unit-weighted composite of OPQ scales that best predicted overall performance. In sum, we found that (a) personality scales vary widely in their ability to predict leader performance in different contexts, (b) the same scale can have a positive relationship with performance in one context and no relationship or a negative relationship with performance in another context, and (c) prediction is dramatically higher when personality scales are selected for the context. Predicting leader performance within contexts yielded three times better prediction on average than was possible when we did not incorporate context.
This is the first large-sample research that has documented multiple specific contexts that influence personality assessment validity and in what way. In the real world, however, leaders rarely operate within a single context, and this has been a major impediment to context research (Johns, 2006). Indeed, we found that leaders in our study had an average of 6.8 contexts operating simultaneously, with 80% having between 3 and 11. It would be impossible to directly compute the validity of an OPQ composite that is based on multiple challenges because selecting a sample of leaders with the same configuration of challenges would result in extremely small sample sizes. This presented a significant issue in creating a realistic practical application from this research. To address this issue, we developed an innovative application that uses the logic of synthetic validation to estimate validity while considering multiple contexts simultaneously. Rather than breaking a job down into its relevant job components and using component-level validities to estimate validity for the whole job (Johnson et al., 2010), we break the leader role down into its relevant contexts and use within-context validities to estimate validity for all contexts simultaneously. Not only are we able to avoid the problem of considering context effects in isolation, we found that validity is improved when the job is described by multiple contexts. This innovative process allows us to consider multiple contexts simultaneously, allowing a single algorithm to be applied to the leader role, which is described by all relevant contexts.
Leader Edge
Based on this research, we were able to create Leader Edge, an application that uses a data-driven approach to automatically match leaders to the contextual challenges of the role. The development of Leader Edge followed an extensive and rigorous market and pilot testing process. We engaged with more than 100 current and potential customers through an iterative process, testing all product aspects ranging from value proposition and positioning to product concept and implementation. Throughout the market testing phases, we learned and evolved the product. Based on the market testing, we fine tuned the positioning and value proposition, obtained feedback on which product features were most appealing and differentiated, identified use cases (e.g., placement, development), and gained feedback on the most valuable reporting features.
We pilot tested Leader Edge with four clients who have used Leader Edge insights to inform multiple leadership decisions, including identification of senior management potential, placement into critical leadership roles, and development of leaders. One of the pilot clients was a US staffing company. This client was looking for tools to enhance its senior management development program. Through its contextual lens, the Leader Edge solution complemented other tools being used and provided a more targeted approach, fine tuning development plans to the specific context in which leaders were operating. Today, other clients are using Leader Edge as objective, valid input into succession management, as well as providing a more tailored experiential development roadmap to prepare leaders to successfully handle critical contextual challenges they will encounter. For example, one client described why and how they are using Leader Edge for leader development and succession management:
We looked at best practices and at what other companies were doing to solve the problem—to take a subjective process and make it data-driven and actionable. And we needed a business case: the financial drivers and business rationale for finding talent and developing people for their next career moves. The most critical part of the new system is that it’s contextual. We look at the six most important challenges someone will face in a new role and compare them to candidates’ skills and competencies, motivations, and runways. We can then focus on what’s needed for a successful transition. We’ve shifted from a gut-driven process to a shared language. (“When Hiring Execs, Context Matters Most,” 2017, p. 21)
Beyond the increase in predictive power, one additional benefit of a contextual approach to leader selection is its ability to improve leader diversity. Our data show that there is no consistent advantage to any ethnic or gender group across OPQ scales. We do find that women and African Americans tend to score higher than men or Caucasians on many of the context-specific solutions in Leader Edge. Thus, Leader Edge not only does not show adverse impact against protected groups, its use is likely to promote greater representation of underrepresented groups in higher levels of leadership.
Understanding that different types of individuals can be successful depending on the context opens the door to considering a more diverse set of candidates when making leader selection and development decisions. Traditional leadership strategies assume that the same characteristics and competencies are needed throughout the organization, but focusing on a generic competency profile diverts attention away from individuals who possess diverse experiences, perspectives, and backgrounds. At the extreme, this practice inadvertently reinforces bias in decisions and results in leadership teams composed of people who sound and look the same. Shifting the focus to context-specific prediction not only optimizes the fit between leaders and their work environment to produce better performance but also increases the possibility that more diverse leader profiles will be considered for key positions.
Conclusion
Leader Edge translates the knowledge we have about the situational specificity of personality validity into a practical application that enhances the predictive ability of personality assessments. Leader Edge provides an interactive platform for organizations to implement data-driven decisions for improved leader selection, placement, and development. Algorithms that power the solution enable users to optimize leadership assessment for the unique combination of challenges present for targeted leadership positions. Challenges can be selected for one particular position, a set of similar positions, or an anticipated future state. As such, in addition to selection decisions, organizations are using Leader Edge to better target leader development and build adaptive leader pipelines. Leader Edge is different from any other leader assessment system currently on the market because it moves away from a one-size-fits-all, static assessment to a customizable, adaptive talent management tool that can support operational HR objectives as well as long-range business strategies.
References
Barrick, M. R., Mitchell, T. R., & Stewart, G. L. (2003). Situational and motivational influences on trait-behavior relationships. In M. R. Barrick & A. M. Ryan (Eds.), Personality and work: Reconsidering the role of personality in organizations (pp. 60-82). San Francisco, CA: Jossey-Bass.
Gartner. (2012). High-impact leadership transitions research report. Retrieved from https://docplayer.net/4194979-High-impact-leadership-transitions-a-transformative-approach.html
Gartner. (2016). Q4 executive guidance: Driving performance in volatile markets. Retrieved from https://www.cebglobal.com/content/dam/cebglobal/us/EN/top-insights/executive-guidance/pdfs/eg2017ann-driving-performance-in-volatile-markets.pdf
When Hiring Execs, Context Matters Most. (2017, September-October). Harvard Business Review, pp. 20-22.
Johns, G. (2006). The essential impact of context on organizational behavior. Academy of Management Review, 31, 386-408.
Johnson, J. W., & Schneider, R. J. (2013). Advancing our understanding of processes in personality-performance relationships. In N. Christiansen & R. Tett (Eds.), Handbook of personality at work (Ch. 3, pp. 30-52). New York, NY: Routledge.
Johnson, J. W., Steel, P., Scherbaum, C. A., Hoffman, C. C., Jeanneret, P. R., & Foster, J. (2010). Validation is like motor oil: Synthetic is better. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 305-328.
Tett, R. P., & Burnett, D. B. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88, 500-517.
Tett, R. P., & Christiansen, N. D. (2007). Personality tests at the crossroads: A response to Morgeson, Campion, Dipboye, Hollenbeck, Murphy, and Schmitt (2007). Personnel Psychology, 60, 967-993.
