The Evolution of Automation in Talent Assessment and Selection

Brendan Neuman, PhD - Basil Consultants and Maura Burke, PhD - HumRRO

With increasing coverage of artificial intelligence of automated talent assessment appearing in the popular press, it’s no surprise that employers’ expectations are moving toward automated selection processes. That said, there is a limit to what can be effectively and responsibly automated, as the interaction between human behavior and workplace context is messy and difficult to quantify. Although AI and automation can be pragmatic aids to recruit talent, selection decisions are more ambiguous, involve higher stakes, and require a great deal of contextual information. In this article, we explore the state of automation in talent assessment and shed light on its benefits and limitations.

How We Got Here: The State of Recruiting Process Automation

Knowingly or unknowingly, consumers experience automation and customization based on their past choices in their everyday lives. For example, we expect Netflix to know what shows will consume our attention. We rely on Spotify to build the perfect playlist or to recommend our new favorite podcast. However, although the technology and methods might look similar in talent assessment processes, predicting a candidate’s fit with a job and an organization is much more complicated, and the cost of errors even more significant than the examples noted above.

Especially for large organizations, attracting and recruiting candidates presents time and logistical burdens that are ripe for automation. Job postings, applicant data, candidate communications, and administrative tasks all require decisions that are generally small in scope but large in volume. These are the kinds of tasks perfectly suited for recruitment process automation. Applicant Tracking Systems (ATS) represented the first wave of recruitment process automation, and in turn, introduced job seekers to the resume “black hole.” More recently, HR technology has spurred significant process improvements in recruiting, including automation in:

• Interview scheduling
• Managing job descriptions and distributing postings to job boards and social channels
• Sourcing and engaging with passive candidates
• Resume screening and candidate ranking (i.e., topgrading)
• Digital (video) interviewing
• Background and reference checking

Additionally, a tight labor market has led employers to become increasingly sensitive to the candidate experience and to consider every candidate a potential customer. There is a renewed emphasis on employment branding, and responsiveness to candidate feedback for the serious talent competitors. Increasingly, technology is helping to make introductions between candidates and the jobs that they may never have otherwise met. Further, recruitment marketing has become foundational for attracting candidates, and has opened up an entire area of recruitment automation in its own right. Just as customer relationship management (CRM) software like Salesforce enables salespeople to replace time spent on administrative tasks with customer intimacy, recruitment CRMs (e.g., SmashFly, Avature) nurture candidate pipelines, allowing recruiters to spend more time having meaningful conversations with engaged candidates. Evidence of the increased focus on the candidate experience can be found in the Talent Board’s annual Candidate Experience Awards, which reports that over the previous 4 years, job applicants in North America have shown a gradual upwards trend in satisfaction with their recruiting experience.

Automation’s Progress From Attracting to Assessing Talent

Beyond helping attract talent, algorithms and natural language processing tools are helpful in identifying candidates whose work histories, skills, and abilities align with the demands of a job. As recruitment technology providers demonstrate their value to employers, they have continued to offer solutions designed to accelerate and automate hiring further down the recruiting funnel:

• Job boards like Indeed now have in-house industrial-organizational (I-O) psychologists developing assessments, and tech-specific websites like HackerRank combine candidate sourcing and skills assessment in one platform.
• Wade and Wendy augments recruiters with AI to engage, source, and screen undiscovered candidates.
• Whether they build it or buy it, technology companies like IBM and Cornerstone are merging ATSs with traditional skills, personality, and cognitive ability testing.
• Video interviewing platforms such as HireVue and Montage have moved beyond the efficiency gains of digital interviews to offering automated scoring and ranking of candidate submissions. This, in addition to recent mergers such as Shaker International and Montage, will inform how new technologies influence and are influenced by traditional testing and selection concepts (e.g., concepts related to test validation).
• Although firms like Gartner and Forrester have provided research and advisory services on corporate IT solutions for decades, the talent acquisition technology ecosystem has become complex there is an emerging cottage industry of advisors focused on recruiting technology strategy.

One thing many of the existing recruiting technologies have in common is their singular focus on candidates’ skills, abilities, or work histories. What remains to be automated is measurement of the confluence (or interaction) between candidate characteristics and the role, team, and organization into which candidates are hired. Automated talent assessment has added efficiency and productivity to hiring processes; yet, as we describe below, there are challenges beyond sorting and ranking candidates that technology has not fully addressed.

The Importance of Context and Expert Judgment

Important aspects of a candidate’s fit with a role are difficult to quantify, such as the culture of a team, the strategic direction of the organization, or the candidate’s long-term development needs. Increasingly, organizations are working to sharpen and communicate the elements of their employment brand on their career pages and on sites like Glassdoor. Recruiting for culture fit is not controversial, but selecting for it is a little trickier and subject to debate. It is here at the bottom of the recruiting funnel where individual hiring decisions are being made that context and expert judgment matter the most. There is a growing appetite for, and thus the potential for over-reliance on, the use of algorithms in hiring. Unfortunately, it’s not always clear how much is too much when it comes to automation in talent selection.

Morris et al. (2014) point out in their meta-analysis of individual psychological assessment that, from a strictly empirical point of view, statistical integration of a candidate’s assessment data (e.g., personality, interview scores, and cognitive ability) is at least as predictive of job performance as expert human integration of the same information. In other words, a simple linear regression is, on average, as good or better than expert humans at predicting a good hire. Yet, there remain numerous contextual benefits of involving expert assessors alongside algorithms:

• Many jobs do not have sufficiently large samples of incumbents to create a valid prediction algorithm.
• Algorithms don’t necessarily circumvent or eliminate the possibility of bias (e.g., gender, race).
• The use of common interviewers with structured interview questions across candidates provides a common frame of reference and is likely to increase the predictive validity of the hiring process (Campion et al., 1997).
• Assessors provide a human face that can put candidates at ease and answer their questions, communicate an organization’s values, competencies, and recruitment brand to candidates, creating a more personal candidate experience.
• Assessors know the organizations they support, their history, their strategy, and the context of important talent decisions.

The most successful organizations will wisely automate recruitment processes and use intelligent algorithms in conjunction with expert perspective to manage their talent acquisition strategies. As Jon Willford writes in this same Leading Edge Consortium (LEC) series, there is a strong case to be made for human–algorithm collaboration.

The Transition From Assessment to Selection

The precise point when recruiting talent transitions into selecting talent is unique to different jobs and organizations. Specifically, as the importance of a hiring decision increases, the more an organization should invest care and concern in making that decision. On this basis, we recommend employers consider the following when choosing how much of their hiring process to automate:

• The cost of a suboptimal hiring decisions: Replacement recruiting costs, succession planning needs, and business continuity.
• The volume of both candidates and job incumbents: Team productivity and performance may suffer as a result of conducting interviews.
• The level and complexity of the work involved in the job: Greater task complexity is difficult to assess through automation.
• The degree to which the role influences or shapes the organization’s culture: Intangible and unique candidate characteristics are even more important these jobs.

It is not our opinion that organizations (nor I-O psychologists) should disregard the important advances in assessment technology or automation. Automation of menial tasks is valuable; scheduling, logistics, and data management sap the creativity and potential of talent acquisition professionals. However, when it comes to using technology to make decisions on talent, it is a case of and not or.

I-O psychologists provide critical value to our clients by tempering and guiding their expectations for the use of algorithms and by being part of the ongoing discussion of how to use artificial intelligence responsibly (cf. Chamorro-Premuzic et al., 2016). Technology can be convincing—even when it’s wrong. Just as blind adherence to GPS directions can steer us into danger, poorly understood or invalid hiring algorithms have potentially harmful consequences for organizations and candidates. The unique role psychologists have in this space is a comprehensive understanding of how talent assessment relates to the science for a smarter workplace.