SIOP Releases Recommendations for AI-Based Assessments

In January, SIOP released recommendations for artificial intelligence-based assessments.

Considerations and Recommendations for the Validation and Use of AI-Based Assessments for Employee Selection can now be found on the SIOP website. This document supplements SIOP’s statement on the use of AI for hiring, which was created by the SIOP Task Force on AI-Based Personnel Assessment and Prediction in spring 2022.

“There has been a growing concern about the use of AI in employment decision making, and both federal and state governments are enacting laws to address many of these concerns,” said Task Force Chair Chris Nye. “Many SIOP members have expertise in areas related to employee selection, assessment, and the use of artificial intelligence in the workplace. Therefore, we wanted to leverage this expertise to provide additional guidance, backed by scientific research, on how AI-based assessments can be used effectively and legally in employee selection contexts.”

AI is changing the way that organizations assess and hire talent. These changes are happening rapidly yet with little guidance about how to effectively validate, implement, interpret, and use scores produced from AI-based assessments in this context. The purpose of the new document is to provide scientifically based recommendations for the effective use of AI for assessing and hiring talent.

A key theme that emerges in the recommendations is that AI-based assessments used to make hiring and promotion decisions require the same level of scrutiny and should meet the same standards that traditional employment tests have been subjected to for decades. However, the way that these standards are evaluated and met may be unique to AI-based assessments. The recommendations discuss the unique challenges and considerations that arise in the development, evaluation, use, and interpretation of AI-based assessments.

These recommendations are discussed in detail in five sections:

• Section 1. AI-Based Assessments Should Produce Scores that Predict Future Job Performance or Other Relevant Outcomes Accurately
• Section 2. AI-Based Assessments Should Produce Consistent Scores that Reflect Job-Related Characteristics (e.g., upon re-assessment)
• Section 3. AI-Based Assessments Should Produce Scores that are Considered Fair and Unbiased
• Section 4. Operational Considerations and Appropriate Use of AI-Based Assessments for Hiring
• Section 5. All Steps and Decisions Relating to the Development and Scoring of AI-Based Assessments Should be Documented for Verification and Auditing.

“These recommendations build on previous standards and best practices but discuss the nuances involved with the use of AI-based assessments,” Nye said. “We believe that these recommendations will be useful for anyone who is currently using or is planning to use AI-based assessments in their organization.”

The SIOP Task Force on AI-Based Personnel Assessment and Prediction launched in fall 2021 in response to the increasing use of AI in employee selection and promotion. Members of the task force have met with U.S. Equal Employment Opportunity Commission (EEOC) Commissioner Keith Sonderling twice to discuss AI in the workplace. With members who are experts on topics such as hiring and talent management, employee assessment, measurement bias, and the use of AI-based technologies in the workplace, SIOP is uniquely suited to provide guidance and leadership on the issues related to the development and implementation of AI/ML-based assessments that are used for employment decision making.

Additional reading:

• SIOP Issues Statement on AI-Based Personnel and Assessment Prediction
• SIOP Task Force on AI-Based Personnel and Assessment Meets With EEOC