I-O psychologists rely on a variety of preemployment assessments to get an in-depth picture of job candidates, which organizations use to determine whom to hire. If those assessments were not completed by the candidate, but rather an AI (like ChatGPT or Claude), would you still hire them? Already, at the push of a button, “generative AI” based tools and services automatically search for jobs, create polished resumes, and write tailored cover letters. SIOP President-Elect Richard Landers showed that users can prompt ChatGPT to offer advice on how to respond to popular personality assessments. I-O psychologists must be prepared for a world where AI is an integrated part of a candidate’s application, and candidates find themselves at a disadvantage if they do not use AI.

To highlight the urgency that I-O psychology professionals must adapt to a world where AI use is prevalent in the preemployment process, this year’s SIOP Machine Learning competition seeks to examine how vulnerable common preemployment assessments are to being gamed by AI. Teams are invited to develop AI “candidates” capable of presenting themselves as ideal job applicants across various assessments, including personality tests, cognitive ability exams, and interview questions.

The results of the competition are intended to offer insights that can inform assessment developers how to adapt.

“This competition serves as a dynamic benchmark, evolving with technological advancements to proactively guide I-O psychology through the impact of generative AI on assessment,” said Isaac Thompson, co-chair of the competition.

Drawing parallels to cybersecurity where systems are tested and improved by identifying critical weak points in simulated attacks, Co-Chair Sebastian Marin said, “We want to ‘red team’ talent assessment tools with AI to expose their vulnerabilities.”

The competition results could also highlight how AI benefits applicants, if it can ace assessments. By providing pre-assessment training and feedback, AI has the potential to help a broader range of candidates perform at the highest level on assessed competencies necessary for the job, and achieve their career goals.

Co-Chair Ivan Hernandez said, “Equitable access to AI that enhances responses to valid job-related assessments could help reduce selection gaps from structural inequities.”

Running from mid-February to mid-March 2025, key features of the competition include:

  • open-source data built specifically for this challenge,
  • opportunity to work with cutting-edge AI technologies,
  • addressing real-world concerns about AI’s impact on hiring practices, and
  • recognition for top-performing teams.

The competition is open to all, regardless of prior machine learning experience, encouraging a diverse range of participants from academia and industry.

Interested teams should register before February 15 via this form.

This competition is brought to you by Ivan Hernandez, Isaac Thompson, Sebastian Marin, and Briana Squires

Post Type

Source

Topic

2025 Annual Conference, Artificial Intelligence (AI), Events & Education, Machine Learning