Data, Code From Third Annual SIOP Machine Learning Competition Now Available

Code and data from the winning solutions of the third annual SIOP Machine Learning Competition was recently released. Winning teams were announced at the 2021 SIOP Annual Conference.

This year’s competition, engineered and designed by SIOP members Nick Koenig, PhD, and Isaac Thompson, PhD, and principal data scientists at Modern Hire, the SIOP Machine Learning Competition enables participants to transform hiring for the better. The goal of this year's competition was to leverage the latest machine learning techniques to develop an algorithm that predicts employee performance and encourages diversity of qualified candidates. Using a massive open data set from a Fortune 100 company, participants were given a real-world challenge that is extremely critical in today's hiring environment.

"Getting access to such a rich real-world data set of pre-employment and post-employment data was a strong foundation for this year's competition. We are not only open-sourcing the data, which creates an organic and evolving benchmark for innovation of the most fair and valid methods, but also the winning teams' solutions," said Thompson. "The interesting results spur a novel yet informed conversation around approaches to fairness that have maybe not been considered or should be reconsidered."

More than 200 teams from across industry and academia entered and submitted over 1,500 models to this year's 2021 SIOP Machine Learning Competition. Approximately 600 individuals participated, 60% for whom it was their first ever machine learning competition. The winners of the 2021 competition, which were announced in April, are:

• First place: Feng Guo and Samuel T. McAbee from Bowling Green State University.
• Second place: The Axiom Consulting Partners team, composed of Ian Burke and Ashlyn Lowe from Axios Consulting Partners, Goran Kuljanin from DePaul University, and Robin Burke from the University of Colorado, Boulder.
• Third place: Brain Costello and Willy Hardy from Red Hat.
• Fourth place: The Colorado State University team, composed of Joshua Prasad, Steven Raymer, Kelly Cave, and Shayln Stevens of Colorado State University and Jason Grant Prasad from the Georgia Institute of Technology.

The competition was hosted on EvalAI, an open-source AI challenge platform. Because all algorithms were written in open source and fully reproducible code, any company will be able to leverage the created algorithms to make more fair and accurate hiring decisions. All algorithms and data have been made available openly on GitHub and can be accessed here. Similarly, a recording of the presentation can be found on SIOP's official YouTube channel.