Consulting Psychology Journal, Vol 78(1), Mar 2026, 1-26; doi:10.1037/cpb0000288
A wide range of instruments is employed within the talent selection pipeline, and, with the progressive evolution of artificial intelligence (AI), the development of AI solutions has made inroads into certain areas of staff selection, notably the processes of reviewing applicant resumes and conducting asynchronous video interviews. However, the use of AI models with machine learning to interpret text-based assessment center (AC) simulation responses has remained largely unexplored. The main aim of this study was to assess the convergent and criterion validity of an AI model. A secondary objective of the study was to see if the AI algorithm utilized more scale points in rating written simulation responses compared to human assessors’, whose scores typically suffer from range restriction. AI was used to analyze the text-based AC simulation outputs of 15,000 leaders, comprising 33 million words, and 38 competencies were discovered. The AI model was then used to score the simulation results of three separate samples of leaders to assess its convergent and criterion validity. The results showed convergent validities ranging from 0.63 to 0.73 and criterion validities ranging from 0.51 to 0.54. In terms of range utilization, the standard deviation indicated that the AI model utilized a wider range of scores than human assessors did. Empirical results presented in this study across three samples suggest that AI algorithms can score written text in a similar way as human raters. Second, the AI algorithms are better at utilizing the full range of scores available. The AI model seems to be on par with the human rater, with accuracy and recall metrics at 0.91, indicating the possibility of augmenting or replacing the human assessor. (PsycInfo Database Record (c) 2026 APA, all rights reserved)