'Figuring out what they're after': a common thread between assessment performance and job performance?

A while back we shared a review of the Ability To Identify Criteria (ATIC), suggesting that difference in how people perform on a selection process like an interview is due partly how good they are at figuring out what the process wants to hear. The article suggested that this may not be entirely bad, as ATIC appears to have a role in job performance as well. Now the authors have published empirical work looking closer at this issue. Their data suggests that figuring out situational demands may have a very substantial hand in both selection and job performance, and may even be the major link between the two.

First author Anne Jansen and colleagues (principally from University of Zürich) recruited 124 participants into a simulated assessment process, pitched as a way to give them experience of job selection. Participants were incentivised to do well, with the top two candidates each day financially rewarded, and had to pay a small fee to enter the process. This encouraged motivated participation that was more in line with real selection experiences. Participants were informed of the job description ahead of time, and on assessment day, turned up in groups of 12 to undertake interviews, a cognitive test, presentations and group discussions, observed by multiple assessors (Occupational Psychology MSc students).

After each exercise, participants were asked to document their hunch of what dimensions it was trying to measure; this was compared to answers given by the assessors beforehand, with close matches leading to higher situational demand/ATIC scores. No such information was explicitly provided (otherwise ATIC becomes redundant) so participants had to rely on indirect cues, such as the job descriptions, reading between the lines of instructions, being sensitive to what assessors seemed to be attuned to. In addition, each participant gave authorisation for their real-work supervisors to be contacted online to give feedback on their real job performance; in total, 107 responded.

Overall assessment centre scores correlated with job performance, with a relationship of .21. Both AC scores and job performance also correlated with the ATIC scores for participants: someone who was savvy in figuring out what the AC asked of them did better in the AC, and also did better in the workplace. Jansen's team constructed a statistical model in which cognitive ability fed ATIC, which itself strongly contributed to performance on assessments and in the workplace. Once all of these factors were accounted for, assessment performance itself was no predictor of workplace performance. This suggests, at the least, that ATIC and the factors that sit behind it are a substantial underpinning of how assessments adequately predict workplace performance.

One way to look at this is the growing identification of 'just another factor': IQ, EI, resilience, practical intelligence - that researchers argue counts in the workplace. But actually, this line of research advocates a shift in perspective. It asks us to accept that performance doesn't just depend on the resources you bring to the job, but to your perception of what the job is. This interactionist perspective is less concerned with raw capability and more about orientation. And it raises new considerations: in jobs where orientation is clear-cut - four duties, get on with it - shouldn't we be minimising it in selection? Whereas at the other extreme, could applicants for jobs with high ambiguity be tasked with finding their own way through the application process?

Jansen A, Melchers KG, Lievens F, Kleinmann M, Brändli M, Fraefel L, & König CJ (2013). Situation assessment as an ignored factor in the behavioral consistency paradigm underlying the validity of personnel selection procedures. The Journal of applied psychology, 98 (2), 326-41 PMID: 23244223

Further reading: The original review is

Kleinmann, M., Ingold, P., Lievens, F., Jansen, A., Melchers, K., & Konig, C. (2011). A different look at why selection procedures work: The role of candidates' ability to identify criteria Organizational Psychology Review, 1 (2), 128-146 DOI: 10.1177/2041386610387000