Are predictions based on situational judgement tests precise enough for feedback in leadership development?

Abstract

Situational judgement tests (SJTs) have much to recommend their use for personnel selection, but partly because of their low reliability, the role of SJTs in behavioural training is largely unexplored. However, research showing that SJTs cannot measure homogenous constructs very well is based exclusively on internal analyses, for example, alpha reliability and factor analysis. In this study, we investigated whether patterns of correlations with external criteria could be used to show that SJT dimension scores are homogenous enough for feedback purposes in leadership development. A multidimensional SJT was designed for 268 high-potential leaders on a development programme and used in conjunction with a multisource feedback instrument that measured the same competency framework. The SJT was criterion-keyed against the multisource feedback instrument using an N-fold cross validation strategy. Convergent and divergent correlations between the SJT scores and corresponding multisource dimension scores suggested that SJT scores can be constructed in a way that permits dimension-level feedback that would be useful in leadership development.