False Friends in Personnel Selection – How Certain Traits Can Lead to the ‘Wrong’ Choice
by Moritz Leistner (2nd semester Master Psychology – Human Performance in Sociotechnical Systems, Dresden University of Technology)
1. Abstract
Personnel psychology has a lot of answers for how the job selection process should be designed. Even though, in reality there is a divide between research and practice and even experienced practitioners fall into the trap of ‘false friends’, meaning attributes that influence our decisions, despite having nothing to do with actual job performance.
This blog examines some of the effects, as well as the reason for the lack of understanding and the difficulties in acquiring knowledge about the effects.
This leads to practical implications for practitioners to increase the quality of personnel selection and how psychology as a science can help with this process.
2. Introduction – The Job, the Personality and You!
Try to imagine the following: Your job requires that you oversee the personnel selection process of a company. You may have heard of some predictors for job performance, like IQ (Schmidt et al., 2016) or certain personality traits, that are known to be good predictors of job performance. But these traits are not the only thing that is taken into consideration. Traits don’t even have to be about personality. More generally a trait is a certain attribute a person has over a long period of time. And sometimes these stable attributes appear to positively affect work quality, even though – in actuality – they don’t. On the other hand, there are also traits that we assume negatively impact our work quality, but actually have no effect at all or even positively contribute. In this blog I tried to find these ‘false friends’ of personnel psychology and have a look at whether or not these traits actually affect the job selection process and how we can avoid those ‘false friends’. But that is easier said than done. To understand why this is a far less understood problem than it may seem, one must understand the principles of publication bias in psychology. To be more specific, publishing a scientific paper in a journal is way easier if you find an effect, than if you don’t or just show that something doesn’t work (Dwan et al., 2008), a problem that is yet to be fully addressed (Lin & Chu, 2017). But those “we didn’t find anything” papers are what we should be interested, if we want to answer this question. This may lead to the assumption that even personnel managers with a lot of experience aren’t fully aware of factors they use to predict job performance on a daily basis, that aren’t valid predictors in the first place.
Some of these factors may not even be conscious. Imagine for example you are looking for a new programmer in your company. Ideally your decision may only be influenced by the job-relevant qualifications the person has, but then a candidate with a promising portfolio comes into your office and – to your disappointment – has no social skills whatsoever. He doesn’t talk, looks at you like he doesn’t want to be there and gives you a generally bad first impression.
What to do? This is hard question, because on paper he may seem extremely competent. You may have good reasons to see this behaviour as a problem, like him not being able to talk to co-workers. But you may have just made a biased decision on something that has no effect whatsoever on his job performance. If you may have ignored this behaviour, given him e.g. an IQ test and hired him on that basis, he would have – maybe – performed splendidly. You wouldn’t know and there is a reason to assume that personal managers may have the same biases and make the same bad choices. And a lot of these traits that we subconsciously perceive and that influence our decisions may have had a good reason to exist throughout our evolutionary history. But are these kinds of judgments still valid predictors in the work environment of the 21. century?
3. How to Personnel – Traits and Predictive Validity
To understand, how the value of these traits can be assessed, one must understand the concept of predictive validity.
Validity means that if we measure something, how sure can we be, that what we measure is what we want to measure. To understand validity, we first need something called reliability. This is a measurement of the accuracy of the testing, regardless of what is being tested. One could for example want to test body height and measure body temperature instead. If the measurement of temperature is accurate and consistent, the testing would have high reliability, but low validity. Reliability therefore is a precondition for validity, but high reliability doesn’t guarantee validity. (Eid, Gollwitzer & Schmitt, 2015)
To understand validity itself the scientific process entails two things: (1) stating a hypothesis and (2) collecting data that can prove or disprove the hypothesis.
If we want to test the validity of intelligence tests, we get a big sample size of people who do the test and then compare the results to things that should correlate with these results (academic success, e.g.) as well as things that should not correlate with the results (e.g. body height or personality) (Schmitt & Gerstenberg, 2014)
This principle can be used to assess whether a trait really predicts job performance or if it is a false friend.
The Beauty is in the Eye of the Beholder
Beauty. Less subjective and more influential than most of us think. A lot of judgement mistakes fall under the umbrella of so called ‘halo effects’, meaning that we assume certain qualities a person doesn’t have based on other qualities that they in fact do have, that – in general – don’t have anything to do with them (Forgas & Laham, 2016).
One of the most famous examples is the “what is beautiful is good” heuristic. Perceivers tend to attribute positive qualities to attractive people and negative qualities to less attractive people [and] as a result, they form more positive first impressions of attractive compared to unattractive targets (Messner, et al, 2011; Dion, 1973; Dion, Walster, & Berscheid, 1972; Langlois, Kalakanis, Rubenstein, Larson, Hallam, & Smoot, 2000). This happens even with experts that should be aware of this phenomenon (Hosoda et al., 2003). What all these studies indicate is that being attractive greatly increases one’s chances of getting a job, across different levels of qualification. (Hosoda et al., 2003)
Fascinatingly – this may be less of a false friend than one may imagine. Attractiveness increases the perceived intelligence and academic results. (Ritts et al. 1992; Clifford, 1975), so on paper (in this case only figuratively since a lot of personnel selection is done by interview), attractive people often appear ‘more qualified’.
There is however reason to believe, that they could actually have a slight edge in qualifications. In 1968 Rosenthal und Jacobson demonstrated the so called ‘Pygmalion-effect’. After telling teachers that randomly chosen students were gifted bloomers (that in reality weren’t gifted at the start of the study and didn’t show significantly more potential than their peers), the teachers changed their behaviour towards the students, leading to actual increases in mental capabilities.
Research indicates that this also works for job performance. If managers expectations regarding subordinate performance are raised, it boosts subordinate performance (Eden, 1992; Goddard, 1985). Something similar could be at work with attractiveness on a bigger social scale.
So, applying the idea of the Pygmalion study, if we perceive and interact with attractive people as if they were smarter, this may lead to us fostering those attributes and attractive people actually becoming smarter and thus more qualified. Seeing this then reinforces the stereotype and creates a cycle of attractiveness fostering. This could mean that attractive people applying for jobs will be on average better qualified.
This theory fits with the results of a study done by Messner et al. (2011) that found that unconscious decisions (based on traits like attractiveness) improved the decision making in personnel selection processes.
If it were true, this would mean that ‘falsely’ using attractiveness as a predictor for job performance actually in turn leads to this predictor becoming more and more valid. Thus, attractiveness would be a false friend for the same level of qualification, but a valid predictor for the level of qualification itself. In the end, more research is needed to support this hypothesis.
Male Manager – Female Secretary?
Even in the 21. century gender biases matter in the job selection process. This makes sense if one imagines an unscripted, not standardized interview. But this bias even occurs when using standardized testing rather than interviews. If applying standardized tests, female students perform worse, although their academic performance is higher, while men perform worse if academic success is the primary deciding factor (McKinney & Miles, 2009). This indicates that standardized tests predict performance in a gender-dependent manner.
Interviews however have a whole lot of different problems. When searching for candidates for high level positions, men have an advantage over women. (Davidson & Burke, 2000; Jawahar & Mattsson, 2005). This could be because more men work in higher positions, and we therefore assume a fit of attributes (correspondence bias) (Gilbert & Malone, 1995).
Big Person – Big Money
When explaining the difference between correlation and causation, global warming and the number of pirates are often used as examples.
If one looks at the correlation it is almost perfect. With dwindling numbers of pirates, the global temperature rises. If one does not understand the difference between correlation and causation, this may lead them to believe that we need more pirates to combat global warming.
This is – however – not the case. In reality, the industrial revolution caused both, increases in global warming and fewer pirates. Piracy was more common in pre-industrial revolution time, before widespread coal burning was available. So, there is a combined factor responsible for both phenomena.
Body height often seems to fall into this category as well, something that may seem to have nothing to do with job success but correlates with it anyways.
However – in recent years the idea of a causal relationship between height and job success (and therefore higher chances in personnel selection) has come into favour.
First you could mention that – at least in males – height is strongly linked to attractiveness. There it is again – what is beautiful is good. (Pisanski and Feinberg 2013; Sear and Marlowe 2009; Sorokowski et al. 2012)
Second, it is reasonable to assume that we learned over the last centuries as a society that hight is associated with socio-economic status. When food was rare, growing tall was a privilege of those who had enough food. Therefore, we may have created what Richard Dawkins called a cultural meme (2006): An idea everybody has that persists over generations. Consequently, individuals from higher socio-economic groups (i.e., the well-off and better educated) are found to be taller than individuals from lower socio-economic groups (Boström, Diderichsen, 1997; Heineck, 2005)
Lastly, Judge & Cable (2004) linked height and self-esteem to what they call “social esteem” (i.e., how tall individuals are perceived and evaluated by others in society, including, for example, customers or employers).
“Self-esteem and social esteem each may then affect both objective and subjective performance (i.e., how, for example, managers evaluate an employee’s performance, which clearly has an impact on career outcomes such as promotions or earnings). Consequently, while they hypothesize that height is positively associated with earnings, they also suggest that height is more strongly related to subjective outcomes than objective outcomes.” (Heineck, 2005)
Tests that Just not Work
A professor of mine once described the Myer-Briggs Type Indicator (MBTI) as “astrology for people with higher education”.
While there is a use for personality tests like the NEO-FFI (Big Five) in personnel selection – meta studies found moderate positive correlations between conscientiousness and job performance (Barrick, Mount & Judge, 2001) – the MBTI has terrible results in validity and retest-reliability.
Boyle (1995) stated that limitations to reliability and validity of the MBTI raise concerns about its use by practitioners, while Pittenger (1993) explicitly stated that there is no evidence to show a positive relation between MBTI type and success within an occupation. That – however and sadly – doesn’t mean that it is not used. On their website (https://www.myersbriggs.org/), the Myers – Briggs Company claims that “[they] have helped organisations such as Siemens and the NHS to identify talent, create development plans or high-potential managers, and upskill HR teams.”
4. Implications
Through this analysis the problem with unstandardized interviews becomes apparent. Factors like attractiveness, gender and hight influence our decision-making process.
But even when using standardized testing some of these factors still influence the results through social effects on the macroscale, given that the test is valid and reliable in the first place.
What should be done is relatively simple. There is a solid understanding in personnel psychology about the factors that have the highest predictive validity. IQ (or more specific general mental ability), integrity, structured interviews, interests and conscientiousness are constructs or methods that are well stablished in the scientific literature and have been thoroughly tested (Schmidt & Schaffer, 2016).
The focus of every practitioner should be to measure these factors as accurately as possible, while being aware of other factors that may influence one’s judgement.
These factors need to be explored in more detail by the scientific community and scientists that do experiments that test the predictive validity of certain traits that don’t yield significant results should be published more often.
Lastly a wider understanding of those principles must be achieved by seminars, reading or presentations for practitioners. To quote Neil Anderson, (2017) “There has been growing concern, expressed by several authors internationally in Industrial, Work, and Organizational (IWO) psychology, of an increasing divide between research and practice in personnel selection.
While many would recognize that selection psychology has flourished as a research-based professional practice, there have been unambiguous signs that the practitioner and scientific wings of the discipline have been moving away from each other over the past decade.”
Anderson, N. (2017). Relationships between practice and research in personnel selection: Does the left hand know what the right is doing?. The Blackwell handbook of personnel selection, 1-24
Boström, G., & Diderichsen, F. (1997). Socioeconomic differentials in misclassification of height, weight and body mass index based on questionnaire data. International journal of epidemiology, 26(4), 860-866.
Boyle, G. J. (1995). Myers‐Briggs type indicator (MBTI): some psychometric limitations. Australian Psychologist, 30(1), 71-74.
Davidson, M. J., & Burke, R. J. (2000). Women in management: current research issues volume II. Women in management: Current research issues, 2, 1-8.
Dawkins, R. (1976). The selfish gene (40th-anniversary edition). New York: Oxford University.
Dion, K. K. (1973). Young children’s stereotyping of facial attractiveness. Developmental Psychology, 9(2), 183.
Dion, K., Berscheid, E., & Walster, E. (1972). What is beautiful is good. Journal of personality and social psychology, 24(3), 285.
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & Reporting Bias Group. (2013). Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PloS one, 8(7), e66844.
Eden, D. (1992). Leadership and expectations: Pygmalion effects and other self-fulfilling prophecies in organizations. The Leadership Quarterly, 3(4), 271-305.
Eid, M., Gollwitzer, M., Schmitt, M., GmbH, J. B., & KG, C. (2015). Statistik und Forschungsmethoden (4., überarbeitete und erweiterte Auflage). Beltz.
Forgas, J. P., & Laham, S. M. (2016). Halo effects. In Cognitive illusions (pp. 286-300). Psychology Press.
Gilbert, D. T., & Malone, P. S. (1995). The correspondence bias. Psychological bulletin, 117(1), 21.
Goddard, R. W. (1985). The Pygmalion effect. Personnel Journal.
Heineck, G. (2005). Up in the skies? The relationship between body height and earnings in Germany. Labour, 19(3), 469-489.
Hosoda, M., Stone‐Romero, E. F., & Coats, G. (2003). The effects of physical attractiveness on job‐related outcomes: A meta‐analysis of experimental studies. Personnel psychology, 56(2), 431-462.
The Myers & Briggs Foundation. (o. D.). The Myers & Briggs Personality Test. 2003–2022, The Myers and Briggs Foundation. Abgerufen am 29. Juli 2022, von https://www.myersbriggs.org/
Jawahar, I. M., & Mattsson, J. (2005). Sexism and beautyism effects in selection as a function of self-monitoring level of decision maker. Journal of Applied Psychology, 90(3), 563.
Judge, T. A., & Cable, D. M. (2004). The effect of physical height on workplace success and income: preliminary test of a theoretical model. Journal of Applied Psychology, 89(3), 428.
Langlois, J. H., Kalakanis, L., Rubenstein, A. J., Larson, A., Hallam, M., & Smoot, M. (2000). Maxims or myths of beauty? A meta-analytic and theoretical review. Psychological bulletin, 126(3), 390.
Lin, L., & Chu, H. (2018). Quantifying publication bias in meta‐analysis. Biometrics, 74(3), 785-794..12817
McKinney, A. P., & Miles, A. (2009). Gender differences in US performance measures for personnel selection. Equal opportunities international.
Messner, C., Wänke, M., & Weibel, C. (2011). Unconscious personnel selection. Social Cognition, 29(6), 699.
Barrick, M. R., & Mount, M. K. (1995). The Big Five personality dimensions: Implications for research and practice in human resource management. Research in Personnel and Human Resources Management, 13, 153-200.
Pisanski, K., & Feinberg, D. R. (2013). Cross-cultural variation in mate preferences for averageness, symmetry, body size, and masculinity. Cross-Cultural Research, 47(2), 162-197.
Pittenger, D. J. (1993). Measuring the MBTI… and coming up short. Journal of Career Planning and Employment, 54(1), 48-52.
Rosenthal, R., Jacobson, L. (1968). Pygmalion in the classroom. The Urban Review. 3 (1): 16–20.
Schmidt, F. L., Oh, I. S., & Shaffer, J. A. (2016). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years. Fox School of Business Research Paper, 1-74.
Schmitt, M., Gerstenberg, F., & Heim, A. C. (2014). Psychologische Diagnostik kompakt. Beltz.
Sear, R., & Marlowe, F. W. (2009). How universal are human mate choices? Size does not matter when Hadza foragers are choosing a mate. Biology Letters, 5(5), 606-609.
Sorokowski, P., Sorokowska, A., Fink, B., & Mberira, M. (2012). Variable preferences for sexual dimorphism in stature (SDS) might not be universal: Data from a semi-nomad population (Himba) in Namibia. Journal of Cross-Cultural Psychology, 43(1), 32-37.