The competitive world of psychometrics

​Many companies use psychometrics to help them in their development and selection initiatives. But many do not pay enough attention to what these tools are actually doing

For many companies, psychometrics are an invaluable part of their development programmes and often add a solid edge to what can be seen as 'soft' development. When a new psychometric arrives on the market, or an existing one is glossily promoted, we are sometimes asked to use them with our clients, or give opinions on them. A recent discussion on Twitter brought the well-known Insights Discovery tool into the spotlight, following its use in a development programme for the NHS in the north of England. Many professionals, including those in HR/L&D and occupational psychologists, are concerned about its theoretical basis and accuracy.

Insights Discovery is a great-looking tool - it’s popular with its users and many HR people. However, its performance in a number of areas leads many to question its accuracy. The British Psychological Society has rated it ‘inadequate’ on a number of counts, including its so-called ‘construct validity’, or the extent to which it measures what it says it does. ​

Other tools, like the NEO Personality Inventory, are based on more up-to-date science and have much more to offer HR and L&D professionals in real terms. Like others looking to base their psychometric assessments from a strong evidence base, we will not use Insights with our clients, but rely on NEO and other tools based on the more accurate Big 5 (or five factor model, considered the most accurate measure of personality).

In the meantime, it continues to bother us that many companies decide which psychometric instruments to buy off the back of pure sales and marketing, i.e. external sales people, and not necessarily those who actually understand them. If you’re using psychometrics to tick boxes for development, then so be it - but most companies want to do better than this, and in fact, think they are doing. But in reality, without checking themselves or taking the advice of independent experts, they may be buying ineffective products.

Of course, this is not to claim that only psychologists can advise on these matters! However, with years of experience in this industry, we have witnessed the effects of using tools that do not do what they purport to - which is at best sub-optimal development and unnecessary cost. Many psychometrics still popular today are based on largely outdated Jungian ideas despite the emergence of new, and better tools. It can seem that the desire to have this stuff easy, quick to understand and attractive is overwhelming for some.

“But users like it"
​
The frequent response we hear to any criticism like this is that ‘users like the tool’ as if that’s a good enough reason alone to use it. Imagine, for a second, the equivalent of expecting my GP to prescribe me chocolate simply because I like the taste! In our view, it isn’t a good enough reason, though of course it should be a factor in the ultimate decision. We might hope that more is done to increase the appeal of more rigorous psychometrics, but meanwhile we would recommend users of MBTI and Insights Discovery to look at the independent analysis published by Patrick Vermeren (from the agency, Evidence Based HRM) on this subject - see here.

Don’t be a sucker

In 1948, psychologist Bertrand R. Forer published a study on just this problem. He’d presented people with a supposedly detailed and accurate analysis of their personality, based on an assessment they’d previously completed. When asked to rate how accurate it was, on a scale of 0-5, people answered that their analysis was highly accurate - on average, 4.25/5. In reality, everyone had been given the same feedback. Because they’d been told that the assessment was accurate, they thought it was - and though this might seem benign, such inevitably unreliable assessment might well lead people to judge themselves on entirely inaccurate criteria, which won’t help them, or anyone else.

The effect Forer described is also known as the Barnum effect, after the 19th century circus owner and showman who famously said “there is a sucker born every minute”. Don’t let yourself or your organisation become a sucker. If you are thinking about using a psychometric tool we encourage you to check it out first - the British Psychological Society has a dedicated website for assessing psychometrics - https://ptc.bps.org.uk/.

We strongly believe in using evidence-based psychometric tools, based on scientific research, to support the work we do with our clients - we look forward to seeing the data first.

We provide psychometric assessments and assistance on the selection and use of tools. Do get in touch if you'd like to discuss further. t: 07711 503382 e: enquiries@managingchange.org.uk