With the growing myriad of highly talented, competent and skilled job seekers and incumbents, are you facing the chaos to differentiate a high potential from a high performer- a person who will go extra mile to achieve the business goals.
Psychometric Testing - Things To Keep In Mind Before You Subscribe For Them
Would you not want to hire somebody who is not only skilled technically but also is behaviorally equipped with the right disposition? Psychometric tests, as a matter of fact, have become an integral part of the recruitment, training and development, and succession planning process more than ever before, as organizations see benefits across the talent value chain in making key people decisions.
But how do you determine the right solution for your organization?
The quality of the psychometric tests has a vast impact on how well personnel-related decisions “cut the mustard” in the end. A few rules of thumb can be used to assess the most appropriate tests, even if you are no expert.
As a simple rule of thumb, every psychometric test should have a product description just like the one you see on a juice can: a list of nutrients and additives that you can use to assess the desired impacts of the product. Only these will enable the commensurable comparison of different psychometric tests.
A SUCCESSFUL PSYCHOMETRIC TEST IS JUST THE SUM OF IT'S PARTS.
A good psychometric test should include an exhaustive report on the competencies being measured and a behavioral interview guide. But it may not have any significance if the tests being used are technically poor or misleading in their results.
There has to be a high correlation between the interviewer’s interpretation and the test results. When you put a test and interpretation together, you get reliable personnel assessment. Thus, good psychological tests are an inevitable tool in personnel evaluation.
Are you aware that every test seller highlights the reliability of his/her tests – only a few can actually validate their arguments? Do you understand when is a test “reliable” and why validity has a paramount significance for a good psychometric test?
CAN YOU JUDGE PSYCHOMETRIC METHODOLOGY BY IT'S FACE VALUE?
The mushrooming of psychometric tests providers with extensive marketing activities and strong sales pitches is leaving the organizations in a confused state. But are the promises turning out to be sensible and technically appropriate? A common factor in almost all the technically not-so-good tests is that they tell the users exactly what they want to hear. The test providers invoke new and revolutionary research results and underline how easy, quick, and accurate the tests are. But are they meeting the technical requirements of test construction based on International Test Construction Standards?
The recruiting stakeholders often question the simplicity and the authenticity of the tests designed on the basis of the International Test Construction Standards due to a couple of attractive and engaging tests available. But if you look at how well different tests predict work performance, the best ones are traditional personality and aptitude tests. They may be conservative, but they get the job done, as they control the human biases and random variations to the greatest extent.
It requires a thorough understanding of the subject to distinguish a functional and technically appropriate method from the gibberish. No wonder why the personnel department in most of the organizations have given up and outsourced testing in its entirety to a few international organizations specializing in it for decades. This limits the use of psychometric tests in developing countries like India due to lesser cost-effectiveness and their relevance for the Indian sub-continent.
BUT WHOM AND HOW CAN YOU TRUST?
Is the situation as audacious as it seems? Technically, good psychometric tests are always preceded by research studies on the target group before their implementation in real life- a validation study. Often some kind of a reliability study is presented as a validation study. But this is misleading.
It is statistically proven that “a reliable test is always valid.” But often the test providers show the reliability studies as the only way of somehow comparing different test types. By demanding the results of the validation studies, you can easily ensure at least the fact that the test provider uses an appropriate methodology and is familiar with the statistical techniques it uses.
The test user should always ask for the empirical studies related to the tests being used. When there is proof that a test works, there are always parameters that enable comparison with the related constructs. If these are missing, you should keep on looking for other options.
Another critical factor to observe is the test’s norm group. It is a sample of test takers who represent the population for which the test is intended. This acts as a reference group for the future test takers from the target population. If the norm group is poorly selected or too small, the test result is not valid either. For example, a group of software engineers from France is not an applicable point of reference to assess Indian corporate executives.
The three anchors of a good psychometric test are standardization process, reliability, and validity. And even if just one of these psychometric properties is off the mark, the test fails to be a valid and reliable test.
You can assess each of these yourself without training if you are able to understand their significance in practice.
STANDARDIZATION PROCESS - THE KEY TO FAIRNESS
The foremost factor to consider is the standardization process of the test. If the test has been standardized using a similar language, educational, and occupational group as the one it is being targeted at; everything is usually in order. In order for the sample to be statistically representing the target group, the norm group should include at least more than a hundred tested individuals from the target group.
Just the fact of transporting or transferring the test data from one language or one geography to another constitutes reason for re-standardizing the test if the norm group is not sufficient. If the test was originally developed and standardized for an English-language population, it is not suitable as such for use in China.
Thus, standardization assures a fair reference or norm group. It, however, does not tell anything about the technical quality of the test. You need to understand the reliability and validity of the test further.
RELIABILITY: CONSISTENCY AND REPEAT-ABILITY OF TEST RESULTS
The statistical authenticity of the test is reflected by its reliability. Reliability can be used to determine how prone the test is to various statistical errors and to what extent can it deliver consistent results. If the test is not reliable, retesting of the same test on the same person can produce a totally different score for the same person.
For example, a personality test that produces very different results for the same person due to some biases or random errors cannot be considered reliable due to random variation in the test scores. This implies that the personality test is not reliable, and the result is impacted by other random factors such as anxiety, activity level, or environmental factors.
The reliability of a good test is – on a scale of zero to one – at least around 0.7–0.8. In practice, this means that the measurement result shows less than half the error produced by random variation. In aptitude tests, reliability is by nature greater, and in personality tests slightly smaller.
VALIDITY: THE ACCURACY AND PREDICTABILITY OF THE TEST
A test is valid when it measures what it is supposed to measure. To establish or increase the validity of a test requires professional and technical expertise in test construction.
The psychometric test, if valid, should be able to predict work behavior and performance at work, which is the purpose of psychometric testing. It is an essential decision-making aid when dealing with human behavior at work, so a test should be strongly linked to other psychological indicators related to the work behavior to be “valid.”
Often, common sense is sufficient to tell whether test questions are measuring what they are supposed to measure. If you find yourself filling in a questionnaire with questions that are not measuring what you would want to measure, you should ask the consultant demonstrating the assessment directly what is the purpose of the test and what constructs does it measure. If the test provider has adopted fair measures to develop and validate the test, he or she will be able to tell you what the test is supposed to measure.
Unlike reliability, there is no single and unambiguous method to establish validity as it is difficult to implement all types of behavior indicators into one indicator.
So as a good rule of thumb for evaluating the validity, you should look for the more the test correlates to other established, and well-tried tests the better is the validity indices. And if the test correlates with the actual performance (predictive validity), the stronger the predictive validity index.
CHALLENGE AND DEMAND
The next time you are purchasing or approached for a psychometric test, ask the assessment consultant about the psychometric properties of the tests they develop and market. In addition, you can ask for a short and simple description of what each test is supposed to measure. This is your right.
Do not settle for any vague discussion, but demand empirical evidence for reliability and validity. When you have all the empirical data available, the evaluation of the right psychometric test provider is much easier. Also, a reliable and valid psychometric test will be able to provide ROI (return-on-investment) with increased productivity, reduced turnover, and an effective, high performing organization.
Topics: Psychometric Test