Mettl’s assessments have been the biggest filter in our recruitment process. Their platform has helped us reach out to a higher volume our applicant numbers. Mettl constantly keeps innovating on their products and tries to introduce a new aspect to everything.
Around September, CBSE issued a rather strict set of measures on all its affiliate schools. While some constituted a certain level of hygiene in terms of a security upgrade and councils for child grievances, a couple left schools in general turmoil. To get to the point, the mandate instructs the psychometric assessment of all school personnel – teaching or non-teaching.
At such a time, it’s only natural for us to help break it down for you – the schools. While psychometric assessments are simple enough to deliver, the results do largely depend on their design and development. In fact, the sheer effectiveness or even its validity hinges on it. It makes it important for you to understand exactly what it is you’re deploying, or better put – if it even serves the purpose.
For this iteration of the piece, we’ll restrain ourselves to merely the teaching staff.
Test Standard Basics
When it comes to psychometric assessments, aside from validity and reliability, a key proponent to what makes the test work is in its norm or standardized group. Here’s a quick explanation for those terms:
A measure of how well a test measures what it claims to measure. You can’t have a test measure a shy person as extroverted, for example.
When assessment results are the same or nearly the same when carried out repeatedly under identical conditions and variables. After all, a person’s personality isn’t something that can change over time.
It refers to a sample of the test-takers representative of the population for whom the test is intended. Psychometric assessments work on percentiles, making the norm group important for comparative measures.
Today, despite Mettl’s status as one of the world’s most innovative assessment technology firms, we continue to believe in the process of an advisory panel to help us define a stellar standard of validity, reliability, and norm groups.
In terms of who makes the panel, we have educators and psychometricians alike. From Mr. Arun Kapur – Director of Vasant Valley School in Delhi, Dr. Gaurav Pandey – Electrical Engineering Department in IIT Kanpur, and Dr. Chirantan Chatterjee – Professor in Economics and Public Policy at ISB, to Dr. Puranjaya Singh – Ph.D. in Industrial Relations with 15+ years in corporate psychometrics.
At the end of the day, it’s the minimum standard of quality to look at when in consideration for psychometric assessments, especially in a defined job role; as is with teachers.
Who & What Are We Measuring?
The good thing is, your need from the CBSE mandate remains largely defined. It helps with the assessment curation process. This would include thorough job analysis, competency mapping, assessment & report configuration, benchmarking with respect to the norm group, and predictive validation.
Questions you’re likely to ask here:
Well, based on our analysis of the job role, with the help of the advisory panel and subject matter experts (SMEs), we map competencies likely to propel success in said job role into the assessment framework. This helps the assessment measure what it is meant to measure, ergo validity.
This is a lot easier to answer. When an assessment provides you with a result, how likely is it set to predict actual behavior and performance? That correlation is the essence of predictive validation.
In terms of what job-roles we had identified for the test creation process, it included teachers of a post-graduation level in a chosen subject, and teacher educators also.
The effort translated into four defined meta-competencies that echoed into the assessment:
By definition, a teacher is highly accountable for his or her actions. It marks an ability to confidently hold a strong presence in the classroom environment, but with a humbleness that enabled the recognition and acceptance of one’s own mistakes. Another remarkable trait here is in a teacher’s creative ability to explore new methodologies.
Recognizing child development and learning procedures is important during early education. Considering how a school functions within the limitations of a calendar, a teacher is required to adhere to its schedules and plan in accordance. This would imply solving bottlenecks as and when they occur.
This competency is driven by empathy, and a teacher must always remain empathetic to his or her students. Establishing trust, remaining fair, and staying patient marks a few innate traits necessary to handle students well.
They say a student learns mot from those they truly admire; teachers who invest in their students’ growth and development possess the right characteristics to motivate, mentor, and guide students. After all, engagement and modern methods of communication outside the limitations of a classroom does enable a student’s potential.
While these cover the broad spectrum of the assessment framework, each individual meta-competency is also broken into defined sub-competencies. To maintain length and time, it’s best to avoid discussing it in this article.
The point of this explanation is to detail the importance of method and deliberation. From covering definable quality criteria such as validity and reliability, a considerable effort is moved into outlining the standardized test group also. Add to it, SME and Advisory Panel influenced frameworks and test creation; it spells the recipe for a powerfully qualitative assessment.
What if they fake it?
Personality assessments are built on a self-rating mechanism with several statements presented on a strongly agree to disagree response format – the Likert Scale. And while this format popularizes its use across different levels, the easy methodology has led to problems along the lines of poor or inaccurate information.
Given that these assessments are self-rated, it’s natural to assume the worse with the candidates; that they would try to present themselves in a positive light to look desirable for the role. It’s what we call the social desirability bias. These tests are easy to cheat.
Additionally, Likert formats also lead to responding errors such as:
Choosing extremes, i.e., a high frequency of responses as either Strongly Agree or Strongly Disagree.
Staying neutral to most statements.
This proves that these assessments aren’t 100% foolproof. Information from these assessments serves more of a guiding purpose to decisions that the actual decision making itself.
However, at Mettl, we use a hybrid approach otherwise known as the Semantic Differential Scale. It presents applicants with two equally desirable options, removing the transparency of the more socially correct options inherent in the Likert Scale.
At the end of the day, schools exist to springboard children into the future; to help them build a tomorrow that we couldn’t hope to achieve in a lifetime. Teachers augment that experience. Wouldn’t it better to leave the psychometric assessments for teachers in the hands of someone who cares just as much as you do?
Originally published March 14 2018, Updated July 2 2020