Subscribe To Our Blog

Online Assessment   | 8 Min Read

SpeechX: An AI-Powered Automated English Evaluation Tool

Written By Shashank Shekhar

Contents

1. The Established Guidelines for Hiring in Customer-facing roles
2. Challenges in the Existing Process
3. Introducing SpeechX
4. SpeechX and CEFR
5. High Accuracy with Carnegie Speech
6. Incorporates Best Practices of Customer-facing Roles
7. Challenges with the Existing Tools in the Market
8. How SpeechX Solves These Pain-Points
9. Key Elements/USP of SpeechX
10. The Mercer | Mettl Way
11.Conclusion

A globally interconnected marketplace has lent English considerable clout in providing a level-playing field for the exchange of information and ideas. The primacy of English-speaking skills in any consumer-centric role in the service sector, cutting across industries, from hospitality, retail, travel, and tourism to insurance, sales, and banking is paramount. It plays a critical role in the right communication of the nuances of a product or service up for sale. It is, therefore, of notable importance among recruiters scouting for the right talent to further their organization’s commercial objectives. The argument especially holds true for BPO players, who must hire the right set of candidates with apt English-speaking skills to maintain a favorable consumer interface. They are indeed on the lookout for people to become the first line of contact for their existing and probable consumers. The scope of an error is virtually non-existent.

Hiring is a complicated and multi-layered process and demands to summon the very best of human and technological skills. From managing the logistical aspect to ensuring the sanctity of the exercise itself, the process of hiring is complicated and involves several stages. 

1. The Established Guidelines for Hiring in customer-facing roles 

In the established process, BPO companies evaluate the English-speaking skills of the candidates by hiring Voice and Accent (VNA) trainers/assessors -  who are professionals trained in a specific English framework called the ‘Common European Framework of Reference for Languages’ (CEFR) (For the uninitiated, CEFR is a set of guidelines used to determine the achievements of learners of foreign languages throughout Europe and now, increasingly, the world over.) While these trainers/assessors may be trained on other frameworks, they are primarily trained on the CEFR framework - which is a common standard for English evaluation guidelines.

1.1 How Do VNA Trainers Assess Prospective Hires?

VNA trainers evaluate the English-speaking ability of a prospective hire from several perspectives. It includes pronunciation, fluency, accent, and grammar. A VNA trainer/assessor scrutinizes for errors in the above mentioned criteria to understand whether a candidate can be trained in the discrepancies. VNA trainers evaluate and focus on discrepancies that have been divided into trainable and non-trainable erros. For instance, it would be counted as a non-trainable error if a candidate says 0-01-1
would constitute a non-trainable error. These fatal/non-trainable errors may also be grammatical in nature. 

Conversely, trainable/non-fatal errors are the ones that can be worked upon and addressed in a relatively lesser amount of time. For instance, a definite ‘T’ or ‘D’ sound while speaking are examples of non-fatal and trainable errors. This process enables VNAs to ascertain English-speaking and comprehension skills of candidates to determine their suitability for employment. 

However, this process is not rigid and has its nuances. Over the years, each company/organization has built and developed its own set of nuances and best practices to make hiring a seamless and hassle-free exercise. They subject candidates to multiple rounds of screening using VNA trainers/assessors who have a defined set of processes to make hiring decisions. 

0_Challenges in the existing process

 


2. Challenges in the Existing Process

GOTOTOP

2.1 Scaling for Mass-Hiring Is Resource-Intensive:

The process of employing VNA trainers/assessors by BPOs works exceptionally well for hiring and, therefore, has rightly been the established norm for a long time. Its efficacy in hiring at a smaller-scale is undisputed, but BPOs often struggle when planning to hire at a larger-scale. Large-scale hiring by using VNA trainers/assessors poses a unique set of challenges. Scaling the process is both resource-intensive and time-consuming, and is highly likely to make the entire recruitment process sluggish, consequently, adversely impacting business plans and the company’s balance sheet. 

2.2 Human-Led Methods Are Prone to Bias:

Not to speak blithely of their efforts, but a human-led process by VNA trainers is certainly not free from bias as every VNA trainer has his/her own inherent understanding, likes and dislikes. Given the human-led intervention, the visible lack of consistency across multiple trainers is also a challenge in hiring at scale. As mentioned, the BPO industry is always faced with high attrition rates and is therefore perennially in the hiring mode. Such a labor-intensive process is prone to errors. It may dilute the outcome, inadvertently lowering the quality of the hire.

Given these constraints in scaling the VNA-led process quickly and in a financially viable manner, BPOs usually rely on automated English assessment tools to aid the evaluation process for hiring the right set of candidates.

Is There an Answer to the Challenge of Conducting Mass Assessment for Customer-facing Roles in BPOs? 

BPOs routinely face the challenges mentioned above. However, there is a tool available in the marketplace to address these pain-points by auto-evaluating the English-speaking skills of the candidates and simulating the VNA experience. An Artificial Intelligence (AI)-based automatic English evaluation tool can bypass the problem faced by BPOs in undertaking mass hiring by employing VNA trainers/assessors.

3. Introducing SpeechX

Mercer | Mettl is going to unveil an innovative tool called ‘SpeechX’ to address these existing challenges of the BPO industry. Powered by reliable Artificial Intelligence Speech Technology, this assessment tool is fully machine-administered and auto-graded to test the ability of a non-native speaker to speak and understand English. It is a scalable means of assessing the capability of prospective hires with a high-level of accuracy by simulating a VNA trainer. It is also a beneficial and ready-to-use assessment solution for corporate houses to hire for critical client-facing roles as well as for sales profiles. 

SpeechX is a video-proctored assessment that analyzes a candidate’s proficiency across two key dimensions - the ability to listen and to articulate clearly. It reviews linguistics to identify what was correct and incorrect in the candidate’s speech and detects errors in reading sentences and extempore speech. It also undertakes para-linguistic voice analytics to measure the quality and clarity of a candidate’s statements. 

0_VNA Trainer

 


Why Does Speechx Simulate a VNA Trainer? 

As a VNA trainer determines a candidate’s employability based on his/her guidelines designed on the CEFR framework, SpeechX, too, simulates a VNA trainer. SpeechX provides a rating to the candidate on the lines of a VNA trainer to establish the level of the candidate’s English proficiency and employability. It checks for critical parameters of pronunciation, grammar, fluency, and listening skills, factoring in the nuances examined by VNA trainers, thereby simulating the entire process. 

4. SpeechX and CEFR

Put together at the beginning of the 1990s, by the Council of Europe to foster collaboration among language teachers across the continent, the CEFR framework is one of the many frameworks to assess the English-speaking proficiency of a non-native English speaker. It is interesting to note that as the framework was designed for all European languages, it can also be applied to test a person’s French or Spanish-speaking skills!

The framework has three core dimensions of language activities, domains, in which they occur and competencies drawn when engaged. It principally divides learners into three segments, which is then further divided into six levels of language proficiency. The three segments include basic users, independent users, and proficient users. A basic user is also categorized into two levels, i.e. A1 (beginner) and A2 (elementary). An independent user is classified into B1 (intermediate) and B2 (upper-intermediate). Similarly, a proficient user is bracketed into C1 (advanced) and C2 (proficiency). A CEFR test determines the level of competency and provides a score to the candidate, which is utilized to decide on his/her employability. 

SpeechX has incorporated the elements of CEFR and condensed them into four components of listening, comprehension, grammar, and fluency. 

0_Pronunciation

 

Pronunciation: To ascertain trainable and non-trainable errors.
 

0_Fluency
Fluency:
To understand whether a candidate can speak fluent English while conversing. 


0_Grammar


Grammar:
To verify a candidate’s level of understanding of grammar, and to detect trainable and non-trainable errors.


0_Listening ComprehensionListening Comprehension: A candidate is subjected to listening comprehension and assessed on his/her ability to listen and comprehend.
 

5. High Accuracy with Carnegie SpeechGOTOTOP

SpeechX uses Proprietary Speech Analytics and Carnegie Speech’s patented Speech Recognition Engine and Pinpointing Technology. It is combined with proprietary voice analytics. With over thirty years of experience in undertaking assessments, the robustness of the platform needs no further validation. It can listen to a non-native speaker of English and determine even the individual sound or phoneme level to identify errors. As a result, Carnegie Speech's engine is often referred to as the most accurate system on the market by universities in the USA, including Stanford, Yale, and Northwestern, to name a few. By processing hundreds of millions of speech assessments every year from countries the worldover, Carnegie Speech has gained unparalleled expertise in Speech Technology.

0_Industry

 


6. Incorporates Best Practices of Customer-facing roles

SpeechX has been designed after assimilating insights from various SMEs, VNA trainers, and BPO industry experts to combine the best practices of the BPO industry. On the lines of a VNA trainer who assesses trainable and non-trainable errors, SpeechX flags whether the gaps shown in the evaluation of a candidate are trainable. 

While evaluating pronunciation, it checks for more than eight critical non-trainable mistakes. SpeechX reviews fluency by measuring it across more than ten dimensions, such as the speaking rate, prosody, intonation, and pausing, etc. SpeechX evaluates a candidate’s understanding of grammar rules. Further, it assesses a candidate’s listening ability, looking for fact and inference-based knowledge. 

7. Challenges with the Existing Tools in the Market

0_Impersonation and cheating

 

7.1 Impersonation and Cheating:

Firstly, impersonation and fraud are intrinsic to such large-scale recruitments, and despite the best intentions of VNA assessors, it is a fact that such means are employed. The use of IVR by current tools available on the market are susceptible to cheating as they cannot be monitored. 

0_Accuracy7.2 Accuracy:

There have been several reports of glaring discrepancies between the results obtained by the VNA and the existing tool. This dichotomy raises a question on the validity of the test and calls for a re-evaluation. 

0_The lack of ease of use-

 

7.3 The Lack of Ease of Use:

The IVR-based setup is challenging to administer, and the user experience often suffers too. It also requires putting in place considerable logistics, which can be time-consuming and resource-intensive for organizations.  

8. How SpeechX Solves These Pain-Points

0_Impersonation and cheating

 

8.1 Impersonation and Cheating:

SpeechX uses AI-based video monitoring to deter candidates from using unfair means, thereby ensuring the sanctity of the exercise. It auto-generates cheating flags by using AI-proctoring technology.

0_Accuracy

8.2 Accuracy:

SpeechX is powered by Carnegie Speech’s world-class speech evaluation and recognition technology. This patented and reliable technology ensures a high degree of accuracy in the assessment results. 

0_The lack of ease of use-

 

8.3 The Lack of Ease of Use:

It is a computer-based process and does not face challenges enumerated in the IVR process. It is a smoother and hassle-free experience overall. 

9. Key Elements/USP of SpeechXGOTOTOP

0_Detailed Report

 

9.1 A Detailed Report:

A candidate’s performance is summarized in a comprehensive, actionable, and objective report which is immediately available for action. The report can be accessed in real-time and compared across multiple applicants, as well as across business and educational enterprises. It includes a CEFR and SpeechX score.

0_Accessibility9.2 Accessibility:

It is accessible on the cloud. Therefore, test results can be accessed at a moment’s notice, without the hassle of maintaining logistics for the same. It can be used on computers and smartphones. Thus, it provides a high degree of mobility to the candidates. 

10. The Mercer | Mettl Way: Why Mercer | Mettl Is Best Suited to Meet Your Needs

SpeechX is a part of a holistic suite of assessments. Mercer | Mettl’s suite of solutions are geared to provide you comprehensive, 360-degree assessments. These include psychometric, domain, language proficiency, and cognitive tools. A gamut of these offerings ensures that you are provided with all the possible means of assessment at one place with the highest levels of safety, security, and convenience to hire the right candidate by making better talent measurements.

11.Conclusion

Changing times calls for newer methods of talent assessment. The BPO industry is one of the few sectors amid a constant influx of the workforce. It is challenged with a considerably high attrition rate, as much as ten percent higher than the industry average of 35 percent. It requires putting in place a cost-effective, smart, and time-saving mechanism to improve efficiency and lower input costs. With an AI-backed solution, companies can be assured of hiring the right talent at scale, without exerting themselves financially or otherwise.

 

Topics: Online Assessment

Originally published January 22 2020,updated February 22 2020

Would you like to comment?