Mettl’s assessments have been the biggest filter in our recruitment process. Their platform has helped us reach out to a higher volume our applicant numbers. Mettl constantly keeps innovating on their products and tries to introduce a new aspect to everything.
What pops in your mind when you first think of and hear the word ‘examination?’
Most of us, born and bought up in the 1980s and 1990s, associate it with never-ending rows of students, writing and contemplating in the same breadth, and invigilators walking at a leisurely pace almost in a mechanical fashion. The morbidly scary silence would break periodically, as some would ask for a complimentary sheet, much to the chagrin of the rest! There was no other way to assess students. Pen-paper examinations, hosted at designated centers, were the only means to evaluate them. There was no way to find loopholes in the only available process as you did not have a point of reference. Then came computers, the internet and a digital-savvy populace – the millennials – who embraced the ‘online’ world. The world now had the option of giving and taking examinations online. It was a choice exercised by a chosen few and widely considered an indulgence. The internet was expensive; computers were the gadget of the rich, and colleges and universities did not wish to burden themselves will ‘extra’ duties.
This dillydallying and fits-and-start approach concerning online examinations worked well as life continued as usual until one fine day, the world went into a tizzy. The Coronavirus pandemic swept the world, impacting continuity in an unheard-of manner. Schools, colleges, universities were shut down with no respite in sight. Vanity turned into a necessity! Online assessments suddenly became a rage, safeguarding the education community from a certain delay in academic calendars. The most vocal critics of online exams embraced them with enthusiasm. Online examination systems were deployed from Miami to Noida to ensure students continued their education and universities continue with their established academic schedules.
A semblance of continuity is now restored as the pandemic has somewhat subsided – the vaccines are out, and hope and faith on going back to the olden days have yet again gained mainstream currency. Much akin to a deluge subsiding, leaving behind a trail of devastation, the disruption that wreaked havoc on the education sector the world over has made us sit up and take note of the loopholes in physical examination processes. Pen-and-paper examinations have challenges that were, thus far, for the lack of a well-accepted alternative swept under the rug. Such a lackluster approach can no longer serve education sector insiders. What was once considered the only way forward can no longer be considered mandatory!
We always believed that online tests offered a scalable, robust, cost-effective and human resource-light alternative to traditional, pen-paper exams. These assertions were proven beyond doubt if one takes stock of the success stories of universities and centers of education the world over.
How about we go one step at a time. I do not wish to swarm you with gobbledygook but merely offer you a rationale, systematic understanding of the challenges involved in giving and taking a pen-and-paper examination. Let us begin.
Pen-and-paper examinations have been employed since the institution of examination has come into being. They are undertaken at designated bricks-and-mortar venues and involve the physical presence of all concerned stakeholders. The traditional examination process, much like any other undertaking, begins by understanding the intricacies and outlining the procedural requirements.
The foremost step in conducting a pen-paper exam is to possess an unambiguous clarity on the contours of the assessment process. What is the purpose of the examination? Is it to shortlist students to be admitted in the institution, i.e., an entrance examination? Or a semester test to analyze their learnings, evaluate their abilities and ascertain whether they can be promoted to the next semester or given the coveted degree? What would the test entail? What types of questions would the assessment involve? It may vary substantially, depending on the subject concerned or the purpose of the examination – as detailed above. The question paper may be a multiple-question type, diagram-based, long-form (descriptive), or involve solving mathematical equations, among many other such permutations and combinations.
Sounds elaborate, does not it? It is.
Let us go one step at a time!
There are numerous steps and challenges in creating and scheduling an examination for an administrator and other concerned stakeholders.
The offline exam process involves all levels of professionals associated with an institute/university/ education center. Teachers are responsible for setting up the question paper. Administrators are tasked with ensuring a hassle-free execution of the examination process, also overseeing the entire exercise from conception to completion. Then there are others involved in various aspects of the process. Seamless coordination among all stakeholders is a massive challenge. It isn’t easy to have everyone available at the same time, often repeatedly, to iron out specifics. It costs workhours, distracts officials from their principal duties and diminishes their efficiency.
Any physical or pen-paper mode exam process is comparatively cost-intensive simply because it involves more human resources. More human resources require more financial inputs, making the process resource-intensive. It also necessitates greater infrastructural inputs in meeting halls, expense on refreshments, stationery, and many others. (Hey, we don’t mean you stop serving those lovely cuppas and freshly-baked cookies, and please don’t blame us for the cost cuts!)
Any pen-and-paper examination is either constrained by the scale of the undertaking or highly expensive, capable of denting revenue and eating into your profits. Financial viability is critical for any undertaking. Therefore, it is a choice between a rock and a hard place! Even if you host a large-scale pen-paper exam by making massive investments, you would need to schedule them in installments to ensure your limited staff can attend to them while ensuring the quality expected of your institution. Or be prepared to pay inordinate sums to undertake the exam in one go. Told you! It is a tough choice to make.
You may have shown the courage to host an offline exam involving hundreds of thousands of test-takers, but you would still undergo tremendous strain in assigning invigilation duties. Who would invigilate a specific batch of test-takers? How many invigilators would an examination hall require? How many helpers would an examination hall need to tend to test-takers, offer them water, supplementary sheets, etc.?
Have you braved this too? Good for you! Wait, there is more. You have limited intake capacity as your examination halls can seat only 6,000 test-takers concurrently, and you have 47,000 test-takers! You would need to slot candidates and assign invigilation duties. These are extremely time-consuming and intricate processes, with no room for errors.
All hassles concerning candidate slotting and outlining the invigilation duties are behind you. Now, it is time to book pricy examination venues and address several other procedural challenges. Prepare to spend a bomb on printing those question papers, often extending to 10-20 pages, and those answer sheets too. You must factor in the hassle of transporting the question papers from your institute to the printing center and back, all the while remaining genuinely concerned about the question papers’ safety and secrecy.
Storing the examination-related material is equally a stressful and resource-intensive process. We have often heard of question paper leakages or even their loss because of uncontrollable circumstances, such as fire, water leakages or even pure, unadulterated sabotage.
Scheduling hassles in large-scale examinations is one of the most challenging aspects of any exam-giving process. Such examinations, especially entrance tests, need to be spread across days and destinations as test-takers from distant corners of a country/region partake in such assessments. It means you must pay for staff’s travel, lodging and food expenses and involve them in an extended activity, taking their focus away from their foremost duty - teaching and institution management.
This is non-negotiable because, as an institution, you are duty-bound to offer equal opportunities to all test-takers and ensure that geographical distances do not become bottlenecks, stifling their desire to attain a high-class education.
The challenge of establishing the grading system is yet another tricky issue in administering an offline exam. It is given that the process will be humanly administered. You must assess who would be grading how many papers, besides establishing unequivocally the relevant timelines within which assessors must finalize the grading process. It may require multiple meetings to establish the process, clarify doubts and assuage concerns, and placate those who may be miffed at additional work. Coordination, coordination and more coordination!
You are one-third through the examination process. It is time to set up the invigilation mechanism. It refers to allocating invigilation duties and firming up details on how many test-takers would be monitored by one invigilator. Let us understand some of the issues concerning this critical step in the exam-giving process.
Any physical invigilation process mandates involving your most scarce and valuable assets – professors and teachers. They are always in short supply, usually over-worked and juggling multiple priorities. Any institution burdening their most-valuable assets with the additional task of invigilating test-takers is stretching them thin. And to my mind, doing gross injustice. Let them teach!
Additionally, institutions must pay considerable sums, over and above their salaries, to engage them in such an exercise. Involving wonks in such laborious undertakings, and paying inordinate sums, is bad planning and execution.
I am not questioning your staff’s integrity, insinuating nothing, only stating the obvious. Despite your staff’s best intentions, question papers are open to leakages and manipulations in any pen-paper exam. Such an unfortunate affront may not involve your staff members but does irreparable damage to your institution’s repute. It also does not absolve you of your responsibilities toward your test-takers and other stakeholders, directly or indirectly involved in the institution’s functioning. Leakages and manipulations are untoward associates of offline exams, and they are complimentary. Enjoy!
Offline exams are monitored in-person, meaning more invigilators and support staff for every batch of students. It makes the process further resource-intensive, with no avenues of cutting down on the desired ratio. Any salami slicing of invigilators and staff allocation is likely to lower the test’s integrity, inadvertently jeopardizing the exercise’s objective.
You have streamlined the issues mentioned above and established all the prerequisites to offer the examination. Now comes the most important aspect of the entire exercise – the execution. It is time to let the test-takers visit the center or centers, depending on the undertaking’s scale, and take the test for which you have put in massive resources, planned for days at length, and stretched your team thin! Let us briefly analyze the bottlenecks in the process.
We live in volatile, disruptive times. It is not a commentary on your local mayor’s, state’s Governor’s (or Chief Minister’s), even your federal government’s functioning. But even the best of risk managers had no prescience of the COVID-19’s onslaught and how it would disrupt every established process and challenge age-old wisdom. It did!
There may be other challenges. There could be political demonstrations, natural calamities, such as floods or torrential rains, or, God forbid, a terror attack. Such uncontrollable events could throw your plans into a tizzy, causing you much heartburn, financial loss and rescheduling hassles, among others. And I have not even mentioned the harrowing experience of countless test-takers who would spend inordinate sums to revisit the centers, besides undergoing much physiological turmoil that stems from uncertainty. I am sure you would detest such an experience for your stakeholders, test-takers and staff, as a responsible administrator.
Pen-and-paper examinations are significantly more time-consuming than their online counterparts. Such tests require everyone involved to travel long distances and reach the venue well in advance, find parking spots for their vehicles, and partake in many other such rituals that are deeply frustrating for those who have bigger demons to slay. (Take their tests!)
An offline exam is highly restrictive. Such tests involve multiple stakeholders. Therefore, finding a timeline conducive to all is a major challenge. They ought to be given at particular venues at specific times, which may lead to coordination hassles, and not offer the flexibility much desired by students today. Allow me to say this: Institutions that do not offer flexibility to their students and teachers/professors are considered regressive and not forward-looking. So much for those brand-building exercises!
You have a well-defined and well-allocated duty roster for your entire faculty, choreographed to the minutest detail. However, you cannot be assured of the invigilation quality. I say this with a profound sense of responsibility that I am not questioning the teaching staff’s intentions. Neither do I doubt their resolve to offer a glitch-free and cheating-free assessment environment. However, test-takers have been routinely using ingenious ways and employing high-tech gizmos, such as Bluetooth-enabled earpieces and glasses, besides pressing into service the good old chits, to answer the test questions.
Human invigilation can be gamed, is expensive, and its efficacy is difficult to ascertain. Let us keep it at that!
The post-examination part also involves several processes, and each has its complexities, dealing with which requires extensive planning and immaculate execution. Despite your best intentions, you are bound to hit the wall a few times. Let us ascertain the procedural challenges in the post-exam phase of a pen-paper exam.
The test administrators are tasked with the safekeeping of the answer sheets, which is an extremely challenging endeavor. Imagine having to keep hundreds and thousands of test copies safely; imagine the resources involved in transferring them from the examination venues and safely storing them. Answer sheets are open to manipulation, loss and require serious resources to maintain them safely before they can be handed over to the designated teachers/evaluators for the assessment.
You must safely store the answer sheets and undergo an extensive process involving the allocation of copies to your professors and teachers. Those evaluators must assess hundreds of copies a day to cope up with the thousands of answer sheets that must be evaluated to offer results. Any delay in setting up the evaluation timeline can significantly impact the result declaration, having a cascading impact on the admissions process, or commencing the next semesters, both of which would derail your academic calendar. Not the most auspicious of starts for a new batch of students!
When comparing an online exam vs. offline exam, another challenge involving the process is that it is driven purely offline, which does not offer you any real-time updates on the evaluation process. The information gathering on the progress made, daily or even intermittently, is a nightmare for administrators as they must connect with several evaluators, analyze their progress and tally them with their schedule calendars to assess whether everything is on track. That sounds much work to me!
My gratitude to Alexander Pope for this timeless quote: “To err is human.” No matter how experienced your faculty and how noble their intentions, their evaluation may have some inadvertent errors, requiring corrections. Be prepared to walk a thin line, choosing between expediting the evaluation process and ensuring the utmost quality. I am sure you would choose the latter. Either way, you are squandering your one constituency’s goodwill. Hey, I never said it was easy!
Congratulations! You have crossed all the hurdles and navigated your way to the final phase of your offline exam-giving process. I would still caution against pre-mature celebrations as you are about to undergo several more challenges before you call it a day. Let us have a look:
We live in the age of instant gratification. Gone are the days when patience was a virtue. Test-takers wish to access their results as soon as possible, simply because it offers them better clarity on their admission prospects, allowing them to plan and re-calibrate their academic strategies better. Simply put, they expect you to tell them how they fared as soon as possible. No matter how quick and efficient your faculty is in evaluating those never-ending bundles of answer sheets, you are looking at a few weeks, at least, before you can declare the results.
No evaluation process, no matter how immaculate, can be free from glitches. At least, you cannot stop test-takers from filing for a re-evaluation; it is their right as examinees and your obligation as the examination authority. Such a re-evaluation process requires continual communication between the evaluators and the test-takers, further delaying the overall process. It is also cumbersome explaining why such and such marks were awarded for a particular question. Such a process may even stretch to months because of your limited faculty, their other ongoing duties, and a large number of test-takers vying for a thorough re-evaluation -another striking difference when comparing the challenges and benefits between an online exam vs. offline exam.
Data safety and privacy are the foremost concerns these days. These are not some moral obligations but legally binding requirements on you as an organizer. The specifics vary, depending on the national or regional agreements on data laws and compliances. Still, any theft or leak of test-takers’ data can have hefty consequences for you. Not to mention the loss of reputation and bad press it would garner. I am not deterring you, merely laying out the facts and challenges in hosting a pen-paper mode exam.
This marks the culmination of your months’ efforts to host and conduct an offline. I have not deep-dived too much for fear of turning this blog into a research paper, running into many pages. But you understand what I am trying to establish! I am not saying that the traditional pen-and-paper examination is a failed engine. It has served the education sector’s needs until computers and the internet became ubiquitous. It still continues to be the principal means of conducting examinations. However, they are elaborate, time-consuming and resource-intensive exercises and consume much energy and focus. All of which could have been used to ensure a superlative examination giving and taking experience by opting for online examinations, also saving significant work hours and financial resources. It also offers peace of mind to all stakeholders involved, and you cannot put a price tag on it. So think again when the next time you intend to host a large-scale examination or a semester test using the traditional means of assessment. The education sector has never seen such a massive transition to the online ecosystem as in the year gone by, and the results have been superlative, to say the least.
Here are the answers to some frequently asked questions on the subject.
An offline exam is an assessment conducted using the traditional means of pen-and-paper to assess students’ skills and subject matter expertise. An offline exam is often called a pen-paper exam. Such tests are conducted at designated centers at fixed dates and times.
There is an elaborate process in conducting an offline exam. There are several preparation stages, including mooting the examination, planning the question types and marking scheme, candidate slotting and invigilation duty slotting, etc. Candidates are provided with the exam admission cards, with all details about the center date and time of the assessment. After the test is hosted, answer sheets are evaluated, and results are declared based on the evaluation.
The debate on pen-paper exams vs. computer-based exams has gained mainstream currency, especially in the year gone by, with COVID-19 impacting academic continuity the world over. Pen-paper exams have been the mainstay of the education sector for time immemorial. However, easy access, lighter financial and human-resource footprints, hassle-free exam hosting and evaluation and the ability to proctor exams remotely have cumulatively turned the tide in favor of online exams. They are being preferred for their multiple advantages.
Originally published March 15 2021, Updated April 22 2021
Shashank has been working in the publishing and online industry for eight-plus years now. He has donned many hats and has reported on diverse industry verticals, including aviation, tourism, hospitality, etc. He is currently the senior editor at Mercer | Mettl.
Online examination, also known as virtual examination, is conducted remotely on a computer with high-speed internet. Like a classroom exam, it is time-bound and usually supervised through a webcam and proctor, making it cheating-free, secure and easily scalable.