Learning and Life: Assumptions of Conventional Exam vs Assumptions of LMS Based Online Exam

What are the underlying assumptions and presuppositions of conventional examinations and how do they differ from their counterpart assumptions and presuppositions of the LMS based Online Exams especially those conducted in asynchronous mode. The differences are explained below:

Here is a brief description of an extemporaneous talk that I recorded before putting the thoughts down more formally on this blog. This discussion is at a philosophical level designed to highlight the conceptual assumptions.

Assumption of Conventional Exams vs LMS Based Online Exams

Secret Question Paper vs Open Rubric/Open Criteria

Secret Question Paper is the holy grail of conventional examination. Secrecy engulfing the rules, processes, responsibilities of preparation of questions, then their printing, transportation, safekeeping, and their final unveiling at the time of exam is a huge undertaking of logistics (which has been heavily critiqued by many academics [13: Case Against Grades]). The moment we enter the realm of asynchronous examinations and studies in an LMS based online examination system, suddenly the assumptions behind the secret question paper fell apart.

It is simply not possible to guard the question paper in reaching the student in an environment characterized by physical distance, asynchrony, and connectivity issues. Hence, the entire assumption of the presence of the start-time of an exam, where after the question paper can be unveiled fails. The attention therefore must be diverted from the emphasis on secrecy of the question paper to a huge increase on the type of questions being asked as explained below. However, the important point to note is that the rubric on which the student is to be graded or evaluated should be open and well known in exam.

This is nothing new. Employees do not have to undergo an exam each year to test their performance. Every employee's annual appraisal is based on a rubric pre-specified by the employer. The rubric criteria is well known such as being a team player, meeting deadlines, maintaining regularity, showing initiative, participating in development etc. Similarly the performance measurement of students can also be in the form of rubrics for the questions that are well known in advance. A simlar rubric is commonly used in open competitions and by examination bodies (such as CIE's marking scheme), accreditation of universities, programs, and assessments of MS/PhD Students etc. See https://teaching.unsw.edu.au/assessment-rubrics.

Closed Book vs Open Book

From the above discussion it follows that the format of exams can not be closed book because connectivity issues, power breakdowns and equipment failures can not be ruled out. Hence, video cameras, bio metric verification, and other technological solutions for ensuring invigilation can all be defeated through clever contraptions. Hence, instead of fighting a losing battle against technological adversaries, the university needs to focus on the philosophical assumptions of open book exams of the type with which we are well accustomed in our MS/PhD thesis defense. The keyword then becomes creativity and originality. If it can be done for highly prestigious degrees like MS/PhD then it can also be done for other exams!

Synchronous: Same Time vs Asynchronous: Different Times

Technology and connectivity issues specially in geographically and demographically diverse country like Pakistan preclude the possibility of invigilators and students getting together during the same time interval on a reliable technology platform. Hence, the format of the exam needs to be asynchronous in time. The moment we acknowledge the need for asynchrony, the conventional duration of 3 hours goes out of the door. You can design exams spread over days, even months, even semesters. Please note that HEC's minimum duration for a PhD thesis is 2 years after a one year of mandatory course work, and HEC's recommended duration for an MS/MPhil thesis is one year.

Same Place vs Different Places

Just as there is asynchrony in time, there is also asynchrony in place. COVID-19 simply rules out the presence of the students gathering at the same place, or in a specified examination hall, or a stipulated testing center or decentralized physical examination centers. Therefore, the examination must be designed with the assumption that a student can be at any place and with anyone. Hence, elaborate mechanisms for invigilation predicated on a well known examination center no longer hold.

Taking Help is Cheating vs Collaboration, a 21st Century Skill

In a conventional exam, there are invigilators standing on the head of the students taking exams trying to ensure that the students do not collaborate, take help or give help. This is known as cheating, and is punished by extreme penalties. The assumption being that the information being written is so well known that anyone can easily help another.

However in an LMS based online examination, the assumption underlying the exam questions is quite different. What is expected from the student is to express something that is not known in advance, nor which is readily available i.e unique personal and individual insights. This assumption is similar to MS/PhD thesis, students are expected and encouraged to join support groups, collaborate and help each other in pursuit of an open ended problem. The student designs a solution and thesis that is original. The assumption here is that the problem is hard and the solution requires original thought and composition. The objective is to come up with something original and do a proper attribution to all the resources that were used. However, the demand on originality and rigor is quite different for a freshman or sophomore undergraduate vs a senior undergraduate and is also different for undergraduate levels as compared with higher research degree levels.

Please note that 21st Century skills also put a great emphasis on creativity, collaboration and team work (https://en.wikipedia.org/wiki/21st_century_skills). Thus, questions that are designed for online examination need to encourage students to collaborate in teams such as those that are formed for Final Year Projects or Senior Design Projects in engineering and computer science, capstone projects in social sciences, and other studio courses and group projects in fine arts and media science departments. In developing your thesis, all contributions made by others need to be properly acknowledged and attributed.

However, please note that collaboration is like group study. However, it does not mean impersonation, sitting for another person, getting the assignment outsourced. This is known as fraud and is strictly dealt with often leading to expulsion.

Reproducing exactly is OK vs Reproducing exactly is plagiarism

In conventional closed book exams, reproducing something exactly is OK. Actually most of the time what is being tested is regurgitation or rote memorization ("ruttafication"), which lies at the bottom two levels of Bloom's taxonomy, recall and understanding. This has led to a state where education system is now producing non-readers and non-writers [14: Non-Readers and Functional Illiterates].

However, in the open book exams, there is a high premium on originality. All material should be original and if copied should be properly attributed and paraphrased in one's own words. There are elaborate rules of attributions when you need to refer to an existing work, or use an existing framework or structure to put forward your own idea. Plagiarism invites strict penalties. There are sophisticated tools and technology that can identify plagiarism. Furthermore, just as we can identify the fake candidate with a highly impressive CV through a few minutes interview, it is easy to identify plagiarism through a few minutes viva or evaluation of a few minutes video of student's presentation.

Remember/Understand Levels vs Create/Evaluate Levels of Bloom's Taxonomy

21st century skills consider creativity skills to be at the highest pedestal. Questions falling at the Recall/Remember and Understanding levels of Bloom's taxonomy should not be asked in an online exam as they can be easily googled. Actually, they should not even be asked even in conventional exams, but they unfortunately are and hence Outcomes Based Accreditation and Assurance of Learning standards of international and now even national accreditation bodies are encouraging the universities to move towards questions belonging to higher levels of Bloom's taxonomy [1: Venderbilt].

In LMS based online exams, especially in asynchronous mode, the exams must focus on the higher levels of Bloom's Taxonomy i.e. create, evaluate and analyze. Questions at the higher levels can be designed in a manner to examine students' original contributions.

https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

To get an idea about the type of examination questions, the faculty needs to focus on the action verbs in questions that indicate the Bloom's taxonomy level of learning. These verbs are given in the table below that can help the teacher in designing the questions. However, this post is just an introduction. You need to go in further depth through reading, reflection and experiential learning.

For example, at the create level, the students can be asked to do the following: To write a blog about a particular issue; to build or assemble some new idea from existing components or ideas; to animate or make an animation regarding a process flow; to collaborate in materializing a project or in online publishing of a magazine; to compose some poetry, song or music; to direct a mission, play, or an expedition; to devise a solution for a problem; to create a wiki on some category; to write an essay or a plot or a short story; to make a film; to play a role or play acting a role; to mix some pieces to design some thing new or to mix some ingredients to prepare a recipe; to manage a meeting or a project; to negotiate a settlement etc. The details about these action verbs based questions can be easily obtained through googling [2: Bloom's Taxonomy Action Verbs ].

The general criteria is that you are fine if you keep the questions as near to the real life situations as possible. The more close you are to real life situations the easier would be for you to identify plagiarism.

The Evaluate level of Bloom's Taxonomy challenges the students to evaluate a particular phenomenon through a review of existing analysis and trends. It asks the students to argue in favor or against a proposition; to validate or test a particular hypothesis; to score performances on the basis of developing some criteria for evaluation; to assess a particular performance on some rubric; to critique a proposition; to debate a particular issue from different perspectives; to defend a particular idea; to set up an experiment to observe a particular phenomenon; to hypothesize after setting up some relationships; to measure some thing using one or more instruments; to moderate a discussion; to predict based on identifying some trends; to review/editorialize some findings. Please note that at this level you are relying on the work done by students at the analysis level.

Testing vs Learning

Conventional exams test the students to assign a grade and some score to measure their learning. The entire assumption is to arrive at a numeric score or a letter grade. The assumption changes for LMS based online examination, as the activity is not meant to just score or assign a grade, but is meant to further the learning of the student. A conventional exam is considered to be an activity that is different from teaching. However, in an LMS based online exam, our assumption is to further the learning activity, an extension of learning that has preceded it.

Examining Student vs Examining Teacher or Educational System

I think that originally examinations were held as a quality assurance for processes of learning and teaching institutionalized in schools and universities. Original intention was how the education system is doing and how much the teachers have been able to impart their knowledge. However, the emphasis of quality assurance of system shifted from examining the systems and teaching to examining the students' learning. From quality assessment of teachers and education system, the examinations became an instrument for assessing the students' learning, and in judging the behavior and attitude of students. [3: Who Fails when a Student Fails]

Judgmental/Labeling vs Encouragement/Self Improvement/Self Analysis

Grading and examinations became a tool for repression of students, instrument of their behavioral control through conditioning, through behavior modification using carrot and stick, labeling and discriminating the students, judging the students prematurely, and labeling them and confining them in artificially constructed boxes for the convenience of administrators. Hence, there is now an emerging consensus on the need to move away from grading as a tool for labeling the students to design of exercises that invite the students to self-evaluation, self accountability, self-analysis and discovery of self and its potential through reflection [4: Grading: The Issue is Not How But Why].

Prescribed Answer Sheet vs Semester Work

In conventional exams, the exams are "summative". A student's grade depends exclusively on the contents of the answer sheet submitted by the student in the examination hall at the end of the stipulated exam period. However, in an LMS based online exam conducted in an asynchronous manner, the type of exams are "formative" [5: Formative vs. Summative Assessments]. That is the grade depends upon the portfolio [6: ePortfolio] of activities carried out by the students throughout the semester which may consist of project artifacts, rough work, class work, notes, reflections, homework, assignments, quizzes etc. These may also consist of the teacher's assessment of students' readings of assigned blog-posts, watching of assigned videos, or listening of audios, and other resources. Fortunately all of this is available in LMS. A qualitative assessment of all of this work forms the foundation of the assigned grade.

Adversarial (Teacher vs Student) vs Mentor-ship (Mentor-Mentee) Relationship

As described above the teacher-student relationship gets severed through the grading strategy in conventional exams. It becomes judgmental and is often characterized by labeling or name calling such as he is an A-Grader or she is an F-grader [7: Standardized Testing and Its Victims]. This is adversarial relationship where the students often perceive the teacher as someone who is out there to fail them, and the teacher often thinks that the students are out there to hoodwink him, mislead him, gang against him etc. However, LMS based online education has a potential of establishing mentor-ship relationship where the mentor-mentee interactions are characterized by trust, confidence and hope. Teacher is often ready to help the student in furthering the learning by customization the learning exercises as per requirements of groups of students [8: Testing/Grading vs Motivation].

Standardization vs Excellence

The ultimate aim of centralized conventional exams is standardization. The problems of standardized testing and its harmful impact have been the topic of concerns for academics for a long time [ 9: Punished by Rewards ]. Unlike standardization that condemns the students to an hypothetical norm, the LMS Based Online Education has the potential for nurturing excellence among the students. Unlike going for subsistence with a passing grade, the learning environment can propel the students towards excellence [10: Excellence vs Guzara; 11: Excellence ; 12: Fairness in Grading].

21st Century Imperatives

These assumptions and critique of conventional examination is predicated on the 21st Century Learning Skills ((https://en.wikipedia.org/wiki/21st_century_skills) on which a broad consensus seems to be emerging. The conventional classroom and by implication conventional examination hall is no longer working. Motivation of students in the universities is suffering. There is too much rote learning in conventional schools and universities. Demands of the changing workplace and emerging technology is forcing the changes to a switch in these assumptions.

Please note that in an ideal environment, online education can mimic a physical synchronous classroom. However in the presence of connectivity issues, power issues and equipment issues which are difficult to solve for a diversified population of students of a typical university the switch in these assumptions would greatly further the development of much needed skills.

LMS BASED EDUCATION