Electronic Exams Are Now Available!

Electronic exams (i.e. taken in a computer lab, not with pen and paper) are now available at Macquarie Uni (where suitable). Electronic exams maintain all the integrity characteristics you expect from supervised paper exams; students have only the scheduled time to complete the exam, they don’t know what is in the exam before arriving, and they can’t communicate with anyone else during the exam. Running exams electronically provides a number of important benefits;

• no paper is required
• exam answers and grading are kept indefinitely for review
• a variety of software can be made available during the exam
• exam question analytics are available
• you can mix auto-marked and hand-marked questions as appropriate
• students can get feedback outside semester and remotely, marking is faster
• providing detailed feedback is easier.

What is an electronic exam?

An electronic exam is one that is undertaken in on-campus computing laboratories. Students can review their solutions prior to submitting them but get no other clues. Exams are delivered as iLearn quizzes, academics only need to choose the right iLearn settings to have the quiz operate as an exam. Experienced iLearn users may have known this was possible, but many will be surprised to find out an iLearn quiz can be organised so that:

• Every student sees a different set of questions (randomly selected from a pre-populated question bank)
• The quiz/exam is scheduled for a specific time, can only be taken at that time, by students scheduled to sit the exam at that time
• No feedback is given to students until the results are released
• Auto-marked and hand-marked questions can be mixed and students won’t see the results of auto-marked questions until the results are release
• Students cannot access any other software on the computer while the exam is running (university lab computers can be configured to only allow access to iLearn or other software within the exam environment, more on this below)
• Student cannot access any other web pages or any other files while the exam is running (in university labs)
• Students cannot access anything at all from the Internet while the exam is running (in university labs)
• Markers can quickly mark randomly allocated questions
• iLearn will give detailed statistics on each question.

Is it really as easy as that?

No, of course not, there is set up work involved, but the work required is not as much as you might think.

The primary pre-requisite for electronic exams is that there is a substantial question bank from which questions can be randomly drawn.

Electronic exams cannot be used if the same exam needs to be given to each student.  There are two reasons for this:

• The dependence on computer laboratories means it is hard to schedule large classes to all sit the exam at the same time .
• Exams are taken in a room with dozens of large, back-lit, screens all showing the exam. It is too easy/tempting for a student to glance over at another student’s screen for inspiration.

For these reasons, electronic exams need to be different for each student so that students doing the exam later can’t get useful tips from students who did the exam earlier.

The other requirement is that you can organise iLearn groups that correspond with exam sessions and then set the exam restrictions to reflect this.

For more detail of what goes into setting up an electronic exam and how to set one up yourself visit the wiki page.

Better Exams

It is nice enough to be able to replicate paper exams more efficiently, but electronic exams open up a whole new world of exams as well. It is possible to configure the exam so that students can use particular software during the exam. Would you like a statistics exam where the student can use SPSS or a business exam where students can use Excel? Well, now you can. In courses where auto-marking is appropriate, electronic exams also open up entirely new types of questions, entirely new ways to check what students know. Beyond multiple-choice you can use any of the question-types that iLearn provides, and there are some really wonderful options available. A benefit that has turned out to be more valuable than we ever estimated is the ability to put one or two auto-marked questions into an exam where most questions are hand-marked. We had a six-question exam where one question was a binary-to-decimal conversion.

iLearn was able to auto-mark that question and we hand-marked the rest because seeing students’ problem solving and reasoning was important for those other questions. We were able to choose where our marking and feedback efforts could provide the most benefit to our students.

How is it made secure?

Students cannot access the exam with a normal web browser, only with special software called “Safe Exam Browser” (available only in university computer labs). When Safe Exam Browser is started, the computer “locks down”, allowing only the capabilities the educator has chosen. We have made the “Safe Exam Browser” available for use in any laboratory on campus and we have pre-built two versions: one that allows fully locked-down exams, and another allowing exams with statistical software. Ultimately, this is very much like commercial tools such as ExamSoft but works on all university computers at no cost.

How do you know it works?

In S1 2018 a pilot project team ran more than four-thousand exams using this system. We found all the devilish details, all the problems, and all the solutions. We surveyed staff and students for their opinions and I (at least) won’t ever go back to paper exams. A summary of what we have learned is:

• Students were happy with, or (at least) ambivalent towards, the system
• The only technical problem was students not being able to log-in to iLearn. This only occurred when a student had not yet logged into iLearn and so didn’t know their password
• Markers love the system
• You can spend less time shifting paper and more time giving genuine feedback
• Transcription errors disappeared entirely

I want to know more.

Staff who are interested in using this in their units can find more details on the wiki page. Please contact Matthew Roberts any time to discuss.

Here are some photos of me marking exams in varied and convenient locations, as I don’t have to carry around truckloads of paper exams with me!