Creating an aligned assessment task is a challenging task for teachers at the best of times. Given current demands and a general perception that they are easy to write, many unit convenors rely on multiple choice (MC) item exams for formative and summative assessment. When they are well written and used appropriately, MC questions can engage students’ higher-order thinking processes, such as application, analysis, and evaluation. Often, however, MC quizzes use poorly written items that test knowledge and comprehension (but not necessarily higher- order thinking) along with students’ test-taking skills. Below is a list of ‘do’s and don’ts’ for writing effective MC items.
Some tips for writing effective multiple choice items
- Each item should focus on an important concept or testing point.
- Each item should assess the application of knowledge, not recall of an isolated fact.
- The item lead-in should be focused, closed, and clear; the test-taker should be able to answer the question based on the stem/stimulus alone.
- Keep the options similar in length and level of detail.
- Balance the placement of the correct answer.
- Check for grammatical consistency. This often relates to the use of a/an, verbs/nouns, or singular/plural.
- Avoid clues to the correct answer in the stem or in other parts of the test.
- Avoid negative questions. The use of negatives can confuse students and lead to sentences that are difficult to interpret. If negatives cannot be avoided, highlight the negative in the stem of the question (e.g., in italics).
- Use only one correct or best choice. With one correct answer, alternatives should be mutually exclusive and not overlap.
- Avoid the “All the above” or “None of the above” options.
- Put as much of the necessary wording as possible in the question, not in the options.
- The stem of the multiple-choice question should be free of irrelevant material, i.e., it should hold only material essential for answering the question.
- Present options in a logical (e.g., numerical) order if one exists.
Tips for incorrect answer options (‘distractors’)
- Don’t include options that are obviously wrong. The alternatives should be plausible and attractive to unprepared students.
- Correct options don’t necessarily have to be absolute truths – students can be asked to select the ‘best answer’ or the one ‘which is most likely.’
- Statements based on common student errors and misconceptions make strong distractors.
- True statements that do not answer the question make good distractors.
- Absolute statements (e.g., ‘never,’ ‘always,’ ‘all’) are best avoided as students will rule them out.
- Keep the distractors sufficiently different to the correct response in substance, and not just clever or subtle wording.
- Ensure that the options/distractors are independent and mutually exclusive.
- Avoid absurd, jokey, and idiosyncratic distractors – they are easily spotted by discerning students.
- Make each distractor grammatically match the correct response, and consistent with the stem.
The shape of a good multiple choice item
A well-constructed, one-best-answer MC item has a particular ‘shape’ as shown in the illustration on the left.
The benefits of online multiple-choice item exams
- Versatility – they can be used to measure a wide range of cognitive processes: factual knowledge, inferential reasoning, application of knowledge, problem solving and critical thinking.
- They reduce the problem of subjective grading.
- Analysis of quiz results can provide diagnostic information. Access to diagnostic quiz information is easily available through iLearn.
- Automated marking and feedback assists teachers and speeds up the process for returning grades and feedback to students.
- Stems can include a diverse range of multi-media or just text.
The challenges of online multiple-choice item exams
- Creating plausible and unambiguous distractors that all share the same syntax and grammar are hard to write.
- Avoiding unconscious bias of the material for various groups of test-takers, which means being inclusive of gender, culture, and diversity.
- Creating appropriately challenging and clearly worded stimulus/stems that assess a range of cognitive levels, without making them too long.
- Finding the time to write the MC items, proofread, have them peer-reviewed by a colleague, and then edit based on feedback well before the exam period.
- Setting up an online quiz that reduces the potential for academic integrity breaches. This is complex so you may need additional support from a Learning Designer or Department Administrator to ensure that settings are correct.
Multiple-Choice Item Examples
1. Example of a Less Effective MC Item – Recall/Memory
Which description best characterises whole foods?
a. orange juice
b. toast
c. bran cereal
d. grapefruit*
2. Example of a More Effective MC Item – Memory-Plus Application
Sally’s breakfast this morning included one glass of orange juice (from concentrate), one slice of toast, a small bowl of bran cereal and a grapefruit. What “whole food” did Sally eat for breakfast?
a. orange juice
b. toast
c. bran cereal
d. grapefruit*
3. Example of an Effective MC Item – Interpretation
Why does investing money in common stock protect against loss of assets during inflation?
a. It pays higher rates of interest during inflation.
b. It provides a steady but dependable income despite economic conditions.
c. It is protected by the Federal Reserve System.
d. It increases in value as the value of a business increases.
4. Example of a (Non-Effective) Item with Grammatical Cue
A 12-year-old girl was brought to the school office because of chest pain. She has recently experienced an upper respiratory infection with frequent coughing. Her temperature is 37.2c, her pulse is 120/min, respirations are 22/min, and blood pressure is 95/65 mm Hg. Pulse oximetry on room air shows an oxygen saturation of 99%. Physical examination shows tenderness to palpation over her costochondral joints on the left. Auscultation of the lungs discloses diffuse end-expiratory wheezes bilaterally. Her diagnosis is most likely to be an:
a. asthma attack*
b. costochondritis
c. pleurisy
d. rib fracture secondary to coughing
e. viral pneumonitis
A final bit of advice …
Have a colleague (or two) trial your MC quiz in iLearn before you release it to students.
References
Paniagua,M. & Swygert, K. (Eds) (2016), Constructing Written Test Questions for the Basic and Clinical Sciences, National Board of Medical Examiners (NBME), Philadelphia, USA.
Clay, B. (2001), Is this a trick question? A Short Guide to Writing Effective Test Questions. Kansas Curriculum Center, USA.
Sibley, J. (2014), Seven Mistakes to Avoid When Writing Multiple-Choice Questions, Faculty Focus.
Smith Budhai, S. (2020), Fourteen Simple Strategies to Reduce Cheating on Online Examinations, Faculty Focus
Center for Distributed Learning, Examples of Multiple Choice Items at the Levels of Bloom’s Taxonomy, UCF.
Thank you, Liz. Always great to review do’s and dont’s’.