Center for Teaching Excellence

Murphy Building
600 Forbes Avenue 20 Chatham Square
Pittsburgh, PA 15282
Email: cte@duq.edu
Phone: 412.396.5177

    A A Email Print Share

    Good, Better, Best: Multiple Choice Exam Construction

    Do you remember the following children’s ditty?  “Good, better, best. Never let it rest. 'Til your good is better and your better is best.”  This ditty reminds us as teachers who write multiple-choice exams that there is always room for improvement.  Writing better exams enables better assessment of our students’ learning.

    Before we consider examples of mistakes to avoid, the following example will help us to establish a common language about the parts of a multiple-choice exam item.

    Example and Parts of a Multiple-Choice Item

    3.  What is chiefly responsible for the increase in the average length of life in the USA during the last fifty years? =STEM
    distractor--- a.   Compulsory health and physical education courses in public schools.

     

     

     

    answer--- *b.   The reduced death rate among infants and young children
    distractor--- c. The safety movement, which has greatly reduced the number of deaths from accidents =Alternatives
    distractor--- d. The substitution of machines for human labor.

    To make your good exams better, and to make your better exams the best, try to avoid these exam writing mistakes.

     

    1. Poorly Written Stems

     

    A stem is the section of a multiple-choice item that poses the problem that the students must answer.  Stems can be in the form of a question or an incomplete sentence.  Poorly written stems fail to state clearly the problem when they are vague, full of irrelevant data, or negatively written.

     

    a. Avoid vague stems by stating the problem in the stem:

    Poor Example

    California:
    a.  Contains the tallest mountain in the United States.
    b.  Has an eagle on its state flag.
    c.  Is the second largest state in terms of area.
    *d.  Was the location of the Gold Rush of 1849.

    Good Example

    What is the main reason so many people moved to California in 1849?
    a.  California land was fertile, plentiful, and inexpensive.
    *b.  Gold was discovered in central California.
    c.  The east was preparing for a civil war.
    d.  They wanted to establish religious settlements.

     

    b. Avoid wordy stems by removing irrelevant data:

    Poor Example

    Suppose you are a mathematics professor who wants to determine whether or not your teaching of a unit on probability has had a significant effect on your students.  You decide to analyze their scores from a test they took before the instruction and their scores from another exam taken after the instruction.  Which of the following t-tests is appropriate to use in this situation?
    *a.  Dependent samples.
    b.  Heterogenous samples.
    c.  Homogenous samples.
    d.  Independent samples.

    Good Example

    When analyzing your students’ pretest and posttest scores to determine if your teaching has had a significant effect, an appropriate statistic to use is the t-test for:
    *a.  Dependent samples.
    b.  Heterogenous samples.
    c.  Homogenous samples.
    d.  Independent samples.

     

    c. Avoid negatively worded stems by stating the stem in a positive form:

    Poor Example

    A nurse is assessing a client who has pneumonia. Which of these assessment findings indicates that the client does NOT need to be suctioned?
    a. Diminished breath sounds.
    *b. Absence of adventitious breath sounds.
    c. Inability to cough up sputum.
    d. Wheezing following bronchodilator therapy.

    Good Example

    Which of these assessment findings, if identified in a client who has pneumonia, indicates that the client needs suctioned?
    a. Absence of adventitious breath sounds.
    b. Respiratory rate of 18 breaths per minute.
    *c. Inability to cough up sputum.
    d. Wheezing prior to bronchodilator therapy.

    Note: Test Writing experts believe that negatively worded stems confuse students.

     

    2. Poorly Written Alternatives

    The alternatives in a multiple-choice item consist of the answer and distractors that are inferior or incorrect.  Faculty often find coming up with enough distractors to be the toughest part of exam writing.  Common mistakes in writing exam alternatives have to do with how the various alternatives relate.  They should be mutually exclusive, homogenous, plausible and consistently phrased.

     

    a. Avoid Overlapping Alternatives

    Poor Example

    What is the average effective radiation dose from chest CT?
    a.  1-8 mSv
    b.  8-16 mSv
    c.  16-24 mSv
    d.  24-32 mSv

    Good Example

    What is the average effective radiation dose from chest CT?
    a.  1-7 mSv
    b.  8-15 mSv
    c.  16-24 mSv
    d.  24-32 mSv

     

    b. Avoid Dissimilar Alternatives

    Poor Example

    Idaho is widely known as:
    *a.  The largest producer of potatoes in the United States.
    b.  The location of the tallest mountain in the United States.
    c.  The state with a beaver on its flag.
    d.  The “Treasure State.”

    Good Example

    Idaho is widely known for its:
    a.  Apples.
    b.  Corn.
    *c.  Potatoes.
    d.  Wheat

    Note: The good example tests students’ knowledge of Idaho’s agriculture.  The poor example is confusing because students are unsure if they are answering a question on Idaho’s agriculture, geography, flag or nickname.

     

    c. Avoid implausible alternatives:

    Poor Example

    Which of the following artists is known for painting the ceiling of the Sistine Chapel?
    a.  Warhol.
    b.  Flinstone.
    *c.  Michelangelo.
    d.  Santa Claus.

    Good Example

    Which of the following artists is known for painting the ceiling of the Sistine Chapel?
    a.  Botticelli.
    b.  da Vinci.
    *c.  Michelangelo.
    d.  Raphael.

     

    d. Avoid inconsistent phrasing of alternatives:

    Poor Example

    The term operant conditioning refers to the learning situation in which:
    a.  A familiar response is associated with a new stimulus.
    b.  Individual associations are linked together in sequence.
    *c.  A response of the learner is instrumental in leading to a subsequent reinforcing event.
    d.  Verbal responses are made to verbal stimuli.

    Good Example

    The term operant conditioning refers to the learning situation in which:
    a.  A familiar response is associated with a new stimulus.
    b.  Individual associations are linked together in sequence.
    *c.  The learner’s response leads to reinforcement.
    d.  Verbal responses are made to verbal stimuli.

    Note:  The length of answer in the poor example is longer than the distractors.  Some students are keen at spotting these changes.  Also, the language in the poor example is from the textbook, but the distractors are in the instructor’s own words.  The good example makes the phrasing consistent in length and uses the instructor’s language.

     

    Sources:

    The parts of a multiple-choice item and examples in 1a, 1b, 2b, 2c, and 2d are from Steven J. Burton, et al,  “How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty,” (http://testing.byu.edu/info/handbooks/betteritems.pdf).

     

    The examples in 1c are from Mary McDonald, Systematic Assessment of Learning Outcomes: Developing Multiple-Choice Exams (Jones & Bartlett Publishers, 2002): 94.

     

    The examples in 2a are from Janette Collins, “Writing Multiple-Choice Questions for Continuing Medical Education Activities and Self-Assessment Modules,”  RadioGraphics 26 (2006): 549.  http://radiographics.rsna.org/content/26/2/543.full.pdf+html