School Improvement in Maryland

Characteristics of Good Items

Selected Response Guide

The SR item type can be used to test a wide range of knowledge, application, and reasoning skills. The item asks a student to discriminate among a variety of alternatives and to identify the most appropriate alternative in response to the question. Although the SR format can be used to assess factual recall, it is more appropriately used to test a student’s ability to make inferences, to judge logical relationships, or in other ways to go beyond the information presented. The most common form of this item type is the 4 choice question. Frequently, a set of SR questions can be organized to probe a single stimulus such as a table of data, a graphical figure, or a technical passage. A response to these questions usually takes about 1 minute. Conventional SR items consist of a

  • stem phrased as a question or an incomplete statement
  • response options consisting of the keyed/correct response or answer and distracters.

Item Writing Guidelines

  1. Format
    1. Format the item vertically, not horizontally.
    2. Use good grammar, punctuation, and spelling consistently.
    3. Minimize examinee reading time in phrasing each item.
    4. Avoid trick items, those which mislead or deceive examinees into answering incorrectly.
  2. Content
    1. Base each item on an instructional objective or indicator. Use both a skill and a concept indicator.
    2. Focus on a single problem.
    3. Avoid cueing one item with another; keep items independent of one another.
    4. Avoid items based on opinions.
  3. Structure
    1. State the stem in question form or incomplete sentence.
    2. Keep the vocabulary consistent with the examinees’ level of understanding.
    3. Ensure that directions in the stem are clear and that wording lets the examinee know exactly what is being asked.
    4. Avoid wordiness in the stem.
    5. Do not use negatives in the stem.
    6. Include the central idea and most of the phrasing in the stem.
  4. Response Development
    1. General
      1. Use as many functional distracters as possible (at least four).
      2. Place options in logical, numerical, or alphabetical order.
      3. Keep options independent; options should not overlap.
      4. Keep the length of options fairly consistent.
      5. Avoid the phrase “all of the above.”
      6. Avoid the phrase “none of the above.”
      7. Use positives instead of negative options.
      8. Avoid distracters that can clue test-wise examinees; e.g., avoid Clang associations, absurd options, formal prompts, or semantic (overly specific or overly general) clues.
      9. Avoid specific determiners, such as “never” or “always”.
    2. Correct Option
      1. Position the correct option so that it appears about the same number of times in each possible position for a set of items.
      2. Make sure there is one, and only one, correction option.
    3. Distracter
      1. Use plausible distracters; avoid illogical distracters.
      2. Incorporate common errors of students in distracters.
      3. Use true statements that do not correctly answer the item.
      4. Avoid the use of humor when developing options.

Laboratory Sets

A laboratory set consists of a cluster of items based on one stimulus. The stimulus is usually in the narrative form and describes a situation which leads the students into the investigative process. The questions which follow may ask the student to analyze the content of the scientific investigation, the adequacy of the experimental design and research methods. Students may also be asked to interpret data and/or write conclusions. The following list identifies some possible questions:

  • write an investigative question;
  • provide an hypothesis (including a rationale);
  • analyze procedures;
  • compare results with the hypothesis;
  • identify variables (independent/dependent) and/or controls;
  • organize data in a chart;
  • construct a graph to display data;
  • analyze the collected data;
  • write conclusion to the investigation
  • apply results to a new situation or use them to solve a scientific problem;

Technical Passage

Students may be asked to read and interpret a technical passage related to a scientific issue, or new discovery. The passage may come from an actual publication or may be created for test purposes. A set of questions following the passage will ask the students to respond to background information leading to the issue or discovery, the presence of bias, the adequacy of the results and conclusions.

The following list identifies some possible questions:

  • identify the research question or hypothesis;
  • evaluate the investigative procedures;
  • determine the adequacy of controls and variables;
  • evaluate the accuracy of conclusions;
  • offer ways in which the investigation could be improved;
  • offer arguments to support or refute the author’s or researcher’s conclusions;
  • interpret data from graphic representations;
  • describe implications of scientific findings;
  • use prior knowledge to answer questions about concepts, processes or terms mentioned in the passage.

Classification Sets

A classification set consists of a cluster of items based on one stimulus. The stimulus may be in the narrative form or it may be a graphic representation. The set of questions that follows offer the students the same response alternatives. The questions which follow may ask the student to evaluate a situation, classify ideas or objects.

Creating a Science Task

  1. Identify the scientific concept for the activity:

    What scientific principle should students learn from this investigation?

  2. Describe a real world
    • problem students can solve     OR
    • decision students can make     OR
    • question students can answer

    with information they collected from the activity.

  3. Develop a series of questions ( using SR. BCR, & ECR format ) to guide the student’s thinking through the situation created by the problem to solve, decision to make, or question to answer.
  4. Develop an introduction to the task.