Figure 3, p 5-1 in
TESTING AND EVALUATION IN THE BIOLOGICAL SCIENCES COMMISSION ON UNDERGRADUATE EDUCATION IN THE BIOLOGICAL SCIENCES REPORT OF THE PANEL ON EVALUATION AND TESTING NOVEMBER 1967/CUEBS Publication 20
Available for Download at no charge from: https://provost.ncsu.edu/ofe/wp-content/uploads/sites/2/2023/01/testing-and-evaluation-in-the-biological-sciences-2.pdf
A book review can be found at http://onlinelibrary.wiley.com/doi/10.1002/tea.3660060315/pdf
From NC State instead find the Journal of Research in Science Teaching in our Library, and this 1 page review is in Vol. 6, Issue 3, Sept. 1969, p 294.
Level - Cognitive Domain in Bloom's Taxonomy - labelled "Behavioral Objectives" above
Justification for covering this - we all know it - but making it explicit helps especially for large classes.
Several versions can be found - somewhat differing at highest levels - here's our version using the modern verb format:
Bloom's Taxonomy of Cognitive Objectives - a modern version.
7 levels - usual displays are only 6 levels - leaving out one of the top three above (usually combining Synthesizing and Creating)
Discussion given on pp 5-1 - 5-4 in the e-book linked above- and also see
Verbs for Writing Measureable Objectives
A
thorough presentation of Bloom's Taxonomy
Another
example of applying Bloom's Taxonomy in the contruction of test items.
Bloom's Taxonomy
Section 2 of the e-book covers Essay, Multiple Choice (MCQ) and Matching Questions. We'll concentrate on MCQs.
Characteristics that a satisfactory multiple-choice item should possess: a. the stem sets forth a single precise unambiguous task for the student to do; b. the stem is followed by a homogeneous set of responses, parallel in construction; c. no response can be eliminated because of grammatical inconsistency with the stem; d. the responses contain no verbal associations that provide irrelevant clues to the answer; e. the correct response is not more elaborate in phraseology than the incorrect ones; f. to the student who does not perceive the problem or know the answer, each response may appear to be a plausible answer.Terminology - stem, correct answer, foils.
Example questions - most of e-book
Note: Bloom's level of a Q depends on what has been covered
Exercise I
Remembering, Understanding, Applying, Analyzing
Discussion of these questions
Remembering Although most human factors specialists have earned a graduate degree and have learned several things through experience as a working professional, they may not know everything that they would like to know when working on a particular system. When human factors professionals need to find out information to guide their design decisions, what type(s) of resource(s) might they seek out? A. Data compendiums. B. Human factors design standards. C. Human factors principles and guidelines. D. Both B and C. E. All of the Above. Who wrote Origin of Species ? A. Mendel B. Lamarck C. De Vries D. Darwin E. Weismann Understanding Which of the following structures in the eye do NOT "go together?" A. Cornea and Lens. B. Pigmented Choroid and Purkinje Tree. C. Pupil and Iris. D. Rods and Cones. E. Zonule Fibers and Canals of Schlemm. Examples
The above graph of research data indicates that A. ATP is produced by green cells only when there is light present. B. most of the ATP of green cells comes from photosynthetic activity. C. chlorophyll is essential for photosynthesis. D. water is the source of the oxygen by-product of photosynthesis. E. promotion of ATP production is vital to the world's economy. Applying Whenever Lisa's aunts visit, they kiss her on the cheek. Lisa really dislikes this and eventually learns that they will not kiss her if she is coughing. Now whenever her aunts enter her home, she pretends to cough. Lisa's behavior is an example of _____. A. positive reinforcement B. positive punishment C. negative reinforcement. D. negative punishment. Categorize the pair of entities in each item as to which best describes it. A. I is greater than II. B. I is less than II. C. I is exactly or approximately equal to II D. I may stand in more than one of the above relations to II. -The number of genetic linkage groups in a eucaryotic cell -The number of genetic linkage groups in a procaryotic cell -The number of genes in an E. coli cell growing in minimal medium -The number of genes in the same cell growing in a rich medium Analyzing Which of the following is an example of a response-reinforcer relationship with good contingency but weak temporal contiguity? A. Sending sweepstakes coupons to the clearinghouse. B. Mailing three cereal box tops to receive a plastic toy. C. Putting a sandwich in the microwave to heat. D. Being burned by a hot stove. It is hypothesized that reptiles were the immediate evolutionary ancestors of birds. For each of the following observations, state whether it: A. supports the hypothesis strongly. B. supports the hypothesis weakly. C. has no bearing on the hypothesis. D. opposes the hypothesis weakly. E. opposes the hypothesis strongly. -The turtle egg has a leathery shell; the chicken egg has a calcareous shell. -Wings characterize honeybees, pterodactyls, eagles, and bats.
Exercise II
Evaluating, Creating-Synthesizing
Discussion of these questions
Evaluating Which of the following is most consistent with Norman's principles of interface design? A. A door that has a handle to grasp and requires a push to be opened. B. A software product that displays a ticking clock while processing a request. C. A stove that has all of the burner controls lined up in a straight line along the side. D. A stopwatch with three buttons that have different functions depending on the current mode. E. An entertainment cabinet door (with no handles) that requires a push in order to open it. Which of the following is the best experimental procedure to determine the effectiveness of a vaccine in preventing influenza in parakeets? A. Expose 100 parakeets to the disease and inoculate 50 with the vaccine. B. Inoculate 100 parakeets with the vaccine and expose all 100 to the disease. C. Inoculate 50 with the vaccine and expose all 100 to the disease. D. Inoculate 50 with the vaccine and expose those to the disease. E. Inoculate 50 with the vaccine and 50 with sterile saline and then expose all 100 to the disease. Creating-Synthesizing At the level of the cones, normal color vision is "trichromatic." At a higher level, the system seems to involve "opponent processes." Which of the following possible phenomena or findings would (IF TRUE) be most damaging to the theory of opponent processes? A. The observation that staring at a bright yellow square will result in seeing a blue square afterimage. B. Finding that people require four (not three) distinct "names" to describe spectral colors. C. Finding that people frequently report that they see a color they call "greenish-red." D. The phenomenon of color contrast. E. All of the above would be inconsistent with the theory. In which of the following conditions can a physician most surely pinpoint the specific cause from the data given? A. A boy has never formed pigment in his skin, hair, or irises. B. A woman has a fever of 104 degrees F. C. A dog is shedding much of its hair. D. A girl shows an unusually flushed face. E. A man limps badly with his left leg. Examples
Proportioning test items across the layers
Relate to goals in Learning Objectives for your course(s).
Tips on choice of question type and structure
(added post-workshop)
Q & A
Overview & and Looking Forward
Immediate goal - better questions covering your course learning objectives
Longer range goals - Annotation of test items ⇒ allowing computerized analysis of student learning (especially important in large classes)
Coming Events
Spring 2014 OFD workshop Item Analysis - regarding test item quality
http://hescgi.oit.ncsu.edu/Stop.html
Copyright 2013, 2015, 2020, 2023 by Henry E. Schaffer, Karen R. Young, Emily Wicker Ligon & Diane Chapman Comments and suggestions are welcome, and should go to hes@ncsu.edu
Last modified 11/26/2023