Psychology: Construct Validity in Psychological Measurement

Title: Construct Validity in Psychological Measurement
Discipline(s) or Field(s): Psychology
Authors: Carmen Wilson, Bill Cerbin, Melanie Cary, Rob Dixon, University of Wisconsin – La Crosse
Submission Date: January 15, 2007

Executive Summary

The goal of the lesson is to develop students’ understanding of construct validity as measured by their ability to: 1) explain methods used to determine construct validity for psychological measures and 2) design a study to determine the construct validity of a given measure.

Prior to the lesson. In the two class days prior to the lesson, the instructor presented information on content, criterion, and construct validity. Each type of validity was presented in terms of a question it answered and how it might be assessed. Content validity answers the question, “Do the items represent the domain of interest?” It can be assessed by having an expert in the topic review the test. Criterion validity answers the question, “Do scores on the test predict some non-test behavior?” It can be assessed by correlating scores on the test with some other measure of the behavior (e.g. behavioral observation). Construct validity answers the question, “Does the test measure what it claims to measure?” The lecture highlights several processes to assess construct validity. The answer to the construct validity question is dependent upon what is known about the construct being measured. For example, if the theory about the construct suggests that two groups of people should have different levels of a construct, and the test actually measures the construct, then the groups’ scores should be different.

We evaluated three versions of the lesson across three semesters. In Version 1 (lesson, no lecture – A), students developed a 5-item measure of depression and then designed three research studies to evaluate the validity of their measure prior to receiving any instruction about construct validity. In Version 2 (lesson, no lecture – B) we made minor modifications, but the lesson essentially remained the same. In Version 3 (lesson after lecture), we made significant modifications. The instructor lectured about construct validity first, and in a subsequent class, students analyzed three validity studies and then designed a validity study based on information provided by the instructor.

In Versions 1 and 2, students became bogged down in details of their proposed research studies and missed the more important goal of predicting results that would support the validity of their measure. The team decided to restructure the lesson so that in Version 3 they first heard a lecture, and then read summaries of real validity studies and predicted the results of those studies given the tests were valid. In the last part of the lesson students designed a study to determine if a given test was valid and predicted the results of that study. Interestingly, students who participated in Version 1 of the lesson (lesson, no lecture – A) generally performed better than students who participated in Versions 2 and 3.

Documents related to the Construct Validity Lesson Plan:

Individual Worksheet

Group Worksheet

Lesson Plan Versions 1 & 2

Design Validity Study Exercise

Convergent Divergent Scales

 

Documents related to the Study of the Construct Validity Lesson:

Student Perceptions of the Lesson

Observation Guidelines

Think Aloud Problem

Observation Guidelines Versions 1 & 2

Psychology: Classical Conditioning

Title: Improving Thinking about Classical Conditioning
Discipline(s) or Field(s): Psychology
Authors: Craig Wendorf, Robert Nemeth, Jody Lewis, University of Wisconsin – Stevens Point
Submission Date: August 26, 2007

Executive Summary: This lesson was designed to introduce students to the basic procedures and processes of classical conditioning within the context of a single-semester course in Introduction to Psychology. We usually allocate one-week for the unit on Learning, which approximates a single 50-minute class on classical conditioning (the remaining two classes are devoted to operant conditioning and observational learning, respectively). Thus, the lesson incorporates both lecture and activity and was designed with a multi-media compatible classroom in mind.

After the introduction to the elements and procedure of classical conditioning, students complete a worksheet (see attached) designed to provide students with practice using the terminology of classical conditioning, understanding the ordering of elements in the procedure of classical conditioning, and applying their knowledge to a variety of different examples including a novel one that students create. The worksheet provides an opportunity to assess students’ knowledge of classical conditioning from basic definitions of the elements of classical conditioning to the learning of a conditioned response through a variety of real-world examples.

Four assessments (peer observation of the lesson, grading of the activity worksheet, grading of relevant exam questions, and a student survey) indicated the lesson generally worked well. Both students and observers reported that the lesson was engaging and helped introduce an admittedly difficult topic. The lesson shows promise with several suggestions for improvement included.

General Psychology: Bystander Intervention

Title: Bystander Intervention: Explaining Behavior in Terms of Multiple Variables
Discipline(s) or Field(s): Psychology
Authors: Bill Cerbin, Melanie Cary, Rob Dixon, Carmen Wilson, University of Wisconsin–La Crosse
Submission Date: January 22, 2007

The lesson topic is the psychology of “bystander intervention” (i.e., why onlookers help or do not help a stranger in need of assistance). Research indicates that a number of factors influence how bystanders interpret an incident and whether they assume responsibility to help (e.g., the number of bystanders present, whether the victim appears to need and deserve help, etc.).

Learning Goals. The immediate academic learning goal of the lesson was to develop students’ ability to explain the “bystander effect” and how the presence of other people can affect individual behavior. A broader goal of the lesson was to develop students’ ability to explain human behavior in terms of multiple factors or variables, which is an important facet of social science reasoning. This latter goal is important because students often resist the complexity of multiple factors and tend to rely on a single factor to explain behavior–something we call “The One Factor Theory.”

Lesson Design. The lesson involved students in developing ideas about why bystanders help or do not help people in need. Prior to the lesson students did a homework assignment (pre-test) in which they read “bystander scenarios” that depict people in need of help. For each scenario they predicted whether the onlooker would help the person in need and then gave reasons why an onlooker would or would not help in the specific situation. In class students compared their answers on the pre-test and compiled a set of factors that influence people in bystander situations. The instructor then introduced a research-based model of bystander intervention, and led a discussion comparing students’ ideas to the model. At the end of class each student wrote an individual analysis explaining the similarities and differences between the model and his or her group’s ideas of bystander behavior. As a homework assignment, students analyzed another set of bystander scenarios (post-test exercise).

Major findings about student learning. On the pre-test students tended to explain helping behavior in terms of the bystander’s character and personality (e.g., compassionate people help others). On post-test exercise personality-based explanations decreased and social psychological explanations (i.e., situational factors) increased. During the lesson students generated a wide range of factors involved in bystander intervention and developed plausible explanations for them. In general, they were able to think in terms of multiple factors on the bystander scenarios. Although the lesson evoked the kind of thinking we hoped for, we do not know whether this single lesson changed students’ beliefs about the importance of situational variables as determinants of social behavior. For example, during the group discussion some students maintained that character and “upbringing” are primary determinants of bystander actions. The lesson may have uncovered a belief about human nature that influences students’ willingness to accept social psychological explanations of behavior.