VIEWS: 0 PAGES: 35 POSTED ON: 9/15/2012
Outline 1. Threats to validity 2. Our goal – control 3. 3 ways of achieving control: i. Statistical control ii. Experimental control iii. Replication Threats to validity Earlier, we discussed • Now, we’ll look at some threats to the validity of a sophisticated solutions to conclusion, including threats validity problems to: • internal validity • construct validity, and • external validity Our goal is control We want to eliminate every • Every – every plausible plausible alternative alternative must be ruled explanation for our results. out – if you allow any to – every survive, you don’t know whether your theory is right. Our goal is control We want to eliminate every • Plausible – a judgment, plausible alternative one the science explanation for our results. community makes – every – plausible Our goal is control We want to eliminate every • Alternative explanation – plausible alternative not the one you are explanation for our results. advocating – every – plausible – alternative How do we do this? Three basic ways of • Statistical control countering plausible • Experimental control alternatives: • Replication Types of Control – Statistical control • Experimental control • Replication Statistical control - definition Statistical control • Equating people on means equating paper requires use of subjects on paper when sophisticated they cannot be equated statistical analysis in fact. techniques, beyond the scope of this course Statistical Control – basic idea • Any behavior has • These characteristics many characteristics can be produced by a variety of influences – Is it fast? accurate? kind? absent-minded? – IQ, motivation, education, caffeine in the blood, etc. Statistical Control – basic idea Age • Statistical control Instruction requires measuring and IQ removing the effects of such influences when they are not of interest. Caffeine Mood Time of day Statistical control - example • Suppose you want to • What do you do if compare two methods GPA distributions for of instruction the classes are • You have two 1st year different? university classes to work with, one for each of the methods you’re comparing A note about distributions • A distribution shows how frequently scores Class 1 occur along the dimension being measured Class 2 • Here, the dimension shown on the X-axis is GPA Statistical control - example • What do you do if GPA • Remove from the distributions for the performance scores classes are different? the “contribution” due to GPA • Measure GPA • Remaining variability • Measure performance in performance scores now has nothing to do with GPA Types of Control • Statistical control – Experimental control • Replication Experimental control Experiments give us control, • They establish the in two senses: equivalence of two units (groups, subjects, conditions) before testing. • They constrain and guide sources of variability. Types of Control • Statistical control – Experimental control (1) Establishing equivalence (2) Constraining variability • Replication Types of Control • Experimental control (1) Establishing equivalence A. Between-groups approaches B. Within-subjects approaches (2) Constraining variability Between groups approaches • Subjects serve in C E either a control group or a treatment group, not both Between groups approaches • How do we make groups – Random assignment equivalent before – Matching treatment? Match subjects in treatment groups if… (i) There is a variable which • influences performance you believe : on your task • varies among subjects • can be controlled by matching • is not of interest to you Match subjects in treatment groups if… (ii) You can measure your – First put matched subjects on this variable subjects in sets before you start collecting – Then assign sets of data. matched subjects one to each treatment group. – This assignment should be random Group 1 Group 2 Group 3 89 92 90 111 112 110 Types of Control • Experimental control (1) Establishing equivalence A. Between-groups approaches B. Within-subjects approaches (2) Constraining variability Within subjects approaches • Each subject serves in all – Advantage – best conditions, acting as their possible control own control – Disadvantage – practice effects – See Chapter 11 for more. Types of Control • Statistical control – Experimental control (1) Establishing equivalence (2) Constraining variability • Replication Constraining variability We can control aspects of A. Research preparation the research situation other than how subjects are – What task do subjects assigned to conditions: do? The research preparation McBurney (p. 193) “[O]ne of the researcher’s goals is to choose the most suitable preparation for studying a given problem. Some of the most important contributions to psychology have been made by people who devised a new preparation for studying a given phenomenon .” Research preparation examples Semantic priming • Preparation: subject reads pairs of two words • Used to study the in quick succession, structures and processes responding only to the of our stored knowledge of the world. second in each pair. • RT and accuracy vary with semantic relationship between the two words. Research preparation examples Neglect • Standard neurological test for neglect. – neuropsychological disorder in which • Victor Mark’s modification patients ignore one half of space and everything in it. Constraining variability We can control aspects of A. Research preparation the research situation other than how subjects are – What task do subjects assigned to conditions: do? B. Instrumentation – how is performance measured? Instrumentation • Psychophysics • For 100 years, we believed that stimulus – the study of how intensity (S) was related perceptual experience to perceived intensity (I) varies with the logarithmically: stimulus – S = k log I Instrumentation • Stevens developed new • This new method Magnitude Estimation demonstrated that, in procedure: fact: – subjects assign numbers to stimuli on – S = a In basis of how intense they appear to be. Types of Control • Statistical control • Experimental control – Establishing equivalence – Constraining variability – Replication Replication Finally, one way to deal with • Will results generalize to threats to external validity is other classes of people? to replicate, with new subjects and materials. This lets you answer these • Will results generalize to important questions: other types of stimuli? Summary • When possible, we exert • When such control is not control during data possible, we control collection, by – randomly assigning nuisance variability subjects statistically, by equating – using subjects as their subject groups on paper. own controls – controlling stimulus • When statistical control is presentation. not possible, we control through replication.
Pages to are hidden for
"PowerPoint Presentation"Please download to view full document