Document Sample

JOURNAL OF APPLIED BEHAVIOR ANALYSIS 1978, 113,357-362 NUMBER 3 (FALL 1978) A PROBABILISTIC MODEL OF INTENSIVE DESIGNS TERRY F. PECHACEK1 UNIVERSITY OF MINNESOTA Without internal validity, experimental data are uninterpretable. With intensive designs, most methods presented to quantify a design's internal validity have been subject to criticism. A probabilistic model of intensive designs is presented that demonstrates the high degree of internal validity of these designs without relying on adaptations from traditional inferential statistics. Where the experimenter is able to conform to the re- strictions of the model, the equations provide an estimation of internal validity for either reversal or multiple-baseline designs. More importantly, the model provides mathematical bases for some of the common recommendations and design considerations in intensive research (such as the desirability of within-subject replications and of four or more multiple baselines). DESCRIPTORS: experimental design, intensive design, probabilistic model of signifi- cance, methodology, multiple baselines, guidelines, single organism research Demonstration of internal validity of infer- hypotheses not based on independent variables, ences about the effects of an experimental (in- especially those based on extraneous variables dependent) variable on dependent variables is such as history (psychosocial events concurrent central to experimental investigation (Campbell with but unrelated to experimental influences) and Stanley 1966). While traditional inferential (Campbell and Stanley, 1966, Kazdin and Ko- statistics provide numeric estimations of internal pel, 1975). validity, it can be meaningfully argued that Kazdin (1976) reviewed both the appropri- they can obscure the true relationship between ateness and controversy regarding efforts to independent and dependent variables by aver- adapt group-comparison statistics for single- aging within-subject variability of response to subject designs. The following is an attempt to treatment (Hersen and Barlow, 1976; Sidman, demonstrate by means of a probabilistic model 1960). Hersen and Barlow (1976) outlined that intensive designs do have a high degree of the advantages of single-subject designs in this internal validity without relying on adaptations area, but the internal validity of these designs from traditional group statistics. In formula- bears further elucidation. tions of a more general nature than those pre- Internal validity concerns the ability of the sented by Revusky (1967), the internal validity design to discount or eliminate plausible rival of both the reversal and multiple-baseline de- 1 Portions of this paper were presented at the an- signs can be demonstrated. Also, these formula- nual meeting of the Association for Advancement of tions offer mathematical rationales for some Behavior Therapy, Atlanta, December 1977. The ba- of the common recommendations for using sic work on this manuscript was completed while the these designs. author was a clinical psychology intern at the Palo In this model, the issue of what magnitude of Alto Veterans Administration Hospital, Palo Alto, California. The author would like to express his grati- observed effects or change is significant will not tude to Alan E. Kazdin for his editorial assistance on be addressed;2 rather, consistent with the basic this manuscript. Reprints may be obtained from Terry F. Pechacek, Laboratory of Physiological Hygiene, 2The reader is referred to the discussion of clinical Stadium Gate 27, University of Minnesota, Minne- versus statistical significance in Hersen and Barlow apolis, Minnesota 55455. (1976) and to the chapter by Kazdin (1976) on the 357 358 TERRY F. PECHACEK philosophy of single-subject research, "change" phase. A statistic can be formulated that esti- will be defined as a departure from baseline of mates the likelihood that treatment and reversal a magnitude that would be judged of clinical effects could be due to extraneous, random significance by the research community at factors. This statistic is based on the number of large; "no change" is defined as all other occur- possible patterns of results that could occur if rences. Therefore, these formulations are not the outcomes were due only to the random fac- dependent on traditional statistical analyses, nor tors. In the most general cases, random factors do they preclude their use. could produce three outcomes at each phase. In the development of this model, a basic as- (no change, an increase, or a decrease). Since the sumption must be made. Consistent with infer- experimenter is a priori defining a single, ex- ential statistics, plausible rival hypotheses for pected outcome for each phase, there is a proba- observed treatment effects are considered as a bility of 0.33 that this definition is correct if the single null hypothesis and expressed as the po- outcome is due to random factors. In a design tential for random change (that is, not system- with N experimental, postbaseline4 phases, the atically related to the experimental variables). probability that the outcome in all phases would This basic assumption states that the observed be correctly defined is equal to 0.33 * N.5 behavior has an equal probability of not chang- In some experimental analyses, the observed ing or changing (increasing or decreasing) at behavior may not have the freedom either to any time when measured in the natural state. increase or decrease. Such a case would occur This assumption does not imply that proxi- if the frequency was at or reduced almost to a mate observation or measurement are not corre- zero rate or near the physical or biological lated. Rather, this assumption states that the maximum rate of occurrence. In such cases, the effects of the extraneous factors can become probability of correct definition would be in- manifest at any time and, a priori, there is no creased to 0.50. Table 1 presents the probabili- more than a random chance to suspect that ties for increasing Ns in both the unrestricted they will coincide with the experimental vari- and common restricted types of cases. ables. If this assumption is accepted, the model The statistic can be simply and directly com- computes an estimate of the likelihood that puted for any design. Moreover, if each phase is rival hypotheses based on extraneous, random continued for a sufficient period of observations, factors (such as history) could account for the the intervention or reversal phases can be subdi- observed results in reversal or multiple-baseline vided. For example, the standard A-B-A design intensive designs.3 of two experimental phases can be expanded 4The initial baseline cannot be considered as an REVERSAL DESIGNS experimental phase in this model since the require- ments for obtaining a stable baseline preclude accept- In reversal designs, the experimenter defines ance of the basic assumption. If such extraneous the direction of change that is expected in each factors were to produce significant variability, the logic of the design requires the baseline to be con- tinued until it restabilized. appropriations of various statistical analysis in single- 5In reality, the probability (Pa) of the common case designs. patterns of data may not be 0.33 * N but would ac- 3While this assumption is constructed to be as tually be a function of the true probability (Pal) of consistent as possible with the phenomenon of in- the behavior changing in the desired direction in tensive research, the application of the hypothesis each of the N phases. Hence, the true probability testing approach, which is standard to most psycho- (Pa) = (P.z)(Pa2) . . . (PN), where 0 < Pai 1.0. in the first experimental logical research, does not always suit the endeavors In many situations, such as of the applied researcher whose goal often is the phase after the initial baseline, Pa, may be quite discovery, rather than testing of potent independent small, making Pai = 0.33 a generally conservative variables. estimate. A PROBABILISTIC MODEL 359 Table 1 the experiment during the intervention phase Probabilities for reversal designs with N experimental (A-B-A-B design) (Barlow and Hersen, 1973; phases. Hersen and Barlow, 1976), the experimenter Number of may want to include more than the two experi- N Type of Design Patternsa Probabilityb mental phases. This is especially true where the DESIGNS WITH UNRESTRICTED RANGE dependent measure is at or near either a zero 2 A-B-A 9 rate or the physical or biological maximum rate 3 A-B-A-B 27 0.037 during baseline or after intervention. Likewise, 4 A-B-A-B-A 81 0.012 the model suggests that more complex designs 5 A-B-A-B-A-B 243 0.004 6 A-B-A-B-A-B-A 729 0.001 (see Barlow and Hersen, 1973, or Hersen and DESIGNS WITH INITIAL BASELINE HAVING Barlow, 1976) should have good internal va- RESTRICTED RANGE lidity. 2 A-B-A 6 3 A-B-A-B 12 4 A-B-A-B-A 36 MULTIPLE-BASELINE DESIGNS 5 A-B-A-B-A-B 72 6 A-B-A-B-A-B-A 210 Within multiple-baseline experiments, sev- DESIGNS WITH RESTRICTED RANGE AFTER eral baselines are formed and then the indepen- INTERVENTION dent variables (intervention) are sequentially 2 A-B-A 6 0.167 3 A-B-A-B 18 0.056 introduced into the environments of each base- 4 A-B-A-B-A 36 0.028 line. Treatment effects are demonstrated when 5 A-B-A-B-A-B 108 0.009 each baseline changes in the desired direction 6 A-B-A-B-A-B-A 216 0.005 only after the intervention. Kazdin and Kopel aNumber of Patterns = number of permutations of (1975) stated that the internal validity of the change that could occur due to random factors. bProbability that the expected pattern occurred due multiple-baseline experiment is a function of a to random factors (p = (1 . number of patterns). number of factors, including the rapidity with which the behaviors change in the desired di- to a A-B1-B2-A design of three experimental rection, the variability in the data, and the num- phases. However, such subdivision accentuates ber of baselines used. Again, a probabilistic within-subject variability, which may obscure model can quantify some of these factors, dem- ability to define whether the predicted outcome onstrating the high validity of a well-con- occurred or not. Within the model, it is primary structed, multiple-baseline experiment and pro- that results be distinguishable as either signifi- viding specific recommendations regarding the cant change from baseline level or reversal back important issues of independence of baselines toward baseline level. and the number of baselines required. The On the basis of this model, the internal va- model is derived from the basic process of the lidity of reversal designs can be demonstrated. multiple-baseline design. When an experi- While the model concludes that the traditional menter selects the order of introducing the A-B-A design produces a probability estimate independent variables into baselines, an a priori, greater than the generally accepted criterion expected order of independent, sequential of p < 0.05, this estimate may be conservative changes in the baselines has been defined.6 If (see Footnote 4) and does not take into consid- eration that the data from many A-B-A experi- 6This requirement for a priori definition can run counter to the logic of some intensive experiments, ments could have been considered as A-B1-BW-A since it restricts the flexibility of the experimenter to or A-B-A1-A2 experiments. Nevertheless, in line a predefined order of treatment. Hence, the model with recommendations for direct replications applies only to those designs wherein the order of treatment can be flexible and not affect the intent (A-B-A-B-A designs) and for termination of of the experiment. 360 TERRY F. PECHACEK Table 2 Probabilities for Multiple-Baseline Designs Number of Baselines 3 4 5 6 7 SIMPLE CASEa Number of Patternsb 6 24 120 720 5040 Probability 0.1670 0.0417 0.0083 0.0014 0.0002 PROBABILITIES WITH POTENTIAL FOR SIMULTANEOUS CHANGEd Number of Patterns 15 64 325 1956 1.37 X 104 Probability 0.0667 0.0156 0.0031 0.0005 7.3 X 10-5 PROBABILITIES WITH POTENTIAL FOR SIMULTANEOUS AND/OR No CHANGEe Number of Patterns 35 208 1233 2.10 X 104 1.00 X 105 Probability 0.0286 0.0048 0.0008 4.8 X 10-5 1.0 X 10-S aDerived with formula fi(n) = n!. bNumber of permutations of change that could occur due to random factors. Probability of predicted pattern occurring due to random factors. dDerived with Equation 1. eDerived with Equation 2. the observed effects are inferred to be due to Unfortunately, the multiple-baseline experi- extraneous, random factors, then the observed ment is somewhat more complex. First, the re- order of change could have been any one of quirement of independence of change across several. If the basic assumption of this model, baselines is difficult to achieve (Kazdin and namely: that each observed behavior has an Kopel, 1975). Often, the baselines are not truly equal probability of not changing, changing in independent, and change in one increases the the undesired direction, or changing in the de- probability of change in others. Moreover, the sired direction at any time when measured in simple formula expressed above only considers the natural state,' is accepted, then the number sequential, nonsimultaneous changes in base- of possible orders of changes can be computed lines. The reality of interdependence creates by the formula, fi(n) - n!, where n is the num- the potential for simultaneous changes (defined ber of baselines in the design. When the experi- as the case where two or more baselines display menter can a priori define a single pattern of significant changes in the desired direction the expected results (see Footnote 6), the likelihood intervention is introduced in only one of the that this would occur due to random factors baselines). When this aspect of the design is would be 1- n!. Table 2 provides those proba- incorporated into the model, the formula be- bilities for increasing numbers (ns) of baselines. comes more complex. Equation 1 will compute the number of possible orders of change under 7While each behavior may not have an equal prob- these conditions. ability of changing or not changing, the likelihood nf(n that baselines will change in the desired direction is (1) f2() , of critical importance. Therefore, the experimenter -1 (n (- )! may need to select randomly the order of introducing the intervention into baselines or, where that is not Table 2 provides probability estimates for these possible for therapeutic or experimental reasons, be- more complex cases. gin with the baseline with the least probability of change and conclude with the baseline of highest In the simple formula, f, = n!, and in Equa- probability for desired change. The model is specifi- tion 1, it is assumed that all baselines will be cally inappropriate where the order of treatment is observed to change during the experimental selected to maximize the likelihood of producing the analysis. In reality, one or more of these base- desired results at each stage, since this approach maximizes the interdependence of the baselines. lines may in fact show no changes judged as A PROBABILISTIC MODEL 361 significant (see Footnote 2). Equation 2 com- provides a mathematical rationale supporting putes the number of possible patterns of results the most common recommendations regarding that could be observed under conditions that the need to use four or more baselines (Jeffrey, allow for one or more baselines to change simul- 1975; Kazdin and Kopel, 1975). In fact, the taneously and/or one or more baselines not to model demonstrates the rapidly increasing change significantly. power of the design when more than four base- 1n lines are used (see Table 2). f3(n) = n + The model also enables the experimenter = to evaluate the less desirable pattern of results t t! t 2 when interdependence of baselines produces { >2E[ r ) ((t - )!) + t!t nonsequential or simultaneous change in two (2) or more baselines. Kazdin and Kopel (1975) Table 2 provides the resulting probabilities discussed the problems in this area and the diffi- for increasing numbers of baselines under these culty in predicting the sought interdependence conditions. in baselines. However, the probabilistic model All probabilities in Table 2 are based on can quantify the situation of two baselines the optimum outcome in a well-designed, mul- changing simultaneously and displaying inter- tiple-baseline experiment, namely, an observed dependence and offers some estimates about pattern of independent, sequential, and signifi- the relevance of the results. Equation 3 com- cant changes across baselines in the exact order putes the number of ways that this type of pat- in which the independent variables were intro- tern could occur with n baselines plus the single, duced. When this does occur, even under condi- desired pattern of independent change across tions where the baselines are somewhat inter- all baselines. dependent, if the order of introducing the f4(n) = + ((n 1 (3) 2)!) intervention into the baselines was not biased toward the desired results (see Footnote 7), The results of Equation 3 define the subsam- the model would predict that possibly with three ple of patterns that, when compared to the re- and more definitely with four baselines, plausi- sults from Equations 1 and 2, provide ratio ble rival hypotheses for the results could be estimates of the number of baselines that will be rejected with a p < 0.05. Therefore, the model needed in order to reject the influences of ex- Table 3 Ratios with Designs Having Potential for Interdependent Baselines Number of Baselines 3 4 5 6 7 RATIOS BASED ON EQUATION 1 Possible experimental patternsa 6 12 20 30 42 Total patternsb 15 64 325 1956 1.37 X 104 Ratio 0.4000 0.1875 0.0615 0.0153 0.0031 RATIOS BASED ON EQUATION 2 Possible experimental patterns 6 12 20 30 42 Total patternsc 35 208 1233 2.10 X 104 1.00 X 105 Ratio 0.1714 0.0577 0.0162 0.0014 0.0004 aDerived with Equation 3. bDerived with Equation 1. cDerived with Equation 2. 362 TERRY F. PECHACEK traneous factors with an acceptable degree of of the strength of the obtained results. How- certainty should some pair of baselines display ever, the model does not offer a solution to the a degree of interdependence. Table 3 shows controversy regarding the best method in which these ratios for increasing numbers of baselines. equivocal results should be analyzed. Rather, In the interpretation of these results, a decision the model is based on the assumption that must be made regarding the appropriateness interventions will produce clinically significant of the ratios based on either Equation 1 or Equa- results, with the significance defined by the tion 2. The ratios based on Equation 2 require experimenter or experiment as appropriate. that all baselines display unequivocal change Hopefully, the application of the model where in the desired direction. Ratios based on Equa- appropriate will help to support the legitimacy tion 1 are valid as long as all baselines display of single-subject designs and encourage their a pattern of change distinctly related to the in- application to even more diverse experimental tervention, except for the dependent member of questions. the pair. Therefore, the model provides a mathe- matical rationale for using five, six, or more REFERENCES baselines depending on the experimenter's con- Barlow, D. H. and Hersen, M. Single-case experi- fidence in the expected results. mental designs: Uses in applied clinical research. Archives of General Psychiatry, 1973, 29, 319- Overall, this probabilistic model demon- 325. strates the high degree of internal validity that Campbell, D. T. and Stanley, J. C. Experimental and well-constructed, multiple-baseline experiments quasi-experimental designs for research. Chicago: can display. In fact the model predicts that as Rand McNally, 1966. Hersen, M. and Barlow, D. H. Single case experi- few as three baselines are sufficient for an ex- mental designs: strategies for studying behavior periment with independent baselines displaying change. New York: Pergamon Press, 1976. sequential changes distinctly related to the in- Jeffrey, D. B. Treatment evaluation issues in research on addictive behaviors. Addictive Behaviors, 1975, tervention. Moreover, the model permits the 1, 23-26. evaluation of less obvious results, such as those Kazdin, A. E. Methodological and assessment con- that occur when a pair of baselines display siderations in evaluating reinforcement programs in applied settings. Journal of Applied Behavior interdependent change; however, ensuring Analysis, 1973, 6, 517-531. against this occurrence may require the planned Kazdin, A. E. Statistical analyses for single-case use of at least six baselines. experimental designs. In M. Hersen and D. H. In summary, where the experimenter is able Barlow, (Eds), Single case experimental designs: strategies for studying behavior change. New to conform to the a priori definition process York: Pergamon Press, 1976. outlined in the model, and when the obtained Kazdin, A. E. and Kopel, S. A. On resolving am- results are judged sufficiently definitive and biguities of the multiple-baseline design: Prob- lems and recommendations. Behavior Therapy, unequivocal, the probabilistic model provides 1975, 6, 601-608. an estimation of internal validity for either a Revusky, S. H. Some statistical treatments compati- reversal or multiple-baseline type intensive ex- ble with individual organism methodology. Jour- nal of Experimental Analysis of Behavior, 1967, periment. In a process similar to inferential 10, 319-330. statistics, plausible, rival hypotheses are com- Sidman, M. Tactics of scientific research. New York: bined into a single null hypothesis that can be Basic, 1960. rejected with a specific limit of confidence. In Received 18 November 1975. this process, the model provides specific evidence (Final Acceptance 3 February 1978.)

DOCUMENT INFO

Shared By:

Categories:

Tags:
dependent variable, independent variable, Dependent and independent variables, independent variables, independent and dependent variables, dependent variables, Controlled variables, How to, Dictionary Definition, amount of snow

Stats:

views: | 27 |

posted: | 5/1/2010 |

language: | English |

pages: | 6 |

Description:
dependent-and-independent-variable pdf

OTHER DOCS BY xld14276

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.