the study

Document Sample
the study Powered By Docstoc
					                                    The Decline of Positivist Perspective:
                    Early Correlates of Post-Positivist Development – An Empirical Study

                            M.M. McGee & R.S. Brown, Mosquito Creek Institute,
                                  J.O. Anderson, University of Victoria



         Brown, McGee and Anderson (1992)i formulated the field of post-positivist development in their
seminal study of the early correlates of the prototypical positivist/non-positivist divergence. They clearly
articulated the relationship of positivism’s lapse to the change of worldview from the explainable world of
mechanical objects with observable workings and analogue displays to the digital world of closed boxes,
square numbers, cryptic icons and blinking colons. Many of the studies generated from this perspective
further elucidated the dichotomization of educational research and identified the indicators of schism.
         Wilson, Shulha & Kirby (1993)ii identified knowledge of carburetors as a key indicator of a
positivistic predisposition. In a study of tenured faculty at a Canadian university they showed that
individuals who would simply attempt to identify engine components when the hood of a car was raised
were 54% more likely to be positivists in their research orientation. Those individuals who could
distinguish between carbureted and fuel injected engines upon inspection were solidly positivistic (88%).
Those individuals who didn’t know that there was a hood to a car nor that it concealed the engine (and that
the engine was what powered their vehicle) were predominantly social-constructivist-phenomenologist in
their research views and were intrigued by the question. Anderson (1999) iii demonstrated the relationship
of early exposure to mechanical clocks to the adoption of a positivist perspective. In a longitudinal study
he argued that if a child were exposed in a meaningful and accessible manner to the workings of a
mechanical clock as opposed to a digital time display there was a 37% greater chance of that individual
being identified as a positivist in their masters research work.
         In a retrospective study of social trends McEwen (1993)iv clearly demonstrated the onset of the
change was clearly linked to the predominant use of bottled water as a significant element of uniform and
the advent of sensible shoes (and boots) as a fashion statement. She demonstrated through a clever
retrieval of information from security camera footage that as the bottled water count increased on the tables
of graduate seminars, the discourse became more and more predominantly post-positivist in nature. This
was generally accompanied by the increased presence of what she termed sensible shoes – shoes that in
past times would have found a welcome home on the shelves of Dr. Scholl’s outlets that were now featured
in the pages of popular publications such as Vogue, Glamour and Gentleman’s Quarterly.


The Study
         The study to be reported in this paper involved the creation and evaluation of instrumentation for
the early identification for pre-dispositions in regard to positivistic/post-positivist perspectives in potential
researchers in the educational domain. The work has been funded through the auspices of the Canadian
Educational Researchers’ Association, the Canadian Security Intelligence Service and the department of



e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                            1
November 10, 2011
Human and Social Development through their job creations initiative. The prime motivation of the work
has been to develop a means of identifying the research perspective predilections of potential educational
researchers so that faculties of education can control and mold their incoming graduate student body to
facilitate the line(s) of inquiry that will most greatly benefit the educational aspirations of the nation or at
least the home university. Through early identification of potential positivists, for example, the
professorate of a faculty could encourage (or discourage) the applicant from pursuing admission and
through time develop a cadre of researchers that enhance the research agenda and program of the faculty –
for the greater good of all. As a collateral effect the individual professors could benefit greatly through the
identification of eager like-minded graduate students to serve as workers on their projects and in this way
enhance their c.v.’s in order to further their own careers and grow their funding record. It may be that the
graduate students could benefit as well although this was not a key focus of this study nor of any apparent
interest to professorial participants.
         The basic developmental steps involved the identification of prototypical positivist and post-
positivist researchers, the identification of key indicators of researcher perspective, the development of a
screening instrument, and the validation of this instrument through a controlled experimental study of a
longitudinal nature. Each step will be described in turn.


The Sample and Initial Data Collection
         Chronologically adult academics who engaged in educational research were the target population
for the identification and clarification of meaningful and accessible indicators. The sampling frame
consisted of the programs of the annual meeting of the Canadian Society for the Study of Education for the
past decade. Each individual who had delivered 5 or more research papers was considered to be an active
researcher. The frame was dichotomously stratified on the basis of research perspective: positivist and
post-positivist. The dichotomization was based on an analysis of titles of papers given. The categorization
was evaluated and validated though the differential item functioning validation panel procedures (DIFVPP)
developed by Rogers, Maguire, Gierl, Ma and Boughton (1999)v. The sample selected and agreeing to
participate consisted of 49 professors – 32 positivists and 17 post-positivists.
         On the basis of the aforementioned studies data was collected on a number of key indicator
variables associated with the continuum of interest: research perspective predilection. The source of these
data were birth records and other pertinent personal information from the Human Resource Development
Canada (Stewart, 2000)vi dataset which has a rich field of highly accurate and sensitive information of
every citizen in the nation.


Instrument Development
         The focus of this research was to develop a means of accurately identifying the research
perspectives of educational researchers – before they had to actually do any educational research. The
conceptual design of the work included two dimensions of data:



e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                           2
November 10, 2011
Table 1: Illustrative Fundamental Dichotomies


                                    Basic     Dimension                                               Extreme Location1
          MAC iComputers                                    WINDOWS                                        DOS/UNIX
          Peligrino/Ramosa                                 Bottled water                                    Tap water
          New imported car                            Ford/Chevy sedan/van                                Pick-up truck
      High fiber veggie-based                          Whatever(with salt)                                   Twinkies
                  Cats                                          Dogs                                         Reptiles
                Spiritual                                      Secular                                      Religious
           No data displays                               Digital displays                                  Analogue
          Personal meaning                                  Explanation                                       Blame
              Complexity                                     Simplicity                                    Sufficiency




                       The key personal variables that tend to be rather stable traits a person has that are
                            essentially hard-wired – perhaps not necessarily of a genetic base but certainly
                            attained at an early age and sustained throughout one’s life. Table 1 provides an
                            illustrative listing of such key indicators.
                       Personal perspectives on pertinent situational vignettes (pppsv) that would reveal
                            more temporally unstable elements contributing to current perceptual perspective
                            states. These looked very much like classic Likert-type questionnaire items.


All of these data were to be collected via stimulus-response questionnaire style administration in either
face-to-face, paper/pencil or computer-interactive formats. The main goal of the project was to develop
and validate such an instrument.
           The identification of the constellation of personal variables involved a structural modelling
approach to the stable personal variables (Nagy, 1995)vii we had easy access to national database though the
complete cooperation of one of funding sources - HRDC. Through vigorous data mining and analysis
coupled with in-depth, repeated interviews we were able to create a weighted composite variable consisting
of 26 key indicators variables. The result was a highly stable and accurate dichotomizer of researchers on
the basis of perspective or worldview, and this would effectively predict adult research predilection from
the age of 4 years 6 months onward. The predictive validity studies were conducted on the longitudinal
datasets available through the aforementioned HRDC database. We termed this weighted multivariable the
Hard-Wired Composite HWC. This composite variable was patented and the research team retains



1
 The extreme position turned out to be a serendipitous finding in that it revealed what were had considered to be a strictly
undimenisonal linear dichotomy turned out to be a folded circularity with the apparent dichotomous positions actually opposing
positions on a maximally curvilinear (circular) variable. This was revealed by the fact that on every extreme position, the individuals


e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                                                  3
November 10, 2011
exclusive ownership of both the variables and the weightings. For this reason we cannot divulge the exact
composition or weighting associated with the HWC. As an aside, this is the first instance of private
ownership of a variable and this ownership is being disputed by both the funders and the home university
of the junior author.
           In developing the pppsv items, a pool of over 100 positivist/post-positivist research perspective
sensitive vignettes was eventually created that met both veridical and statistical criteria. The basic concept
was to have respondents indicate their level of comfort with the actions and intentions of the individuals
portrayed in the vignette. Each vignette was designed to capture at least one element of known indicators
of research perspective predilection as established by previous research and the preliminary work done to
establish the HWC. The following released sample is illustrative of the genrè:


           Vignette
           Alphid is presenting a research report to peers using PowerPoint on the iBook. A colleague asks
           Alphid to support an assertion made concerning the effect of self-directed humor on meaning
           making in the classroom.
           Response prompt
           Which of the following are you most comfortable with?
                       Alphid is a female graduate student                               _____
                       The colleague needs time for reflection                           _____
                       PowerPoint is too constraining as a
                            presentation medium                                           _____


           The items were scored in a binary fashion with u=1 for the maximum post-positivistic response,
u=0 for others. The item score (which is essentially in the dichotomous case a probability of responding
correctly) was termed Strength to coincide with the terminology introduced by Dassa in his classic
compendium of post-positivist terminology (1997)viii. Figures 1 and 2 show typical item characteristic
functions for our initial pppsv’s. Respondent scores were reported on the scale entitled Post-Positivist
Research Perspective (PPRP). The scale typically runs from –3 (a positivist perspective) to +3 (strong
post-positivist). The middle position (PPRP=0) is typically termed the malleable particularly for the
application we were working with – selection of graduate students. We initially used conventional item
response theory approaches (Lord, 1980)ix to item analysis and trait score scaling. Figure 1 illustrates the
performance of an item that would be comfortable (correct in a sense) for an individual with a positivist
perspective – an individual with a PPRP of say –2 or less would respond with maximum strength to this
item. Figure 2 illustrates what is termed the classic PPRP item centred at 0 with a relatively steep slope
continually favouring a strong post-positivist perspective.


located at this position included an equal mix of positivist/post-positivist individuals. However their underlying motivations and
meaning structures for the location were highly divergent.


e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                                             4
November 10, 2011
         Using this approach initially we could develop items and through analysis develop an item pool
that contained items of known performance in the typical IRT fashion. Figure 3 illustrates the usual array
of variations in item types we initially developed. Figure 4a illustrates the performance of a typical
selection item in current use in graduate programs offices in Canada. However, on the basis of some earlier
work Brown (1998)x suggested we expand our analytic horizons and developed second order IRT to better
represent the item functioning we were finding with more sophisticated items:


                                                                   1
                                           Strength           1.7 a ( PPRP2 b )
                                                        1 e


The performance of these items is illustrated in Figures 4b and 4c. As you can see the advantage of this
item configuration is that it allows for the development of a high definition targeted selection instrument.
If the selection objective is to identify those individuals with a strongly focussed research perspective the
item of Figure 4b would be used, whereas if the target was the more desirable malleable then the item in
Figure 4c would be more desirable. Figure 5 provides some samples of the kinds of advanced items we
have now developed for inclusion in our item bank.
         However as we delved deeper into the analysis of our response data to more and more
sophisticated pppsv’s it became apparent that the simplistic unidimensional approach was insufficient to
represent our measurements. There was an obvious strong link between the aforementioned HWC that
we used for the development of the vignettes (and collected from test respondents as well) and the PPRP in
the determination of Strength. Again Brown (1999)xi came through with the remarkable insight that now
seems so obvious:

                                                               1
                                  Strength             1.7 a ( PPRP2 *HWCb )
                                                 1 e

and this is clearly illustrated in Figure 6 in which two views of the performance plots are illustrated.
Although not a classic item in a positive sense, this item shows uniformly high (maximum comfort)
strength for any individual respondent with a positive HWC with the exception of those who also have a
PPRP of 0. In other words the interaction of HWC and PPRP is clearly modelled and the model does
indeed fit both empirical data and conceptual theory. The differential functioning of HWC can be clearly
viewed in Figure 6 a and b where Strength is constant (0.5) for PPRP = 0 but is maximal for values of
HWC at 0.6b and above, and minimal for values of HWC below 0.5. This means that these items can
clearly selected for maleables regardless of the HWC baggage they may be carrying.
         The item modelled in Figure 8 is ideal for identifying complete malleables – those individuals who
have a 0.0 PPRP and a mid-range (0.5) HWC. An administration of 10 of these items should be




e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                        5
November 10, 2011
sufficient to clearly discriminate between complete malleables (these would score a 5 on strength) from
others.
             The instrumentation was intensely evaluated in a jack-knife design using data from the original 47
academics recruited for the development phase of the research. Using the initial classification of research
perspective predilection as the criterion of validation, and personal variables to estimate their item
responses (Traub, 1991)xii it was clearly demonstrated (=0.91) the accuracy of prediction of the item bank
and subsequent tests generated for selection use. We are currently in the midst of the first full-scale
application of the testing system2. Three faculties of education in universities located in diverse regions of
Canada required that all applications for enrollment in graduate programs include the results of what was
termed the Educational Research Aptitude Test – ERAT. The test was created to allow for a broad
spectrum of positive HWC and a relatively narrow band of PPRS centred at 0.0 (the Malleables). Due to
the proprietary nature of the test system none of the details of the test or the test and item statistics can be
revealed, but you can be assured that the highest ethical and psychometric standards were established and
maintained. All students applying for admission, regardless of whether they fit the target response pattern
or not, were granted entry to allow for a non-attenuated sample and subsequent fully accessible to
variation-based analysis and modeling. Early results suggest that the test is viewed by respondents as a
meaningful experience, and the faculties enjoy the excessive funding provided as an incentive for continued
participation. The initial results should be fully analyzed and reported at this time next year.
             This work has allowed for a significant breakthrough in instrumentation for the investigation of
research perspectives not only for incoming graduate students – the future old farts of the academy – but in
the fascinating almost archival research of the contemporary professorate. The correlates of post-positivist
and positivist research perspectives can not only be described and meaning made of them, but the whole
can be explained in a near mechanistic precision potentially leading to the grand theory of genesis of
research perspectives.


In Closing
             Of course this piece is highly antifoundationlist (Berube, 2000)xiii and post-modern since it invites
the reader/listener to cognitively consume both text and data to develop their own primitives leading to
whatever meaning they are to glean from both/either the document and/or the present(ation). In fact some
of you may wish engage in an antagonistic comparative of meanings that are conflicting and cohabiting in
the two texts – the paper and the presentation. Perhaps this closes in on the classic dichotomy (non-
orthogonal) in the academic dialectic and ironically on the back of behavioristic measurement.




2
    This project is funded by some of the residue of funding available from the aftermath of the HRDC Jobs Creation Initiative.


e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                                          6
November 10, 2011
                                     PRPP
                                      0 1- 2- 3- 4- 5-                  1          2     3   4        5                               Figure 2: Classic PPRP item type
                                                      0.0

                                                                                                                                     1.0
                                     2.0

                                                                                                                                     0.8




                        H TGNER TS
                                     4.0




                                                                                                                          STRENGTH
                                                                                                                                     0.6
                                     6.0

                                                                                                                                     0.4
                                     8.0

                                                                                                                                     0.2
                                     0.1


                                                      epyt meti esrevnI :1 erugiF                                                    0.0
                                                                                                                                        -5 -4 -3 -2 -1 0 1            2   3   4   5
                                                                                                                                                      PPRP




                                                                        Figure 3a: PPRP step-item type - centred

                                                                                   1.0


                                                                                   0.8
                                                                        STRENGTH




                                                                                   0.6


                                                                                   0.4


                                                                                   0.2


                                                                                   0.0
                                                                                      -3         -2       -1     0    1          2               3
                                                                                                               PPRP



                                                      Figure 3b: PPRP step-item type - PS                                                         Figure 3c: PPRP step-item type - PPS

                                                      1.0                                                                                             1.0


                                                      0.8                                                                                             0.8
                                           STRENGTH




                                                                                                                                           STRENGTH




                                                      0.6                                                                                             0.6


                                                      0.4                                                                                             0.4


                                                      0.2                                                                                             0.2


                                                      0.0                                                                                             0.0
                                                         -3        -1                    1                 3                                             -3   -1          1           3
                                                                        PPRP                                                                                       PPRP




e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                                                                                                  7
November 10, 2011
                                                                            Figure 4a: Current admissions - PS

                                                                            1.0


                                                                            0.8




                                                                 STRENGTH
                                                                            0.6


                                                                            0.4


                                                                            0.2


                                                                            0.0
                                                                               -3       -1          1                               3
                                                                                             PPRP




                               Figure 4b: High focus demand item                                                      Figure 4c: Low focus demand item

                               1.0                                                                                 1.0


                               0.8                                                                                 0.8
                    STRENGTH




                                                                                                        STRENGTH
                               0.6                                                                                 0.6


                               0.4                                                                                 0.4


                               0.2                                                                                 0.2


                               0.0                                                                                 0.0
                                  -3                -1            1                 3                                 -3                -1          1        3
                                                          PPRP                                                                               PPRP




                                                  Figure 5a: Inclusionary - Malleables                                        Figure 5b: Exclusionary - Malleables

                                                  1.0                                                                         1.0


                                                  0.8                                                                         0.8
                                       STRENGTH




                                                                                                                   STRENGTH




                                                  0.6                                                                         0.6


                                                  0.4                                                                         0.4


                                                  0.2                                                                         0.2


                                                  0.0                                                                         0.0
                                                     -3            -1               1          3                                 -3          -1          1       3
                                                                             PPRP                                                                 PPRP




e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                                                                         8
November 10, 2011
       Figure 6a: PPRS/HWC interaction item - View 1             Figure 6b: PPRS/HWC interaction item - View 2




              Figure 7a: Targeting HWC surfaces - view 1            Figure 7b: Targeting HWC surfaces - view 2




                   Figure 8a: The Complete Mealleable - view 1          Figure 8b: The Complete Mealleable - view 2




e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                     9
November 10, 2011
References

i
  Brown, R.S., McGee, M.M. & Anderson, J.O. (1992). Post Positivism and the Demise of the Mechanical Age. Victoria: West Coast
              Realities Publications.
ii
    Wilson, R.J., Shulha, L. & Kirby, J. (1993). Academic Views on Everyday Life. Ste. Catherines: GM Publications.
iii
    Anderson, J.O. (1999). Children’s Deconstructivist Activities with Timepieces. Milwaukee, WI: Westclox Research Almanac 87
iv
    McEwen, N. (1993). Data mining the deep vein of surveillance video. Applied Qualitative Methodology, 12(3), 334-367.
v
    Rogers, W.T., Maguire, T.O., Gierl, M., Ma, X., & Boughton, K. (1999). Essential dichotomy – the fold-back DIF methodology
              and validation procedure. Annals of Behavioral Measurement & Speculation, 46(2), 123-129.
vi
    Stewart, J. (2000). Archiving the National Jewels: Intensely Personal Information as Our National Heritage. Hull: Liberal Party of
              Canada.
vii
     Nagy, P. (1995). High fiber model building using linear composites and a clean mind. Memphis Methodology Quarterly, 15(5), 46-
              53.
viii
     Dassa, C. (1997). Let’s End the Hieroglyphics Now: A New Age of Communicating the Measurement of Human Perspectives.
              Alexandria: Papyrus Press.
ix
    Lord. F. (1980). Application of Item Response Theory to Practical Testing Problems. Hillsdale, NJ: Erlbaum Publications.
x
    Brown, R.S. (1998). Rasch is not enough in contemporary measurement. New Horizons in Head Testing, 21(1), 1-32.
xi
    Brown, R.S. (1999). Break the bonds of singular dimensionality in cognitive measures: Second order dimensionality.
              Contemporary Meditation and Spirituality, 61(3), 344-377.
xii
     Traub, R, (1991). Conservation in data usage – Re-use data to save the planet. Contemporary Meditation and Spirituality, 53(4),
              450-454.
xiii
     Berube, M. (2000). Teaching postmodern fiction without being sure the genrè exists. The Chronicle of Higher Education, May 19,
              2000.




e86eea4a-b64b-4734-8ae0-fb208e096e58.doc                                                                                           10
November 10, 2011

				
DOCUMENT INFO
Shared By:
Tags: study
Stats:
views:78
posted:12/8/2009
language:English
pages:10
Description: the study