Education RESEARCH

Document Sample
Education RESEARCH Powered By Docstoc
					Research Methods in
This rewritten and updated sixth edition of the long-running bestseller Research Methods in Education
covers the whole range of methods currently employed by educational research at all stages. It has five
main parts: the context of educational research, planning educational research, styles of educational
research, strategies for data collection and researching and data analysis. The book contains references
to a comprehensive dedicated web site of accompanying materials. It continues to be the standard text
for students and lecturers undertaking, understanding and using educational research.
This sixth edition comprises new material including:
   complexity theory, ethics, sampling, sensitive educational research, researching powerful people,
   Internet-based research, interviewing and surveys
   expanded coverage of, and practical guidance in, experimental research, questionnaire design and
   an entirely new part, containing five new chapters covering qualitative and quantitative data analysis
   including content analysis, grounded theory, statistics and how to use them, effect size, and reporting
   data, all with practical examples
   detailed cross-referencing to a major educational resource web site designed specifically to run
   alongside this book.
Research Methods in Education, sixth edition, is essential reading for both the professional researcher and
anyone involved in educational research.
Louis Cohen is Emeritus Professor of Education at Loughborough University, UK.
Lawrence Manion was former Principal Lecturer in Music at Didsbury School of Education, Manchester
Metropolitan University, UK.
Keith Morrison is Professor of Education at the Inter-University Institute of Macau and formerly Senior
Lecturer in Education at the University of Durham, UK.
Research Methods in

Sixth edition

Louis Cohen, Lawrence Manion
and Keith Morrison
First published 2007 by Routledge
2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN
Simultaneously published in the USA and Canada
by Routledge
270 Madison Avenue, New York, NY 10016
Routledge is an imprint of the Taylor & Francis Group, an informa business
© 2007 Louis Cohen, Lawrence Manion and Keith Morrison

This edition published in the Taylor & Francis e-Library, 2007.
“To purchase your own copy of this or any of Taylor & Francis or Routledge’s
collection of thousands of eBooks please go to”

All rights reserved. No part of this book may be reprinted or
reproduced or utilised in any form or by any electronic,
mechanical, or other means, now known or hereafter
invented, including photocopying and recording, or in any
information storage or retrieval system, without permission in
writing from the publishers.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Cataloging-in-Publication Data
A catalog record for this book has been requested

ISBN 0-203-02905-4 Master e-book ISBN

ISBN 10: 0–415–37410–3 (hbk)
ISBN 10: 0–415–36878–2 (pbk)
ISBN 10: 0–203–02905–4 (ebk)

ISBN 13: 978–0–415–37410–1 (hbk)
ISBN 13: 978–0–415–36878–0 (pbk)
ISBN 13: 978–0–203–02905–3 (ebk)
For Lawrence Manion, a wise counsellor and a good friend

    List of boxes                               xiii
    Acknowledgements                           xvii
    Introduction                                  1

    Part 1                                                 Research and evaluation                   41
    The context of educational research                    Research, politics and policy-making      46
                                                           Methods and methodology                   47

1   The nature of inquiry – Setting the
    field                                                   Part 2
    Introduction                                  5        Planning educational research
    The search for truth                          5
    Two conceptions of social reality             7
    Positivism                                    9    2   The ethics of educational and social
    The assumptions and nature of                          research
    science                                      11        Introduction                               51
    The tools of science                         14        Informed consent                           52
    The scientific method                         15        Access and acceptance                      55
    Criticisms of positivism and the scientific             The field of ethics                         58
    method                                       17        Sources of tension                         58
    Alternatives to positivistic social science:           Voices of experience                       61
    naturalistic approaches                      19        Ethical dilemmas                           62
    A question of terminology: the normative               Ethics and research methods in
    and interpretive paradigms                   21        education                                  69
    Phenomenology, ethnomethodology and                    Ethics and evaluative research             70
    symbolic interactionism                      22        Research and regulation: ethical codes and
    Criticisms of the naturalistic and interpretive        review                                     71
    approaches                                   25        Sponsored research                         74
    Critical theory and critical educational               Responsibilities to the research
    research                                     26        community                                  75
    Criticisms of approaches from critical                 Conclusion                                 75
    theory                                       29
    Critical theory and curriculum
    research                                     30
    A summary of the three paradigms             32    3   Planning educational research
    The emerging paradigm of complexity                    Introduction                              78
    theory                                       33        A framework for planning research         78
    Feminist research                            34        A planning matrix for research            87

          Managing the planning of research        93        Part 3
          A worked example                         95        Styles of educational research
          Conclusion                               98
                                                         7   Naturalistic and ethnographic research
       4 Sampling
          Introduction                             100       Elements of naturalistic inquiry        167
          The sample size                          101       Planning naturalistic research          171
          Sampling error                           106       Critical ethnography                    186
          The representativeness of the sample     108       Some problems with ethnographic and
          The access to the sample                 109       naturalistic approaches                 188
          The sampling strategy to be used         110
          Probability samples                      110   8   Historical and documentary research
          Non-probability samples                  113
                                                             Introduction                            191
          Planning a sampling strategy             117
                                                             Choice of subject                       192
          Conclusion                               117
                                                             Data collection                         193
                                                             Evaluation                              194
       5 Sensitive educational research                      Writing the research report             195
                                                             The use of quantitative methods         197
          What is sensitive research?              119
          Sampling and access                      121       Life histories                          198
                                                             Documentary research                    201
          Ethical issues in sensitive research     124
          Researching powerful people              127
          Asking questions                         130   9   Surveys, longitudinal, cross-sectional
          Conclusion                               131       and trend studies
                                                             Introduction                             205
       6 Validity and reliability                            Some preliminary considerations          207
                                                             Planning a survey                        208
          Defining validity                         133
                                                             Survey sampling                          211
          Triangulation                            141
                                                             Longitudinal, cross-sectional and trend
          Ensuring validity                        144
                                                             studies                                  211
          Reliability in quantitative research     146
                                                             Strengths and weaknesses of longitudinal,
          Reliability in qualitative research      148
                                                             cohort and cross-sectional studies       214
          Validity and reliability in interviews   150
                                                             Postal, interview and telephone
          Validity and reliability in
                                                             surveys                                  218
          experiments                              155
                                                             Event history analysis                   224
          Validity and reliability in
          questionnaires                           157
          Validity and reliability in                    10 Internet-based research and computer
          observations                             158      usage
          Validity and reliability in tests        159       Introduction                            226
          Validity and reliability in life                   Internet-based surveys                  226
          histories                                164       Internet-based experiments              239
                                                             Internet-based interviews               241
                                                                                         CONTENTS       ix

   Searching for research materials on the         14 Action research
   Internet                                  242      Introduction                                297
   Evaluating web sites                      244      Defining action research                     297
   Computer simulations                      245      Principles and characteristics of action
   Geographical Information Systems          251      research                                    299
                                                      Action research as critical praxis          302
11 Case studies                                       Procedures for action research              304
                                                      Reflexivity in action research               310
   What is a case study?                     253
                                                      Some practical and theoretical
   Examples of kinds of case study           258
                                                      matters                                     311
   Why participant observation?              260
                                                      Conclusion                                  312
   Recording observations                    260
   Planning a case study                     261
   Writing up a case study                   262      Part 4
   Conclusion                                263      Strategies for data collection and
12 Ex post facto research
   Introduction                              264   15 Questionnaires
   Co-relational and criterion groups                 Introduction                                317
   designs                                   265
                                                      Ethical issues                              317
   Characteristics of ex post facto
                                                      Approaching the planning of a
   research                                  266
                                                      questionnaire                               318
   Occasions when appropriate                268
                                                      Types of questionnaire items                321
   Advantages and disadvantages of
                                                      Asking sensitive questions                  333
   ex post facto research                    268
                                                      Avoiding pitfalls in question writing       334
   Designing an ex post facto
                                                      Sequencing the questions                    336
   investigation                             269
                                                      Questionnaires containing few verbal
   Procedures in ex post facto research      270
                                                      items                                       337
                                                      The layout of the questionnaire             338
13 Experiments, quasi-experiments,                    Covering letters or sheets and follow-up
   single-case research and meta-analysis             letters                                     339
   Introduction                            272        Piloting the questionnaire                  341
   Designs in educational                             Practical considerations in questionnaire
   experimentation                         274        design                                      342
   True experimental designs               275        Administering questionnaires                344
   A quasi-experimental design: the                   Processing questionnaire data               346
   non-equivalent control group design     282
   Single-case research: ABAB design       284
                                                   16 Interviews
   Procedures in conducting experimental
   research                                285        Introduction                                349
   Examples from educational research      287        Conceptions of the interview                349
   Evidence-based educational research and            Purposes of the interview                   351
   meta-analysis                           289        Types of interview                          352

       Planning interview-based research                 Parametric and non-parametric tests        414
       procedures                               356      Norm-referenced, criterion-referenced
       Group interviewing                       373      and domain-referenced tests                415
       Interviewing children                    374      Commercially produced tests and
       Focus groups                             376      researcher-produced tests                  416
       The non-directive interview and the               Constructing a test                        418
       focused interview                        377      Devising a pretest and post-test           432
       Telephone interviewing                   379      Reliability and validity of tests          432
       Ethical issues in interviewing           382      Ethical issues in preparing for tests      432
                                                         Computerized adaptive testing              433
    17 Accounts
                                                      20 Personal constructs
       Introduction                             384
       The ethogenic approach                   384      Introduction                              435
       Characteristics of accounts and                   Characteristics of the method             435
       episodes                                 384      ‘Elicited’ versus ‘provided’ constructs   436
       Procedures in eliciting, analysing and            Allotting elements to constructs          437
       authenticating accounts: an example      385      Laddering and pyramid constructions       439
       Network analyses of qualitative data     388      Grid administration and analysis          439
       What makes a good network?               388      Procedures in grid administration         439
       Discourse analysis                       389      Procedures in grid analysis               439
       Analysing social episodes                391      Strengths of repertory grid technique     442
       Account gathering in educational                  Difficulties in the use of repertory grid
       research: an example                     391      technique                                 442
       Problems in gathering and analysing               Some examples of the use of repertory
       accounts                                 392      grid in educational research              443
       Strengths of the ethogenic approach      393      Grid technique and audio/video lesson
       A note on stories                        394      recording                                 445
                                                         Focused grids, non-verbal grids, exchange
                                                         grids and sociogrids                      446
    18 Observation
       Introduction                             396   21 Role-playing
       Structured observation                   398
       Critical incidents                       404      Introduction                               448
       Naturalistic and participant                      Role-playing versus deception: the
       observation                              404      argument                                   450
       Natural and artificial settings for                Role-playing versus deception: the
       observation                              408      evidence                                   451
       Ethical considerations                   408      Role-playing in educational settings       452
       Some cautionary comments                 410      The uses of role-playing                   452
       Conclusion                               412      Strengths and weaknesses of role-playing
                                                         and other simulation exercises             455
                                                         Role-playing in an educational setting:
    19 Tests                                             an example                                 455
       Introduction                             414      Evaluating role-playing and other
       What are we testing?                     414      simulation exercises                       456
                                                                                       CONTENTS      xi

   Part 5                                            Reliability                               506
   Data analysis                                     Exploratory data analysis: frequencies,
                                                     percentages and cross-tabulations         506
22 Approaches to qualitative data analysis           Statistical significance                   515
                                                     Hypothesis testing                        519
                                                     Effect size                               520
   Introduction                             461
                                                     The chi-square test                       525
   Tabulating data                          463
                                                     Degrees of freedom                        527
   Five ways of organizing and presenting
                                                     Measuring association                     528
   data analysis                            467
                                                     Regression analysis                       536
   Systematic approaches to data
                                                     Measures of difference between groups
   analysis                                 469
                                                     and means                                 542
   Methodological tools for analysing
   qualitative data                         473
                                                  25 Multidimensional measurement and
                                                     factor analysis
23 Content analysis and grounded theory
                                                     Introduction                              559
   Introduction                             475
                                                     Elementary linkage analysis: an
   What is content analysis?                475
                                                     example                                   559
   How does content analysis work?          476
                                                     Factor analysis                           560
   A worked example of content
                                                     Factor analysis: an example               570
   analysis                                 483
                                                     Examples of studies using
   Computer usage in content analysis       487
                                                     multidimensional scaling and cluster
   Reliability in content analysis          490
                                                     analysis                                  576
   Grounded theory                          491
                                                     Multidimensional data: some words on
   Interpretation in qualitative data
                                                     notation                                  579
   analysis: multilayered texts             495
                                                     Multilevel modelling                      583
                                                     Cluster analysis                          584
24 Quantitative data analysis
   Introduction                             501   26 Choosing a statistical test
   Scales of data                           502
                                                     How many samples?                         586
   Parametric and non-parametric data       503
                                                     Assumptions of tests                      591
   Descriptive and inferential statistics   503
   One-tailed and two-tailed tests          504      Notes                                     593
   Dependent and independent                         Bibliography                              599
   variables                                504      Index                                     633

 1.1 The subjective–objective                        4.2 Distribution of sample means showing
     dimension                                  9        the spread of a selection of sample means
 1.2 Alternative bases for interpreting social           around the population mean                107
     reality                                   10    5.1 Issues of sampling and access in
 1.3 The functions of science                  12        sensitive research                        124
 1.4 The hypothesis                            15    5.2 Ethical issues in sensitive research      127
 1.5 Stages in the development of                    5.3 Researching powerful people               130
     science                                   16    5.4 Key questions in considering sensitive
 1.6 An eight-stage model of the scientific               educational research                      132
     method                                    16    6.1 Principal sources of bias in life history
 1.7 A classroom episode                       20        research                                  164
 1.8 Differing approaches to the study of            8.1 Some historical interrelations between
     behaviour                                 33        men, movements and institutions           192
 2.1 The costs/benefits ratio                   52    8.2 A typology of life histories and their
 2.2 Guidelines for reasonably informed                  modes of presentation                     199
     consent                                   53
                                                     9.1 Stages in the planning of a survey        210
 2.3 Close encounters of a researcher
     kind                                      56    9.2 Types of developmental research           214
 2.4 Conditions and guarantees proffered             9.3 Advantages of cohort over cross-
     for a school-based research project       57        sectional designs                         217
 2.5 Negotiating access checklist              59    9.4 The characteristics, strengths and
 2.6 Absolute ethical principles in social               weaknesses of longitudinal,
     research                                  61        cross-sectional, trend analysis, and
 2.7 An extreme case of deception              67        retrospective longitudinal studies        219
 2.8 Ethical principles for the guidance of         10.1 Problems and solutions in
     action researchers                        70        Internet-based surveys                    231
 2.9 An ethical code: an illustration          76   10.2 Geographical Information Systems in
2.10 Ethical principles for educational                  secondary schools                         251
     research (to be agreed before the research     10.3 Location of home postcodes using
     commences)                                77        Geographical Information
 3.1 The elements of research design           79        Systems                                   252
 3.2 Elements of research styles               84   11.1 Possible advantages of case study         256
 3.3 A matrix for planning research            88   11.2 Strengths and weaknesses of case
 3.4 A planning sequence for research          94        study                                     256
 3.5 A planning matrix for research            95   11.3 A typology of observation studies         259
 3.6 Understanding the levels of                    11.4 The case study and problems of
     organizational culture                    96        selection                                 261
 4.1 Sample size, confidence levels and              11.5 Continua of data collection, types
     confidence intervals for random                      and analysis in case study
     samples                                 104         research                                  262
xiv   BOXES

      13.1 Independent and dependent                     17.6 Parents and teachers: divergent
           variables                               273        viewpoints on children’s
      13.2 The effects of randomization            276        communicative competence                 393
      13.3 Interaction effects in an                     17.7 Justification of objective systematic
           experiment                              281        observation in classroom settings        394
      13.4 The ABAB design                         285   18.1 A structured observation schedule        399
      13.5 An ABAB design in an educational              18.2 Non-participant observation:
           setting                                 286        a checklist of design tasks              401
      13.6 Class size and learning in well-              18.3 Structured, unstructured, natural and
           controlled and poorly controlled                   artificial settings for observation       409
           studies                                 296   19.1 A matrix of test items                   419
      14.1 A model of emancipatory action                19.2 Compiling elements of test items         420
           research for organizational change      306   20.1 Eliciting constructs and constructing
      15.1 A flow chart technique for question                 a repertory grid                         436
           planning                                319   20.2 Allotting elements to constructs:
      15.2 A guide for questionnaire                          three methods                            438
           construction                            320   20.3 Laddering                                440
      15.3 A 10-point marking scale in a                 20.4 Elements                                 440
           questionnaire                           330   20.5 Difference score for constructs          441
      15.4 Potential problems in conducting              20.6 Grid matrix                              441
           research                                332   21.1 Dimensions of role-play methods          449
      15.5 A flow chart for the planning of a             21.2 The Stanford Prison experiment           450
           postal survey                           347   21.3 Critical factors in a role-play: smoking
      16.1 Attributes of ethnographers as                     and young people                         454
           interviewers                            350   21.4 Categorization of responses to four
      16.2 Summary of relative merits of                      video extracts                           456
           interview versus questionnaire          352   22.1 The effectiveness of English
      16.3 Strengths and weaknesses of different              teaching                                 464
           types of interview                      353   22.2 The strengths and weaknesses of
      16.4 The selection of response mode          360        English language teaching                464
      16.5 Guidelines for the conduct of                 22.3 Teaching methods                         465
           interviews                              366   22.4 Student-related factors                  466
      16.6 Delineating units of general                  24.1 Frequencies and percentages for a
           meaning                                 371        course evaluation                        507
      16.7 Units of relevant meaning               371   24.2 Cross-tabulation by totals               508
      16.8 Clusters of relevant meaning            372   24.3 Cross-tabulation by row totals           509
      17.1 Principles in the ethogenic                   24.4 Rating scale of agreement and
           approach                                385        disagreement                             510
      17.2 Account gathering                       386   24.5 Satisfaction with a course               510
                                                         24.6 Combined categories of rating
      17.3 Experience sampling method              387
                                                              scales                                   510
      17.4 Concepts in children’s talk             390   24.7 Representing combined categories of
      17.5 ‘Ain’t nobody can talk about things                rating scales                            511
           being about theirselves’                392
                                                                                              BOXES     xv

24.8    How well learners are cared for,             24.32 Significance level in regression
        guided and supported                   511         analysis                               539
24.9    Staff voluntarily taking on                  24.33 The beta coefficient in a regression
        coordination roles                     511         analysis                               539
24.10   Distribution of test scores            512   24.34 A summary of the R, R square and
24.11   A line graph of test scores            512         adjusted R square in multiple
24.12   Distribution around a mean with an                 regression analysis                    540
        outlier                                513   24.35 Significance level in multiple
24.13   A platykurtic distribution of                      regression analysis                    541
        scores                                 513   24.36 The beta coefficients in a multiple
24.14   A leptokurtic distribution of                      regression analysis                    541
        scores                                 514   24.37 Means and standard deviations for a
24.15   Type I and Type II errors              520         t-test                                 544
24.16   Mean and standard deviation in an            24.38 The Levene test for equality of
        effect size                            522         variances in a t-test                  544
24.17   The Levene test for equality of              24.39 A t-test for leaders and teachers      545
        variances                              523   24.40 The Levene test for equality of
24.18   Mean and standard deviation in a                   variances between leaders and
        paired sample test                     523         teachers                               545
24.19   Difference test for a paired sample    524   24.41 Means and standard deviations in a
24.20   Effect size in analysis of variance    524         paired samples t-test                  546
24.21   A 2 × 3 contingency table for                24.42 The paired samples t-test              547
        chi-square                             526   24.43 Descriptive statistics for analysis of
24.22   A 2 × 5 contingency table for                      variance                               548
        chi-square                             527   24.44 SPSS output for one-way analysis of
                                                           variance                               548
24.23   Common measures of relationship        529
                                                     24.45 The Tukey test                         549
24.24   Percentage of public library members         24.46 Homogeneous groupings in the Tukey
        by their social class origin           529         test                                   549
24.25   Correlation scatterplots               532   24.47 Means and standard deviations in a
24.26   A Pearson product-moment                           two-way analysis of variance           551
        correlation                            533   24.48 The Levene test of equality of
24.27   A line diagram to indicate                         variances in a two-way analysis of
        curvilinearity                         533         variance                               551
24.28   Visualization of correlation of 0.65         24.49 Between-subject effects in two-way
        between reading grade and arithmetic               analysis of variance                   552
        grade                                  535   24.50 Graphic plots of two sets of scores on
24.29   A scatterplot with the regression                  a dependent variable                   552
        line                                   537   24.51 A cross-tabulation for a
24.30   A scatterplot with grid lines and                  Mann-Whitney U test                    553
        regression line                        538   24.52 SPSS output on rankings for the
24.31   A summary of the R, R square and                   Mann-Whitney U test                    553
        adjusted R square in regression              24.53 The Mann-Whitney U value and
        analysis                               538         significance level in SPSS              553
xvi   BOXES

      24.54 Frequencies and percentages of               25.10 Factor analysis of the occupational
            variable one in a Wilcoxon test        554         satisfaction items                          574
      24.55 Frequencies and percentages of               25.11 Correlations between (dependent)
            variable two in a Wilcoxon test        554         stress and (independent) satisfaction
      24.56 Ranks and sums of ranks in a                       factors and canonical variates              575
            Wilcoxon test                          555   25.12 Biographical data and stress
      24.57 Significance level in a Wilcoxon                    factors                                     576
            test                                   555   25.13 Students’ perceptions of social
      24.58 Cross-tabulation for the                           episodes                                    577
            Kruskal-Wallis test                    556   25.14 Perception of social episodes               579
      24.59 Rankings for the Kruskal-Wallis              25.15 Person concept coding system                580
            test                                   556   25.16 Reliability coefficients for peer
      24.60 Significance levels in a Kruskal-Wallis             descriptions                                581
            test                                   556
                                                         25.17 Sex, voting preference and social
      24.61 Frequencies for variable one in the
                                                               class: a three-way classification
            Friedman test                          557
                                                               table                                       581
      24.62 Frequencies for variable two in the          25.18 Sex, voting preference and social
            Friedman test                          557         class: a three-way notational
      24.63 Frequencies for variable three in the              classification                               581
            Friedman test                          558   25.19 Expected frequencies in sex, voting
      24.64 Rankings for the Friedman test         558         preference and social class                 581
      24.65 Significance level in the Friedman            25.20 Expected frequencies assuming that
            test                                   558         sex is independent of social class and
      25.1 Rank ordering of ten children on                    voting preference                           582
            seven constructs                       561
                                                         25.21 Sex and voting preference: a two-way
      25.2 Intercorrelations between seven
                                                               classification table                         583
            personal constructs                    562
                                                         25.22 Cluster analysis                            585
      25.3 The structuring of relationships
                                                         26.1 Identifying statistical tests for an
            among the seven personal
                                                               experiment                                  587
            constructs                             563
      25.4 Initial SPSS output for principal             26.2 Statistical tests to be used with
            components analysis                    564         different numbers of groups of
                                                               samples                                     587
      25.5 A scree plot                            565
                                                         26.3 Types of statistical tests for four scales
      25.6 Three-dimensional rotation              566
                                                               of data                                     588
      25.7 The rotated components matrix in
                                                         26.4 Choosing statistical tests for
            principal components analysis          567
                                                               parametric and non-parametric
      25.8 Factor analysis of responsibility for               data                                        589
            stress items                           571   26.5 Statistics available for different types
      25.9 Factor analysis of the occupational                 of data                                     590
            stress items                           573   26.6 Assumptions of statistical tests             592

Our thanks are due to the following publishers and authors for permission to include materials in the

Allyn & Bacon/Pearson Education, for material from      Lawrence Erlbaum Associates, for material from
   Best, J. W. (1970) Research in Education.               Murphy, J., John, M. and Brown, H.
Blackwell Publishers, for material from Dyer, C.           (eds) (1984) Dialogues and Debates in
   (1995) Beginning Research in Psychology;                Social Psychology. London: Lawrence Erlbaum
   Robson, C. (1993) Real World Research;                  Associates.
   Robson, C. (2002) Real World Research (second        McAleese, R. and Hamilton, D. (eds) (1978)
   edition).                                               Understanding Classroom Life. Slough: National
British Psychological Society, for material from           Foundation for Educational Research.
   Adams-Webber, J. R. (1970) Elicited versus           Multilingual Matters Ltd, Clevedon, for figures from
   provided constructs in repertory grid technique:        Parsons, E., Chalkley, B. and Jones, A. (1996)
   a review, British Journal of Medical Psychology,        The role of Geographic Information Systems in
   43, 349–54. Reproduced with permission from             the study of parental choice and secondary
   the British Journal of Medical Psychology © The         school catchments, Evaluation and Research
   British Psychological Society.                          in Education, 10(1), 23–34; for words from
Campbell, D. T. and Stanley, J. C. Experimental            Stronach, I. and Morris, B (1994) Polemical
   and Quasi-Experimental Designs for Research.            notes on educational evaluation in an age
   Copyright © 1963 by Houghton Mifflin                     of ‘policy hysteria’, Evaluation and Research in
   Company.                                                Education, 8(1), 5–19.
Continuum Books, for material from Walford, G.          Patton, M. Q. (1980) Qualitative Evaluation Meth-
   (2001) Doing Qualitative Educational Research,          ods, p. 206, copyright © Sage Publications Inc.,
   pp. 30, 31, 36, 137.                                    reprinted by permission of Sage Publications
Deakin University Press, Deakin, Australia, for words      Inc.
   from Kemmis, S. and McTaggart, R. (1981)             Pearson Education Ltd, for material from Harris, N.,
   The Action Research Planner, and Kemmis, S.             Pearce, P. and Johnstone, S. (1992) The Legal
   and McTaggart, R. (1992) The Action Research            Context of Teaching.
   Planner (third edition) 8 and 21–8.                  Penguin Group UK, for material from Armis-
Elsevier, for material reprinted from International        tead, N. (1974) Reconstructing Social Psychology.
   Journal of Educational Research, vol. 18(3),         Prentice-Hall, for material from Garfinkel, H.
   Edwards, D. Concepts, memory and the or-                (1974) Studies in Ethnomethodology; Smith,
   ganisation of pedagogic discourse, pp. 205–25,          R. W. (1978) Strategies in Social Research.
   copyright © 1993, with permission from Else-         Princeton University Press, for material from
   vier; Social Method and Social Life, M. Brenner         Kierkegaard, S. (1974) Concluding Unscientific
   (ed.), article by J. Brown and J. Sime: A               Postscript.
   methodology for accounts, p. 163, copyright          Reips, U.-D. (2002a) Internet-based psychological
   © 1981, with permission from Elsevier.                  experimenting: five dos and don’ts. Social Sci-
Hughes, J. (1976), for material from Sociological          ence Computer Review, 20(3), 241–9; (2002b)
   Analysis: Methods of Discovery, Nelson Thornes,         Standards for Internet-based experimenting.
   p. 34.                                                  Experimental Psychology, 49(4), 243–56.

        Springer, for Hycner, R. H. (1985) Some guidelines      and Solman, R. (1992) Teachers’ attributions
           for the phenomenological analysis of interview       of responsibility for occupational stress and
           data, Human Studies, 8, 279–303, with kind           satisfaction: an organisational perspective,
           permission of Springer Science and Business          Educational Studies, 18(92), 201–22; McNiff, J.
           Media.                                               (2002) Action Research: Principles and Practice
        Stanford University Press, for material from            (second edition), pp. 85–91; Medawar, P.
           Sears, R., Maccoby, E. and Levin, H. (1976)          (1972) The Hope of Progress; Oldroyd, G. (1986)
           Patterns of Child Rearing (originally published      The Arch of Knowledge: An Introductory Study of
           1957).                                               the History of the Philosophy and Methodology of
        Taylor & Francis, for Brenner, M. and Marsh, P.         Science; Plummer, K. (1983) Documents of Life:
           (eds) (1978) The Social Contexts of Method;          An Introduction to the Problems and Literature of
           Burgess, R. (ed.) (1993) Educational Research        a Humanistic Method; Rex, J. (1974) Approaches
           for Policy and Practice, pp. 119 and 135;            to Sociology; Simons, H. and Usher, R. (2000)
           Burgess, R. (ed.) (1985) Issues in Educational       Situated Ethics in Educational Research, pp. 1–2;
           Research, pp. 116–28 and 244–7; Burgess, R.          Walford, G. (1994) Researching the Powerful
           (ed.) (1989) The Ethics of Educational Research,     in Education; Zuber-Skerritt, O. (1996) New
           p. 194; Cuff, E. G. and Payne, G. (1979)             Directions in Action Research, p. 99; Winter, R.
           Perspectives in Sociology, p. 4; Hammersley, M.      (1982) Dilemma analysis: a contribution to
           and Atkinson, P. (1983) Ethnography: Principles      methodology for action research, Cambridge
           and Practice, pp. 18, 19, 76; Hitchcock, G.          Journal of Education, 12(3), 161–74.
           and Hughes, D. (1995) Research and the             University of Chicago Press, for brief quotations
           Teacher (second edition), pp. 20–2, 41;              from Whyte, W. F. (1993) Street Corner Society,
           Kincheloe, J. (2003) Teachers as Researchers:        pp. 292, 301, 303; Merton, K. and Kendall,
           Qualitative Inquiry as a Path to Empowerment         P. L. (1946) The focused interview. American
           (second edition), pp. 138–9; McCormick, J.           Journal of Sociology, 51, 541–57.

It is seven years since the fifth edition of Research      for materials, computer simulations and
Methods in Education was published and we are             Geographical Information Systems
indebted to Routledge for the opportunity to              very considerably expanded coverage of ex-
produce a sixth edition. The book continues to            perimental research, reflecting the resurgence
be received very favourably worldwide and is              of interest in this method in evidence-based
the standard text for many courses in research            education.
   The sixth edition contains much new material,       Part Four:
including a completely new part on data analysis.         more detailed coverage of questionnaire design
This means that the book now covers all stages of         and administration, with practical guidance on
educational research, from planning and design,           these matters
through data collection to data analysis and              interviewing children and telephone inter-
reporting. While retaining the best features of           viewing.
the former edition, the reshaping, updating and
new additions undertaken for this new volume           Part Five:
now mean that the book covers a greater spread
                                                          an entirely new part, containing five new
of issues than the previous editions. In particular,
                                                          chapters, covering qualitative and quantitative
the following new material has been included:
                                                          data analysis
Part One:                                                 how to conduct a content analysis
                                                          grounded theory and ‘how to do it’
   feminist theory                                        how to present and report qualitative data
   complexity theory and educational research.            computer usage in qualitative data analysis
Part Two:                                                 an introduction to statistics and statistical
   ethical codes and responsibilities to sponsors         hypotheses and how to test them
   and the research community                             variables and how to handle them
   informed consent and deception                         effect size and how to calculate and interpret it
   sampling, confidence levels and confidence               practical ‘hands on’ advice for novice
   intervals, together with the calculation of            researchers, on which statistics to choose and
   sample sizes                                           how to use them, from the simplest statistics
   an entirely new chapter on planning and                to high-level factor analysis and multiple
   conducting sensitive educational research,             regression, and from descriptive to inferential
   including researching powerful people.                 statistics
                                                          advice on how to select appropriate statistics,
Part Three:
                                                          with charts and diagrams to ease selection
   further coverage of documentary research               how to avoid selecting incorrect statistics, and
   postal, interview and telephone surveys                what are the assumptions underlying the main
   an entirely new chapter on Internet-based              kinds of statistics
   research and computer usage, covering Internet         plentiful examples of statistics and how to
   surveys, experiments, interviews, questionnaire        interpret them, with worked examples that use
   design, evaluation of web sites, searching             SPSS output and processing (the Statistical

       Package for the Social Sciences (SPSS) is the       manual for novice researchers, QSR data files and
       most widely used statistical package in the         manual for qualitative data treatment, together
       social sciences).                                   with further statistics and statistical tables.
                                                           (Qualitative Solutions and Research (QSR)
    Additionally there are copious web site references     is a company which had produced software
    in nearly every chapter, most of which provide         such as N-Vivo for qualitative data analysis.)
    free online materials. A signal feature of this        These are indicated in the book. A wealth
    edition is the inclusion of several worked examples,   of supporting materials is available on the
    particularly in the chapters on data analysis in the   web site.
    new Part Five.                                            We have refined the referencing, relocating
       To accompany this volume, a companion web           several backup references to the Notes, thereby
    site provides a comprehensive range of materials       indicating in the main text the most prominent
    to cover all aspects of research (including a full     sources and key issues.
    course on research methods on PowerPoint slides),         We hope that this volume will continue to
    exercises and examples, explanatory material           constitute the first port of call for educational
    and further notes, SPSS data files and SPSS             researchers.
Part One

The context of educational

This part locates the research enterprise in          critical theory links the conduct of educational
several contexts. It commences with positivist        research with politics and policy-making, and this
and scientific contexts of research and then           is reflected in the discussions here of research
proceeds to show the strengths and weaknesses         and evaluation, arguing how much educational
of such traditions for educational research.          research has become evaluative in nature. A more
As an alternative paradigm, the cluster of            recent trend has been the rise of complexity theory,
approaches that can loosely be termed interpretive,   originally from the natural sciences, but moving
naturalistic, phenomenological, interactionist and    inexorably into social science research. This part
ethnographic are brought together and their           introduces the field of complexity theory and steers
strengths and weaknesses for educational research     readers to the accompanying web site for further
are examined. The rise of critical theory             details. That educational research serves a political
as a paradigm in which educational research           agenda is seen in the later sections of this part.
is conducted has been spectacular and its             The intention here is to introduce readers to
implications for the research undertaking are         different research traditions, with the advice that
addressed in several ways here, resonating with       ‘fitness for purpose’ must be the guiding principle:
curriculum research and feminist research (this       different research paradigms for different research
too has been expanded and updated). Indeed            purposes.
1        The nature of inquiry – Setting the field

Introduction                                          out to achieve these ends may be classified
                                                      into three broad categories: experience, reasoning
This chapter explores the context of educational
                                                      and research (Mouly 1978). Far from being
research. It sets out several foundations on
                                                      independent and mutually exclusive, however,
which different kinds of empirical research are
                                                      these categories must be seen as complementary
                                                      and overlapping, features most readily in evidence
   scientific and positivistic methodologies           where solutions to complex modern problems are
   naturalistic and interpretive methodologies        sought.
   methodologies from critical theory                    In our endeavours to come to terms with the
   feminist educational research.                     problems of day-to-day living, we are heavily
                                                      dependent upon experience and authority. It
Our analysis takes an important notion                must be remembered that as tools for uncovering
from Hitchcock and Hughes (1995: 21) who sug-         ultimate truth they have decided limitations. The
gest that ontological assumptions give rise to        limitations of personal experience in the form of
epistemological assumptions; these, in turn, give     common-sense knowing, for instance, can quickly
rise to methodological considerations; and these,     be exposed when compared with features of the
in turn, give rise to issues of instrumentation and   scientific approach to problem-solving. Consider,
data collection. This view moves us beyond regard-    for example, the striking differences in the way
ing research methods as simply a technical exercise   in which theories are used. Laypeople base them
and as concerned with understanding the world;        on haphazard events and use them in a loose
this is informed by how we view our world(s), what    and uncritical manner. When they are required to
we take understanding to be, and what we see as       test them, they do so in a selective fashion, often
the purposes of understanding. The chapter also       choosing only that evidence that is consistent with
acknowledges that educational research, politics      their hunches and ignoring that which is counter
and decision-making are inextricably intertwined,     to them. Scientists, by contrast, construct their
and it draws attention to the politics of educa-      theories carefully and systematically. Whatever
tional research and the implications that this has    hypotheses they formulate have to be tested
for undertaking research (e.g. the move towards       empirically so that their explanations have a firm
applied and evaluative research and away from         basis in fact. And there is the concept of control
‘pure’ research). Finally, we add a note about        distinguishing the layperson’s and the scientist’s
methodology.                                          attitude to experience. Laypeople generally make
                                                      no attempt to control any extraneous sources of
                                                      influence when trying to explain an occurrence.
The search for truth                                  Scientists, on the other hand, only too conscious of
People have long been concerned to come to            the multiplicity of causes for a given occurrence,
grips with their environment and to understand        resort to definite techniques and procedures to
the nature of the phenomena it presents to            isolate and test the effect of one or more of the
their senses. The means by which they set             alleged causes. Finally, there is the difference of

    attitude to the relationships among phenomena.        inevitably bias the conclusions, he proposed in its
    Laypeople’s concerns with such relationships are      place the method of inductive reasoning by means
    loose, unsystematic and uncontrolled. The chance      of which the study of a number of individual
    occurrence of two events in close proximity is        cases would lead to an hypothesis and eventually
    sufficient reason to predicate a causal link between   to a generalization. Mouly (1978) explains it
    them. Scientists, however, display a much more        by suggesting that Bacon’s basic premise was
    serious professional concern with relationships       that, with sufficient data, even if one does not
    and only as a result of rigorous experimentation      have a preconceived idea of their significance or
    will they postulate a relationship between two        meaning, nevertheless important relationships and
    phenomena.                                            laws would be discovered by the alert observer.
       People attempt to comprehend the world             Bacon’s major contribution to science was thus
    around them by using three types of reasoning:        that he was able to rescue it from the death-
    deductive reasoning, inductive reasoning and the      grip of the deductive method whose abuse had
    combined inductive-deductive approach. Deductive      brought scientific progress to a standstill. He
    reasoning is based on the syllogism which was         thus directed the attention of scientists to nature
    Aristotle’s great contribution to formal logic.       for solutions to people’s problems, demanding
    In its simplest form the syllogism consists of a      empirical evidence for verification. Logic and
    major premise based on an a priori or self-evident    authority in themselves were no longer regarded
    proposition, a minor premise providing a particular   as conclusive means of proof and instead became
    instance, and a conclusion. Thus:                     sources of hypotheses about the world and its
      All planets orbit the sun.
                                                             Bacon’s inductive method was eventually
      The earth is a planet.
                                                          followed by the inductive-deductive approach
      Therefore the earth orbits the sun.
                                                          which combines Aristotelian deduction with
    The assumption underlying the syllogism is that       Baconian induction. Here the researcher is
    through a sequence of formal steps of logic, from     involved in a back-and-forth process of induction
    the general to the particular, a valid conclusion     (from observation to hypothesis) and deduction
    can be deduced from a valid premise. Its chief        (from hypothesis to implications) (Mouly 1978).
    limitation is that it can handle only certain         Hypotheses are tested rigorously and, if necessary,
    kinds of statement. The syllogism formed the          revised.
    basis of systematic reasoning from the time of           Although both deduction and induction have
    its inception until the Renaissance. Thereafter       their weaknesses, their contributions to the
    its effectiveness was diminished because it was       development of science are enormous and fall
    no longer related to observation and experience       into three categories:
    and became merely a mental exercise. One of the
    consequences of this was that empirical evidence         the suggestion of hypotheses
    as the basis of proof was superseded by authority        the logical development of these hypotheses
    and the more authorities one could quote, the            the clarification and interpretation of scientific
    stronger one’s position became. Naturally, with          findings and their synthesis into a conceptual
    such abuse of its principal tool, science became         framework.
       The history of reasoning was to undergo a             A further means by which we set out to discover
    dramatic change in the 1600s when Francis Bacon       truth is research. This has been defined by Kerlinger
    began to lay increasing stress on the observational   (1970) as the systematic, controlled, empirical and
    basis of science. Being critical of the model of      critical investigation of hypothetical propositions
    deductive reasoning on the grounds that its major     about the presumed relations among natural
    premises were often preconceived notions which        phenomena. Research has three characteristics in
                                                                TWO CONCEPTIONS OF SOCIAL REALITY               7

particular which distinguish it from the first means     Two conceptions of social reality

                                                                                                                    Chapter 1
of problem-solving identified earlier, namely,
experience. First, whereas experience deals with        The views of social science that we have just
events occurring in a haphazard manner, research        identified represent strikingly different ways of
is systematic and controlled, basing its operations     looking at social reality and are constructed on
on the inductive-deductive model outlined above.        correspondingly different ways of interpreting it.
Second, research is empirical. The scientist turns      We can perhaps most profitably approach these
to experience for validation. As Kerlinger (1970)       conceptions of the social world by examining the
puts it, subjective, personal belief has to have        explicit and implicit assumptions underpinning
a reality check against objective, empirical facts      them. Our analysis is based on the work of Burrell
and tests. And third, research is self-correcting.      and Morgan (1979), who identified four sets of
Not only does the scientific method have built-in        such assumptions.
mechanisms to protect scientists from error as far         First, there are assumptions of an ontological
as is humanly possible, but also their procedures       kind – assumptions which concern the very nature
and results are open to public scrutiny by fellow       or essence of the social phenomena being
professionals. Incorrect results in time will be        investigated. Thus, the authors ask, is social
found and either revised or discarded (Mouly            reality external to individuals – imposing itself on
1978). Research is a combination of both                their consciousness from without – or is it the
experience and reasoning and must be regarded           product of individual consciousness? Is reality of
as the most successful approach to the discovery        an objective nature, or the result of individual
of truth, particularly as far as the natural sciences   cognition? Is it a given ‘out there’ in the world, or
are concerned (Borg 1963).1                             is it created by one’s own mind? These questions
   Educational research has absorbed several com-       spring directly from what philosophy terms the
peting views of the social sciences – the es-           nominalist–realist debate. The former view holds
tablished, traditional view and an interpretive         that objects of thought are merely words and
view, and several others that we explore in this        that there is no independently accessible thing
chapter – critical theory, feminist theory and com-     constituting the meaning of a word. The realist
plexity theory. The established, traditional view       position, however, contends that objects have an
holds that the social sciences are essentially the      independent existence and are not dependent for
same as the natural sciences and are therefore          it on the knower.
concerned with discovering natural and universal           The second set of assumptions identified
laws regulating and determining individual and          by Burrell and Morgan (1979) are of an
social behaviour; the interpretive view, however,       epistemological kind. These concern the very
while sharing the rigour of the natural sciences        bases of knowledge – its nature and forms, how
and the same concern of traditional social science      it can be acquired, and how communicated to
to describe and explain human behaviour, em-            other human beings. How one aligns oneself in
phasizes how people differ from inanimate natural       this particular debate profoundly affects how one
phenomena and, indeed, from each other. These           will go about uncovering knowledge of social
contending views – and also their corresponding         behaviour. The view that knowledge is hard,
reflections in educational research – stem in the        objective and tangible will demand of researchers
first instance from different conceptions of social      an observer role, together with an allegiance to the
reality and of individual and social behaviour. It      methods of natural science; to see knowledge as
will help our understanding of the issues to be         personal, subjective and unique, however, imposes
developed subsequently if we examine these in a         on researchers an involvement with their subjects
little more detail (see       and a rejection of the ways of the natural scientist.
textbooks/9780415368780 – Chapter 1, file 1.1.           To subscribe to the former is to be positivist; to
ppt).                                                   the latter, anti-positivist.

       The third set of assumptions concern human          underlying themes in a search for universal laws
    nature and, in particular, the relationship between    that explain and govern that which is being
    human beings and their environment. Since the          observed (Burrell and Morgan 1979). An approach
    human being is both its subject and object of study,   characterized by procedures and methods designed
    the consequences for social science of assumptions     to discover general laws may be referred to as
    of this kind are indeed far-reaching. Two images of    nomothetic.
    human beings emerge from such assumptions – the           However, if one favours the alternative view
    one portrays them as responding mechanically           of social reality which stresses the importance of
    and deterministically to their environment, i.e.       the subjective experience of individuals in the
    as products of the environment, controlled like        creation of the social world, then the search
    puppets; the other, as initiators of their own         for understanding focuses upon different issues
    actions with free will and creativity, producing       and approaches them in different ways. The
    their own environments. The difference is between      principal concern is with an understanding of
    determinism and voluntarism respectively (Burrell      the way in which the individual creates, modifies
    and Morgan 1979).                                      and interprets the world in which he or she
       It would follow from what we have said so far       finds himself or herself. The approach now takes
    that the three sets of assumptions identified above     on a qualitative as well as quantitative aspect.
    have direct implications for the methodological        As Burrell and Morgan (1979) and Kirk and Miller
    concerns of researchers, since the contrasting         (1986: 14) observe, emphasis here is placed on
    ontologies, epistemologies and models of human         explanation and understanding of the unique and
    beings will in turn demand different research          the particular individual case rather than the
    methods. Investigators adopting an objectivist         general and the universal; the interest is in a
    (or positivist) approach to the social world           subjective, relativistic social world rather than
    and who treat it like the world of natural             an absolutist, external reality. In its emphasis
    phenomena as being hard, real and external to the      on the particular and individual this approach
    individual will choose from a range of traditional     to understanding individual behaviour may be
    options – surveys, experiments, and the like.          termed idiographic.
    Others favouring the more subjectivist (or anti-          In this review of Burrell and Morgan’s analysis
    positivist) approach and who view the social world     of the ontological, epistemological, human and
    as being of a much softer, personal and humanly        methodological assumptions underlying two ways
    created kind will select from a comparable range       of conceiving social reality, we have laid the
    of recent and emerging techniques – accounts,          foundations for a more extended study of the
    participant observation and personal constructs,       two contrasting perspectives evident in the
    for example.                                           practices of researchers investigating human
       Where one subscribes to the view that treats        behaviour and, by adoption, educational problems.
    the social world like the natural world – as           Box 1.1 summarizes these assumptions along a
    if it were a hard, external and objective              subjective–objective dimension. It identifies the
    reality – then scientific investigation will be         four sets of assumptions by using terms we have
    directed at analysing the relationships and            adopted in the text and by which they are known
    regularities between selected factors in that          in the literature of social philosophy.
    world. It will be predominantly quantitative              Each of the two perspectives on the study of
    and will be concerned with identifying and             human behaviour outlined above has profound
    defining elements and discovering ways in which         implications for research in classrooms and
    their relationships can be expressed. Hence,           schools. The choice of problem, the formulation of
    they argue, methodological issues, of fundamental      questions to be answered, the characterization of
    importance, are thus the concepts themselves,          pupils and teachers, methodological concerns, the
    their measurement and the identification of             kinds of data sought and their mode of treatment,
                                                                                                                 POSITIVISM      9

Box 1.1

                                                                                                                                     Chapter 1
The subjective–objective dimension

            A scheme for analysing assumptions about the nature of social science

                  The subjectivist                                                     The objectivist
                  approach to                                                          approach to
                  social science                                                       social science

                  Nominalism                                 ontology                  Realism

                  Anti-positivism                          epistemology                Positivism

                  Voluntarism                             human nature                 Determinism

                  Idiographic                              methodology                 Nomothetic

Source: Burrell and Morgan 1979

all are influenced by the viewpoint held. Some                        his study of the history of the philosophy and
idea of the considerable practical implications of                   methodology of science, Oldroyd (1986) says:
the contrasting views can be gained by examining
                                                                        It was Comte who consciously ‘invented’ the new
Box 1.2 which compares them with respect to a
number of critical issues within a broadly societal                     science of society and gave it the name to which we
and organizational framework. Implications of the                       are accustomed . . . . For social phenomena were to be
two perspectives for research into classrooms and                       viewed in the light of physiological (or biological)
schools will unfold in the course of the text.                          laws and theories and investigated empirically, just
   Because of its significance for the epistemologi-                     like physical phenomena.
cal basis of social science and its consequences for                                                           (Oldroyd 1986)
educational research, we devote much discussion
in this chapter to the positivist and anti-positivist                Comte’s position was to lead to a general
debate.                                                              doctrine of positivism which held that all genuine
                                                                     knowledge is based on sense experience and can
                                                                     be advanced only by means of observation and
                                                                     experiment. Following in the empiricist tradition,
                                                                     it limited inquiry and belief to what can be firmly
Although positivism has been a recurrent                             established and in thus abandoning metaphysical
theme in the history of western thought from                         and speculative attempts to gain knowledge by
the Ancient Greeks to the present day, it                            reason alone, the movement developed what has
is historically associated with the nineteenth-                      been described as a ‘tough-minded orientation to
century French philosopher, Auguste Comte,                           facts and natural phenomena’ (Beck 1979).
who was the first thinker to use the word                                Although the term positivism is used by
for a philosophical position (Beck 1979). His                        philosophers and social scientists, a residual
positivism turns to observation and reason as                        meaning is always present and this derives from an
means of understanding behaviour; explanation                        acceptance of natural science as the paradigm of
proceeds by way of scientific description. In                         human knowledge (Duncan 1968). This includes

     Box 1.2
     Alternative bases for interpreting social reality

                                                          Conceptions of social reality
       Dimensions of comparison        Objectivist                                        Subjectivist
       Philosophical basis             Realism: the world exists and is knowable          Idealism: the world exists but different
                                       as it really is. Organizations are real            people construe it in very different ways.
                                       entities with a life of their own.                 Organizations are invented social reality.
       The role of social science      Discovering the universal laws of society          Discovering how different people
                                       and human conduct within it.                       interpret the world in which they live.
       Basic units of social reality   The collectivity: society or organizations.        Individuals acting singly or together.
       Methods of understanding        Identifying conditions or relationships            Interpretation of the subjective meanings
                                       which permit the collectivity to exist.            which individuals place upon their action.
                                       Conceiving what these conditions and               Discovering the subjective rules for such
                                       relationships are.                                 action.
       Theory                          A rational edifice built by scientists to           Sets of meanings which people use to
                                       explain human behaviour.                           make sense of their world and behaviour
                                                                                          within it.
       Research                        Experimental or quasi-experimental                 The search for meaningful relationships
                                       validation of theory.                              and the discovery of their consequences
                                                                                          for action.
       Methodology                     Abstraction of reality, especially through         The representation of reality for purposes
                                       mathematical models and quantitative               of comparison. Analysis of language and
                                       analysis.                                          meaning.
       Society                         Ordered. Governed by a uniform set of              Conflicted. Governed by the values of
                                       values and made possible only by those             people with access to power.
       Organizations                   Goal oriented. Independent of people.              Dependent upon people and their goals.
                                       Instruments of order in society serving            Instruments of power which some people
                                       both society and the individual.                   control and can use to attain ends which
                                                                                          seem good to them.
       Organizational pathologies      Organizations get out of kilter with social        Given diverse human ends, there is always
                                       values and individual needs.                       conflict among people acting to pursue
       Prescription for change         Change the structure of the organization           Find out what values are embodied in
                                       to meet social values and individual needs.        organizational action and whose they are.
                                                                                          Change the people or change their values
                                                                                          if you can.

     Source: adapted from Barr Greenfield 1975

     the following connected suppositions, identified                        scientists can be formulated in terms parallel to
     by Giddens (1975). First, the methodological                           those of natural science. This means that their
     procedures of natural science may be directly                          analyses must be expressed in laws or law-like
     applied to the social sciences. Positivism here                        generalizations of the same kind that have been
     implies a particular stance concerning the social                      established in relation to natural phenomena.
     scientist as an observer of social reality. Second,                    Positivism here involves a definite view of social
     the end-product of investigations by social                            scientists as analysts or interpreters of their subject
                                                          THE ASSUMPTIONS AND NATURE OF SCIENCE                11

matter. Positivism claims that science provides us    direct experience (Barratt 1971); and evidence,

                                                                                                                    Chapter 1
with the clearest possible ideal of knowledge.        data yielding proof or strong confirmation, in
   Where positivism is less successful, however,      probability terms, of a theory or hypothesis in
is in its application to the study of human           a research setting.
behaviour where the immense complexity of                Mouly (1978) identifies five steps in the process
human nature and the elusive and intangible           of empirical science:
quality of social phenomena contrast strikingly
                                                      1   experience: the starting point of scientific
with the order and regularity of the natural
                                                          endeavour at the most elementary level
world. This point is nowhere more apparent
                                                      2   classification: the formal systematization of
than in the contexts of classroom and school
                                                          otherwise incomprehensible masses of data
where the problems of teaching, learning and
                                                      3   quantification: a more sophisticated stage
human interaction present the positivistic
                                                          where precision of measurement allows
researcher with a mammoth challenge (see
                                                          more adequate analysis of phenomena by
                                                          mathematical means
9780415368780 – Chapter 1, file 1.2. ppt).
                                                      4   discovery of relationships: the identification
   For further information on positivism within
                                                          and classification of functional relationships
the history of the philosophy and methodology of
                                                          among phenomena
science, see Oldroyd (1986). We now look more
                                                      5   approximation to the truth: science proceeds by
closely at some of its features.
                                                          gradual approximation to the truth.
                                                      The third assumption underlying the work of the
The assumptions and nature of science
                                                      scientist is the principle of parsimony. The basic
We begin with an examination of the tenets of         idea is that phenomena should be explained in
scientific faith: the kinds of assumptions held        the most economical way possible, as Einstein
by scientists, often implicitly, as they go about     was known to remark – one should make matters
their daily work. First, there is the assumption      as simple as possible, but no simpler! The first
of determinism. This means simply that events         historical statement of the principle was by
have causes, that events are determined by other      William of Occam when he said that explanatory
circumstances; and science proceeds on the belief     principles (entities) should not be needlessly
that these causal links can eventually be uncovered   multiplied. It may, of course, be interpreted in
and understood, that the events are explicable in     various ways: that it is preferable to account for a
terms of their antecedents. Moreover, not only are    phenomenon by two concepts rather than three;
events in the natural world determined by other       that a simple theory is to be preferred to a complex
circumstances, but also there is regularity about     one.
the way they are determined: the universe does           The final assumption, that of generality, played
not behave capriciously. It is the ultimate aim       an important part in both the deductive
of scientists to formulate laws to account for the    and inductive methods of reasoning. Indeed,
happenings in the world, thus giving them a firm       historically speaking, it was the problematic
basis for prediction and control.                     relationship between the concrete particular and
   The second assumption is that of empiricism. We    the abstract general that was to result in two
have already touched upon this viewpoint, which       competing theories of knowledge – the rational
holds that certain kinds of reliable knowledge        and the empirical. Beginning with observations of
can only derive from experience. In practice,         the particular, scientists set out to generalize their
this means scientifically that the tenability of a     findings to the world at large. This is so because
theory or hypothesis depends on the nature of the     they are concerned ultimately with explanation.
empirical evidence for its support. Empirical here    Of course, the concept of generality presents much
means that which is verifiable by observation and      less of a problem to natural scientists working

     chiefly with inanimate matter than to human             Box 1.3
     scientists who, of necessity having to deal with       The functions of science
     samples of larger human populations, have to
     exercise great caution when generalizing their           1   Its problem-seeking, question-asking,
     findings to the particular parent populations.                hunch-encouraging, hypotheses-producing function.
                                                              2   Its testing, checking, certifying function; its trying out
        We come now to the core question: What is
                                                                  and testing of hypotheses; its repetition and
     science? Kerlinger (1970) points out that in the             checking of experiments; its piling up of facts.
     scientific world itself two broad views of science        3   Its organizing, theorizing, structuring function; its
     may be found: the static and the dynamic. The static         search for larger and larger generalizations.
     view, which has particular appeal for laypeople,         4   Its history-collecting, scholarly function.
                                                              5   Its technological side; instruments,
     is that science is an activity that contributes
                                                                  methods, techniques.
     systematized information to the world. The work          6   Its administrative, executive and organizational side.
     of the scientist is to uncover new facts and add         7   Its publicizing and educational functions.
     them to the existing corpus of knowledge. Science        8   Its applications to human use.
     is thus seen as an accumulated body of findings,          9   Its appreciation, enjoyment, celebration and
     the emphasis being chiefly on the present state of
     knowledge and adding to it.2 The dynamic view,
     by contrast, conceives science more as an activity,    Source: Maslow 1954
     as something that scientists do. According to this
     conception it is important to have an accumulated
     body of knowledge of course, but what really matter       Clearly there are several different types of the-
     most are the discoveries that scientists make. The     ory, and each type of theory defines its own kinds
     emphasis here, then, is more on the heuristic          of ‘proof’. For example, Morrison (1995a) identi-
     nature of science.                                     fies empirical theories, ‘grand’ theories and ‘critical’
        Contrasting views exist on the functions of         theory. Empirical theories and critical theories are
     science. We give a composite summary of these in       discussed below. ‘Grand theory’ is a metanarrative,
     Box 1.3. For the professional scientists, however,     defining an area of study, being speculative, clar-
     science is seen as a way of comprehending              ifying conceptual structures and frameworks, and
     the world; as a means of explanation and               creatively enlarging the way we consider behaviour
     understanding, of prediction and control. For them     and organizations (Layder 1994). It uses funda-
     the ultimate aim of science is theory.                 mental ontological and epistemological postulates
        Theory has been defined by Kerlinger as ‘a set       which serve to define a field of inquiry (Hughes
     of interrelated constructs [concepts], definitions,     1976). Here empirical material tends to be used
     and propositions that presents a systematic view       by way of illustration rather than ‘proof’. This
     of phenomena by specifying relations among             is the stuff of some sociological theories, for
     variables, with the purpose of explaining and          example Marxism, consensus theory and func-
     predicting the phenomena’ (Kerlinger 1970). In         tionalism. While sociologists may be excited by
     a sense, theory gathers together all the isolated      the totalizing and all-encompassing nature of such
     bits of empirical data into a coherent conceptual      theories, they have been subject to considerable
     framework of wider applicability. More than this,      undermining. For example, Merton (1949), Coser
     however, theory is itself a potential source of        and Rosenberg (1969), Doll (1993) and Layder
     further information and discoveries. It is in this     (1994) contend that while they might possess the
     way a source of new hypotheses and hitherto            attraction of large philosophical systems of consid-
     unasked questions; it identifies critical areas for     erable – Byzantine – architectonic splendour and
     further investigation; it discloses gaps in our        logical consistency, nevertheless they are scientif-
     knowledge; and enables a researcher to postulate       ically sterile, irrelevant and out of touch with a
     the existence of previously unknown phenomena.         world that is characterized by openness, fluidity,
                                                         THE ASSUMPTIONS AND NATURE OF SCIENCE                  13

heterogeneity and fragmentation. This book does          and yet must not be so comprehensive as

                                                                                                                     Chapter 1
not endeavour to refer to this type of theory.           to be unwieldy. On the other hand, it must
   The status of theory varies quite considerably        not overlook variables simply because they are
according to the discipline or area of knowledge         difficult to explain.
in question. Some theories, as in the natural            A theory should have considerable explanatory
sciences, are characterized by a high degree of          and predictive potential.
elegance and sophistication; others, perhaps like        A theory should be able to respond to observed
educational theory, are only at the early stages of      anomalies.
formulation and are thus characterized by great un-      A theory should spawn a research enterprise
evenness. Popper (1968), Lakatos (1970),3 Mouly          (echoing Siegel’s (1987) comment that one of
(1978), Laudan (1990) and Rasmussen (1990)               the characteristics of an effective theory is its
identify the following characteristics of an effec-      fertility).
tive empirical theory:                                   A theory should demonstrate precision and
                                                         universality, and set the grounds for its own
   A theoretical system must permit deductions           falsification and verification, identifying the
   and generate laws that can be tested                  nature and operation of a ‘severe test’ (Popper
   empirically; that is, it must provide the means       1968). An effective empirical theory is tested
   for its confirmation or rejection. One can             in contexts which are different from those that
   test the validity of a theory only through the        gave rise to the theory, i.e. they should move
   validity of the propositions (hypotheses) that        beyond simply corroboration and induction
   can be derived from it. If repeated attempts          and towards ‘testing’ (Laudan 1990). It should
   to disconfirm its various hypotheses fail, then        identify the type of evidence which is required
   greater confidence can be placed in its validity.      to confirm or refute the theory.
   This can go on indefinitely, until possibly            A theory must be operationalizable precisely.
   some hypothesis proves untenable. This would          A test of the theory must be replicable.
   constitute indirect evidence of the inadequacy
                                                      Sometimes the word model is used instead of, or
   of the theory and could lead to its rejection
                                                      interchangeably with, theory. Both may be seen as
   (or more commonly to its replacement by a
                                                      explanatory devices or schemes having a broadly
   more adequate theory that can incorporate the
                                                      conceptual framework, though models are often
                                                      characterized by the use of analogies to give a more
   Theory must be compatible with both
                                                      graphic or visual representation of a particular
   observation and previously validated theories.
                                                      phenomenon. Providing they are accurate and do
   It must be grounded in empirical data that have
                                                      not misrepresent the facts, models can be of great
   been verified and must rest on sound postulates
                                                      help in achieving clarity and focusing on key issues
   and hypotheses. The better the theory, the
                                                      in the nature of phenomena.
   more adequately it can explain the phenomena
                                                         Hitchcock and Hughes (1995) draw together
   under consideration, and the more facts it
                                                      the strands of the discussion so far when they
   can incorporate into a meaningful structure
                                                      describe a theory thus:
   of ever-greater generalizability. There should
   be internal consistency between these facts.         Theory is seen as being concerned with the
   It should clarify the precise terms in which it      development of systematic construction of knowledge
   seeks to explain, predict and generalize about       of the social world. In doing this theory employs
   empirical phenomena.                                 the use of concepts, systems, models, structures,
   Theories must be stated in simple terms; that        beliefs and ideas, hypotheses (theories) in order to
   theory is best that explains the most in the         make statements about particular types of actions,
   simplest way. This is the law of parsimony.          events or activities, so as to make analyses of their
   A theory must explain the data adequately            causes, consequences and process. That is, to explain

       events in ways which are consistent with a particular   whatever is ‘out there’. If our perceptions of the
       philosophical rationale or, for example, a particular   world are determined by the concepts available
       sociological or psychological perspective. Theories     to us, it follows that people with differing sets of
       therefore aim to both propose and analyze sets of       concepts will tend to view the ‘same’ objective
       relations existing between a number of variables        reality differently – a doctor diagnosing an illness
       when certain regularities and continuities can be       will draw upon a vastly different range of concepts
       demonstrated via empirical enquiry.                     from, say, the restricted and simplistic notions of
                       (Hitchcock and Hughes 1995: 20–1)       the layperson in that context.
                                                                  So, you may ask, where is all this leading?
     Scientific theories must, by their very nature, be
                                                               Simply to this: that social scientists have likewise
     provisional. A theory can never be complete in the
                                                               developed, or appropriated by giving precise
     sense that it encompasses all that can be known
                                                               meaning to, a set of concepts which enable them
     or understood about the given phenomenon.
                                                               to shape their perceptions of the world in a
     As Mouly (1978) argues, one scientific theory is
                                                               particular way, to represent that slice of reality
     replaced by a superior, more sophisticated theory,
                                                               which is their special study. And collectively,
     as new knowledge is acquired.
                                                               these concepts form part of their wider meaning
        In referring to theory and models, we have begun
                                                               system which permits them to give accounts of that
     to touch upon the tools used by scientists in their
                                                               reality, accounts which are rooted and validated
     work. We look now in more detail at two such
                                                               in the direct experience of everyday life. These
     tools which play a crucial role in science – the
                                                               points may be exemplified by the concept of social
     concept and the hypothesis.
                                                               class. Hughes (1976) says that it offers
                                                                 a rule, a grid, even though vague at times, to use in
     The tools of science                                        talking about certain sorts of experience that have
                                                                 to do with economic position, life-style, life-chances,
     Concepts express generalizations from particu-
                                                                 and so on. It serves to identify aspects of experience,
     lars – anger, achievement, alienation, velocity, in-
                                                                 and by relating the concept to other concepts we
     telligence, democracy. Examining these examples
                                                                 are able to construct theories about experience in a
     more closely, we see that each is a word repre-
                                                                 particular order or sphere.
     senting an idea: more accurately, a concept is the
                                                                                                     (Hughes 1976: 34)
     relationship between the word (or symbol) and an
     idea or conception. Whoever we are and whatever              There are two important points to stress when
     we do, we all make use of concepts. Naturally, some       considering scientific concepts. The first is that
     are shared and used by all groups of people within        they do not exist independently of us: they are
     the same culture – child, love, justice, for example;     indeed our inventions enabling us to acquire
     others, however, have a restricted currency and are       some understanding at least of the apparent chaos
     used only by certain groups, specialists, or members      of nature. The second is that they are limited
     of professions – idioglossia, retroactive inhibition,     in number and in this way contrast with the
     anticipatory socialization.                               infinite number of phenomena they are required
        Concepts enable us to impose some sort of              to explain.
     meaning on the world; through them reality is                A second tool of great importance to the
     given sense, order and coherence. They are the            scientist is the hypothesis. It is from this that
     means by which we are able to come to terms               much research proceeds, especially where cause-
     with our experience. How we perceive the world,           and-effect or concomitant relationships are being
     then, is highly dependent on the repertoire of            investigated. The hypothesis has been defined
     concepts we can command. The more we have,                by Kerlinger (1970) as a conjectural statement
     the more sense data we can pick up and the surer          of the relations between two or more variables,
     will be our perceptual (and cognitive) grasp of           or ‘an educated guess’, though it is unlike
                                                                                           THE SCIENTIFIC METHOD           15

an educated guess in that it is often the                    Box 1.4

                                                                                                                                Chapter 1
result of considerable study, reflective thinking             The hypothesis
and observation. Medawar (1972) writes of the
hypothesis and its function thus:                              Once one has a hypothesis to work on, the scientist
                                                               can move forward; the hypothesis will guide the
  All advances of scientific understanding, at every            researcher on the selection of some observations
  level, begin with a speculative adventure, an                rather than others and will suggest experiments.
  imaginative preconception of what might be true – a          Scientists soon learn by experience the characteristics
                                                               of a good hypothesis. A hypothesis that is so loose as
  preconception which always, and necessarily, goes a
                                                               to accommodate any phenomenon tells us precisely
  little way (sometimes a long way) beyond anything            nothing; the more phenomena it prohibits, the more
  which we have logical or factual authority to believe        informative it is.
  in. It is the invention of a possible world, or of                A good hypothesis must also have logical immediacy,
  a tiny fraction of that world. The conjecture is             i.e. it must provide an explanation of whatever it is
                                                               that needs to be explained and not an explanation of
  then exposed to criticism to find out whether or
                                                               other phenomena. Logical immediacy in a hypothesis
  not that imagined world is anything like the real            means that it can be tested by comparatively direct and
  one. Scientific reasoning is therefore at all levels          practicable means. A large part of the art of the soluble
  an interaction between two episodes of thought – a           is the art of devising hypotheses that can be tested by
  dialogue between two voices, the one imaginative             practicable experiments.
  and the other critical; a dialogue, if you like, between
  the possible and the actual, between proposal and          Source: adapted from Medawar 1981
  disposal, conjecture and criticism, between what
  might be true and what is in fact the case.
                                          (Medawar 1972)
                                                             Second, they are, in Kerlinger’s words, the working
                                                             instruments of theory. They can be deduced from
Kerlinger (1970) has identified two criteria for              theory or from other hypotheses. Third, they
‘good’ hypotheses. The first is that hypotheses               can be tested, empirically or experimentally, thus
are statements about the relations between                   resulting in confirmation or rejection; and there
variables; and second, that hypotheses carry                 is always the possibility that a hypothesis, once
clear implications for testing the stated relations.         supported and established, may become a law.
To these he adds two ancillary criteria: that                Fourth, hypotheses are powerful tools for the
hypotheses disclose compatibility with current               advancement of knowledge because, as Kerlinger
knowledge; and that they are expressed as                    (1970) explains, they enable us to get outside
economically as possible. Thus if we conjecture              ourselves. Hypotheses and concepts play a crucial
that social class background determines academic             part in the scientific method and it is to this that
achievement, we have a relationship between                  we now turn our attention.
one variable, social class, and another, academic
achievement. And since both can be measured,
                                                             The scientific method
the primary criteria specified by Kerlinger can be
met. Neither do they violate the ancillary criteria          If the most distinctive feature of science is
proposed by Kerlinger (see also Box 1.4).                    its empirical nature, the next most important
   He further identifies four reasons for the                 characteristic is its set of procedures which
importance of hypotheses as tools of research.               show not only how findings have been arrived
First, they organize the efforts of researchers.             at, but are sufficiently clear for fellow-scientists
The relationship expressed in the hypothesis                 to repeat them, i.e. to check them out with
indicates what they should do. They enable                   the same or other materials and thereby test
them to understand the problem with greater                  the results. As Cuff and Payne (1979) say: ‘A
clarity and provide them with a framework for                scientific approach necessarily involves standards
collecting, analysing and interpreting their data.           and procedures for demonstrating the ‘‘empirical

     warrant’’ of its findings, showing the match or         Box 1.5
     fit between its statements and what is happening        Stages in the development of a science
     or has happened in the world’ (Cuff and Payne
     1979: 4). These standards and procedures we              1   Definition of the science and identification of the
     will call for convenience ‘the scientific method’,            phenomena that are to be subsumed under it.
                                                              2   Observational stage at which the relevant factors,
     though this can be somewhat misleading for
                                                                  variables or items are identified and labelled, and at
     the following reason: the combination of the                 which categories and taxonomies are developed.
     definite article, adjective and singular noun             3   Correlational research in which variables and
     conjures up in the minds of some people a                    parameters are related to one another and
     single invariant approach to problem-solving, an             information is systematically integrated as theories
                                                                  begin to develop.
     approach frequently involving atoms or rats, and
                                                              4   The systematic and controlled manipulation of
     taking place within the confines of a laboratory.             variables to see if experiments will produce
     Yet there is much more to it than this. The                  expected results, thus moving from correlation to
     term in fact cloaks a number of methods which                causality.
     vary in their degree of sophistication depending         5   The firm establishment of a body of theory as the
                                                                  outcomes of the earlier stages are accumulated.
     on their function and the particular stage of                Depending on the nature of the phenomena under
     development a science has reached. Box 1.5 sets              scrutiny, laws may be formulated and systematized.
     out the sequence of stages through which a science       6   The use of the established body of theory in the
     normally passes in its development or, perhaps               resolution of problems or as a source of further
     more realistically, that are constantly present in           hypotheses.
     its progress and on which scientists may draw
     depending on the kind of information they seek
     or the kind of problem confronting them. Of
     particular interest in our efforts to elucidate the
                                                            consciously and deliberately by selecting from the
     term ‘scientific method’ are stages 2, 3 and 4.
                                                            total number of elements in a given situation. More
     Stage 2 is a relatively uncomplicated point at
                                                            recently Hitchcock and Hughes (1995: 23) suggest
     which the researcher is content to observe and
                                                            an eight-stage model of the scientific method that
     record facts and possibly arrive at some system
                                                            echoes Kerlinger. This is represented in Box 1.6.
     of classification. Much research in the field of
                                                               The elements the researchers fasten on to will
     education, especially at classroom and school
                                                            naturally be suitable for scientific formulation; this
     level, is conducted in this way, e.g. surveys and
                                                            means simply that they will possess quantitative
     case studies. Stage 3 introduces a note of added
     sophistication as attempts are made to establish
     relationships between variables within a loose
     framework of inchoate theory. Stage 4 is the           Box 1.6
                                                            An eight-stage model of the scientific method
     most sophisticated stage and often the one that
     many people equate exclusively with the scientific
     method. In order to arrive at causality, as distinct     Stage 1: Hypotheses, hunches and guesses
                                                              Stage 2: Experiment designed; samples taken;
     from mere measures of association, researchers here               variables isolated
     design experimental situations in which variables        Stage 3: Correlations observed; patterns identified
     are manipulated to test their chosen hypotheses.         Stage 4: Hypotheses formed to explain regularities
     This process moves from early, inchoate ideas,           Stage 5: Explanations and predictions tested;
     to more rigorous hypotheses, to empirical testing                 falsifiability
                                                              Stage 6: Laws developed or disconfirmation
     of those hypotheses, thence to confirmation or                     (hypothesis rejected)
     modification of the hypotheses (Kerlinger 1970).          Stage 7: Generalizations made
        With stages 3 and 4 of Box 1.5 in mind,               Stage 8: New theories.
     we may say that the scientific method begins
                                                 CRITICISMS OF POSITIVISM AND THE SCIENTIFIC METHOD                  17

aspects. Their principal working tool will be the        their illusions, the illusion Kierkegaard was most

                                                                                                                          Chapter 1
hypothesis which, as we have seen, is a statement        concerned about was that of objectivity. By this
indicating a relationship (or its absence) between       he meant the imposition of rules of behaviour
two or more of the chosen elements and stated in         and thought, and the making of a person into an
such a way as to carry clear implications for testing.   observer set on discovering general laws governing
Researchers then choose the most appropriate             human behaviour. The capacity for subjectivity,
method and put their hypotheses to the test.             he argued, should be regained. This he regarded
                                                         as the ability to consider one’s own relationship
                                                         to whatever constitutes the focus of inquiry.
Criticisms of positivism and the scientific               The contrast he made between objectivity and
method                                                   subjectivity is brought out in the following passage:
In spite of the scientific enterprise’s proven success
                                                           When the question of truth is raised in an objective
using positivism – especially in the field of natural
                                                           manner, reflection is directed objectively to the truth
science – its ontological and epistemological bases
have been the focus of sustained and sometimes             as an object to which the knower is related. Reflection
vehement criticism from some quarters. Beginning           is not focused on the relationship, however, but upon
in the second half of the nineteenth century,              the question of whether it is the truth to which
the revolt against positivism occurred on a broad          the knower is related. If only the object to which
front, attracting some of the best intellectuals in        he is related is the truth, the subject is accounted
Europe – philosophers, scientists, social critics and      to be in the truth. When the question of truth is
creative artists. Essentially, it has been a reaction      raised subjectively, reflection is directed subjectively
against the world picture projected by science             to the nature of the individual’s relationship; if only
which, it is contended, undermines life and mind.          the mode of this relationship is in the truth, the
The precise target of the anti-positivists’ attack         individual is in the truth, even if he should happen
has been science’s mechanistic and reductionist
                                                           to be thus related to what is not true.
view of nature which, by definition, defines
                                                                                         (Kierkegaard 1974: 178)
life in measurable terms rather than inner
experience, and excludes notions of choice,              For Kierkegaard, ‘subjectivity and concreteness
freedom, individuality, and moral responsibility,        of truth are together the light. Anyone who
regarding the universe as a living organism rather       is committed to science, or to rule-governed
than as a machine (e.g. Nesfield-Cookson 1987).           morality, is benighted, and needs to be rescued
   Another challenge to the claims of positivism         from his state of darkness’ (Warnock 1970).
came from Søren Kierkegaard, the Danish philo-              Also concerned with the dehumanizing effects
sopher, one of the originators of existentialism.        of the social sciences is Ions (1977). While
Kierkegaard was concerned with individuals and           acknowledging that they can take much credit
their need to fulfil themselves to the highest            for throwing light in dark corners, he expresses
level of development. This realization of a              serious concern at the way in which quantification
person’s potential was for him the meaning               and computation, assisted by statistical theory and
of existence which he saw as ‘concrete and               method, are used. He argues that quantification
individual, unique and irreducible, not amenable         is a form of collectivism, but that this runs
to conceptualization’ (Beck 1979). Characteristic        the risk of depersonalization. His objection is
features of the age in which we live – democracy’s       not directed at quantification per se, but at
trust in the crowd mentality, the ascendancy of          quantification when it becomes an end in itself – ‘a
reason, scientific and technological progress – all       branch of mathematics rather than a humane
militate against the achievement of this end             study seeking to explore and elucidate the gritty
and contribute to the dehumanization of the              circumstances of the human condition’ (Ions
individual. In his desire to free people from            1977). This echoes Horkheimer’s (1972) powerful

     critique of positivism as the mathematization of      opinion, moral judgements and beliefs. Scientific
     concepts about nature.                                explanation seems to be the only means of
        Another forceful critic of the objective           explaining behaviour, and, for them, this seriously
     consciousness has been Roszak (1970; 1972), who       diminishes the very characteristics that make
     argues that science, in its pursuit of objectivity,   humans human. It makes for a society without
     is a form of alienation from our true selves and      conscience. Positivism is unable to answer
     from nature. The justification for any intellectual    many interesting or important areas of life
     activity lies in the effect it has on increasing      (Habermas 1972: 300). Indeed this is an echo
     our awareness and degree of consciousness. This       of Wittgenstein’s (1974) famous comment that
     increase, some claim, has been retarded in            when all possible scientific questions have been
     our time by the excessive influence that the           addressed they have left untouched the main
     positivist paradigm has exerted on areas of our       problems of life.
     intellectual life. Holbrook (1977), for example,         Other criticisms are commonly levelled at
     affording consciousness a central position in         positivistic social science from within its own
     human existence and deeply concerned with what        ranks. One is that it fails to take account of
     happens to it, condemns positivism and empiricism     our unique ability to interpret our experiences
     for their bankruptcy of the inner world, morality     and represent them to ourselves. We can and do
     and subjectivity.                                     construct theories about ourselves and our world;
        Hampden-Turner (1970) concludes that the           moreover, we act on these theories. In failing to
     social science view of human beings is biased         recognize this, positivistic social science is said to
     in that it is conservative and ignores important      ignore the profound differences between itself and
     qualities. This restricted image of humans, he        the natural sciences. Social science, unlike natural
     contends, comes about because social scientists       science, stands in a subject–subject rather than a
     concentrate on the repetitive, predictable and        subject–object relation to its field of study, and
     invariant aspects of the person; on ‘visible          works in a pre-interpreted world in the sense that
     externalities’ to the exclusion of the subjective     the meanings that subjects hold are part of their
     world; and on the parts of the person in their        construction of the world (Giddens 1976).
     endeavours to understand the whole.                      The difficulty in which positivism finds
        Habermas (1972), in keeping with the Frankfurt     itself is that it regards human behaviour as
     School of critical theory (critical theory is         passive, essentially determined and controlled,
     discussed below), provides a corrosive critique of    thereby ignoring intention, individualism and
     positivism, arguing that the scientific mentality      freedom. This approach suffers from the same
     has been elevated to an almost unassailable           difficulties that inhere in behaviourism, which
     position – almost to the level of a religion          has scarcely recovered from Chomsky’s (1959)
     (scientism) – as being the only epistemology of       withering criticism where he writes that a singular
     the west. In this view all knowledge becomes          problem of behaviourism is our inability to infer
     equated with scientific knowledge. This neglects       causes from behaviour, to identify the stimulus that
     hermeneutic, aesthetic, critical, moral, creative     has brought about the response – the weakness
     and other forms of knowledge. It reduces behaviour    of Skinner’s stimulus–response theory. This
     to technicism.                                        problem with positivism also rehearses the familiar
        Positivism’s concern for control and, thereby,     problem in social theory, namely the tension
     its appeal to the passivity of behaviourism and       between agency and structure (Layder 1994):
     for instrumental reason is a serious danger to the    humans exercise agency – individual choice and
     more open-ended, creative, humanitarian aspects       intention – not necessarily in circumstances of
     of social behaviour. Habermas (1972; 1974) and        their own choosing, but nevertheless they do
     Horkheimer (1972) argue that scientism silences       not behave simply or deterministically like
     an important debate about values, informed            puppets.

   Finally, the findings of positivistic social science       The anti-positivist movement has influenced

                                                                                                                      Chapter 1
are often said to be so banal and trivial that they      those constituent areas of social science of most
are of little consequence to those for whom they         concern to us, namely, psychology, social psychol-
are intended, namely, teachers, social workers,          ogy and sociology. In the case of psychology, for
counsellors, personnel managers, and the like. The       instance, a school of humanistic psychology has
more effort, it seems, that researchers put into their   emerged alongside the coexisting behaviouristic
scientific experimentation in the laboratory by           and psychoanalytic schools. Arising as a response
restricting, simplifying and controlling variables,      to the challenge to combat the growing feelings
the more likely they are to end up with a ‘pruned,       of dehumanization which characterize many social
synthetic version of the whole, a constructed play       and cultural milieux, it sets out to study and un-
of puppets in a restricted environment.’4                derstand the person as a whole (Buhler and Allen
   These are formidable criticisms; but what             1972). Humanistic psychologists present a model
alternatives are proposed by the detractors of           of people that is positive, active and purposive, and
positivistic social science?                             at the same time stresses their own involvement
                                                         with the life experience itself. They do not stand
                                                         apart, introspective, hypothesizing. Their interest
Alternatives to positivistic social science:             is directed at the intentional and creative aspects
naturalistic approaches                                  of the human being. The perspective adopted by
Although the opponents of positivism within so-          humanistic psychologists is naturally reflected in
cial science itself subscribe to a variety of schools    their methodology. They are dedicated to study-
of thought each with its own subtly different epis-      ing the individual in preference to the group,
temological viewpoint, they are united by their          and consequently prefer idiographic approaches to
common rejection of the belief that human be-            nomothetic ones. The implications of the move-
haviour is governed by general, universal laws           ment’s philosophy for the education of the human
and characterized by underlying regularities. More-      being have been drawn by Carl Rogers.5
over, they would agree that the social world can             Comparable developments within social
be understood only from the standpoint of the            psychology may be perceived in the ‘science of
individuals who are part of the ongoing action           persons’ movement. It is argued here that we must
being investigated and that their model of a per-        use ourselves as a key to our understanding of
son is an autonomous one, not the plastic version        others and conversely, our understanding of oth-
favoured by positivist researchers. In rejecting the     ers as a way of finding out about ourselves, an
viewpoint of the detached, objective observer – a        anthropomorphic model of people. Since anthro-
mandatory feature of traditional research – anti-        pomorphism means, literally, the attribution of
positivists would argue that individuals’ behaviour      human form and personality, the implied criticism
can only be understood by the researcher shar-           is that social psychology as traditionally conceived
ing their frame of reference: understanding of           has singularly failed, so far, to model people as
individuals’ interpretations of the world around         they really are. As some wry commentators have
them has to come from the inside, not the out-           pleaded, ‘For scientific purposes, treat people as if
side. Social science is thus seen as a subjective                                          e
                                                         they were human beings’ (Harr´ and Secord 1972),
rather than an objective undertaking, as a means         which entails treating them as capable of moni-
of dealing with the direct experience of people          toring and arranging their own actions, exercising
in specific contexts, and where social scientists         their agency.
understand, explain and demystify social reality             Social psychology’s task is to understand people
through the eyes of different participants; the par-     in the light of this anthropomorphic model. Pro-
ticipants themselves define the social reality (Beck      ponents of this ‘science of persons’ approach place
1979) (see           great store on the systematic and painstaking anal-
9780415368780 – Chapter 1, file 1.3. ppt).                ysis of social episodes, i.e. behaviour in context.

     In Box 1.7 we give an example of such an episode                     of various hue possess particular distinguishing
     taken from a classroom study. Note how the par-                      features:
     ticular incident would appear on an interaction
     analysis coding sheet of a researcher employing a
     positivistic approach. Note, too, how this slice of                       People are deliberate and creative in
     classroom life can be understood only by knowl-                           their actions, they act intentionally and
     edge of the specific organizational background and                         make meanings in and through their
     context in which it is embedded.                                          activities (Blumer 1969).
        The approach to analysing social episodes                              People actively construct their social world –
     in terms of the ‘actors’ themselves is known                              they are not the ‘cultural dopes’ or passive dolls
     as the ‘ethogenic method’.6 Unlike positivistic                           of positivism (Garfinkel, 1967; Becker 1970).
     social psychology, which ignores or presumes its                          Situations are fluid and changing rather
     subjects’ interpretations of situations, ethogenic                        than fixed and static; events and behaviour
     social psychology, concentrates upon the ways                             evolve over time and are richly affected by
     in which persons construe their social world.                             context – they are ‘situated activities’.
     By probing at their accounts of their actions,                            Events and individuals are unique and largely
     it endeavours to come up with an understanding                            non-generalizable.
     of what those persons were doing in the particular                        A view that the social world should be
     episode.                                                                  studied in its natural state, without the
        As an alternative to positivist approaches,                            intervention of, or manipulation by, the
     naturalistic, qualitative, interpretive approaches                        researcher (Hammersley and Atkinson 1983).

     Box 1.7
     A classroom episode

       Walker and Adelman describe an incident in the following manner:

       In one lesson the teacher was listening to the boys read through short essays that they had written for homework on the
       subject of ‘Prisons’. After one boy, Wilson, had finished reading out his rather obviously skimped piece of work, the teacher
       sighed and said, rather crossly:

          T: Wilson, we’ll have to put you away if you don’t change your ways, and do your homework. Is that all you’ve done?
          P: Strawberries, strawberries. (Laughter)

       Now at first glance this is meaningless. An observer coding with Flanders Interaction Analysis Categories (FIAC) would
       write down:

          ‘7’ (teacher criticizes) followed by a,
          ‘4’ (teacher asks question) followed by a,
          ‘9’ (pupil irritation) and finally a,
          ‘10’ (silence or confusion) to describe the laughter

       Such a string of codings, however reliable and valid, would not help anyone to understand why such an interruption was
       funny. Human curiosity makes us want to know why everyone laughs – and so, I would argue, the social scientist needs to
       know too. Walker and Adelman (1976), asked subsequently why ‘strawberries’ was a stimulus to laughter and were told
       that the teacher frequently said the pupils’ work was ‘like strawberries – good as far as it goes, but it doesn’t last nearly long
       enough’. Here a casual comment made in the past has become an integral part of the shared meaning system of the class. It
       can be comprehended only by seeing the relationship as developing over time.

     Source: adapted from Delamont 1976

   Fidelity to the phenomena being studied is         perspectives and the categories subsumed under

                                                                                                                   Chapter 1
   fundamental.                                       each, particularly as they refer to social psychology
   People interpret events, contexts and situa-       and sociology. The terms in question are
   tions, and act on the bases of those events        ‘normative’ and ‘interpretive’. The normative
   (echoing Thomas’s (1928) famous dictum that        paradigm (or model) contains two major orienting
   if people define their situations as real then      ideas (Douglas 1973): first, that human behaviour
   they are real in their consequences – if I         is essentially rule-governed, and second, that it
   believe there is a mouse under the table, I will   should be investigated by the methods of natural
   act as though there is a mouse under the table,    science. The interpretive paradigm, in contrast to
   whether there is or not (Morrison 1998)).          its normative counterpart, is characterized by a
   There are multiple interpretations of, and         concern for the individual. Whereas normative
   perspectives on, single events and situations.     studies are positivist, all theories constructed
   Reality is multilayered and complex.               within the context of the interpretive paradigm
   Many events are not reducible to simplistic in-    tend to be anti-positivist. As we have seen,
   terpretation, hence ‘thick descriptions’ (Geertz   the central endeavour in the context of the
   1973b) are essential rather than reductionism,     interpretive paradigm is to understand the
   that is to say thick descriptions representing     subjective world of human experience. To retain
   the complexity of situations are preferable to     the integrity of the phenomena being investigated,
   simplistic ones.                                   efforts are made to get inside the person and
   We need to examine situations through the          to understand from within. The imposition of
   eyes of participants rather than the researcher.   external form and structure is resisted, since this
                                                      reflects the viewpoint of the observer as opposed
   The anti-positivist movement in sociology          to that of the actor directly involved.
is represented by three schools of thought –             Two further differences between the two
phenomenology, ethnomethodology and symbolic          paradigms may be identified at this stage: the
interactionism. A common thread running               first concerns the concepts of ‘behaviour’ and
through the three schools is a concern with           ‘action’; the second, the different conceptions of
phenomena, that is, the things we directly            ‘theory’. A key concept within the normative
apprehend through our senses as we go about           paradigm, behaviour refers to responses either to
our daily lives, together with a consequent           external environmental stimuli (another person,
emphasis on qualitative as opposed to quantitative    or the demands of society, for instance) or to
methodology. The differences between them and         internal stimuli (hunger, or the need to achieve, for
the significant roles each phenomenon plays in         example). In either case, the cause of the behaviour
research in classrooms and schools are such as to     lies in the past. Interpretive approaches, on the
warrant a more extended consideration of them in      other hand, focus on action. This may be thought
the discussion below.                                 of as behaviour-with-meaning; it is intentional
                                                      behaviour and as such, future oriented. Actions
                                                      are meaningful to us only in so far as we are
A question of terminology: the normative              able to ascertain the intentions of actors to share
and interpretive paradigms                            their experiences. A large number of our everyday
So far we have introduced and used a variety          interactions with one another rely on such shared
of terms to describe the numerous branches and        experiences.
schools of thought embraced by the positivist            As regards theory, normative researchers try to
and anti-positivist viewpoints. As a matter of        devise general theories of human behaviour and
convenience and as an aid to communication,           to validate them through the use of increasingly
we clarify at this point two generic terms            complex research methodologies which, some
conventionally used to describe these two             believe, push them further and further from the

     experience and understanding of the everyday               In its broadest meaning, phenomenology is a theo-
     world and into a world of abstraction. For them,           retical point of view that advocates the study of
     the basic reality is the collectivity; it is external to   direct experience taken at face value; and one
     the actor and manifest in society, its institutions        which sees behaviour as determined by the phe-
     and its organizations. The role of theory is to            nomena of experience rather than by external,
     say how reality hangs together in these forms              objective and physically described reality (English
     or how it might be changed so as to be more                and English 1958). Although phenomenologists
     effective. The researcher’s ultimate aim is to             differ among themselves on particular issues, there
     establish a comprehensive ‘rational edifice’, a             is fairly general agreement on the following points
     universal theory, to account for human and social          identified by Curtis (1978) which can be taken
     behaviour.                                                 as distinguishing features of their philosophical
        But what of the interpretive researchers?               viewpoint:
     They begin with individuals and set out to
                                                                   a belief in the importance, and in a sense the
     understand their interpretations of the world
                                                                   primacy, of subjective consciousness
     around them. Theory is emergent and must arise
                                                                   an understanding of consciousness as active, as
     from particular situations; it should be ‘grounded’
                                                                   meaning bestowing
     in data generated by the research act (Glaser
                                                                   a claim that there are certain essential
     and Strauss 1967). Theory should not precede
                                                                   structures to consciousness of which we gain
     research but follow it. Investigators work directly
                                                                   direct knowledge by a certain kind of reflection:
     with experience and understanding to build their
                                                                   exactly what these structures are is a point
     theory on them. The data thus yielded will include
                                                                   about which phenomenologists have differed.
     the meanings and purposes of those people who
     are their source. Further, the theory so generated         Various strands of development may be traced
     must make sense to those to whom it applies. The           in the phenomenological movement: we shall
     aim of scientific investigation for the interpretive        briefly examine two of them – the transcendental
     researcher is to understand how this glossing of           phenomenology of Husserl, and existential
     reality goes on at one time and in one place and           phenomenology, of which Schutz is perhaps the
     compare it with what goes on in different times            most characteristic representative.
     and places. Thus theory becomes sets of meanings              Husserl, regarded by many as the founder of
     which yield insight and understanding of people’s          phenomenology, was concerned with investigating
     behaviour. These theories are likely to be as diverse      the source of the foundation of science and
     as the sets of human meanings and understandings           with questioning the commonsense, ‘taken-for-
     that they are to explain. From an interpretive             granted’ assumptions of everyday life (see Burrell
     perspective the hope of a universal theory which           and Morgan 1979). To do this, he set about
     characterizes the normative outlook gives way              opening up a new direction in the analysis of
     to multifaceted images of human behaviour as               consciousness. His catch-phrase was ‘Back to the
     varied as the situations and contexts supporting           things!’ which for him meant finding out how
     them.                                                      things appear directly to us rather than through
                                                                the media of cultural and symbolic structures. In
                                                                other words, we are asked to look beyond the
     Phenomenology, ethnomethodology and
                                                                details of everyday life to the essences underlying
     symbolic interactionism
                                                                them. To do this, Husserl exhorts us to ‘put the
     There are many variants of qualitative, naturalistic       world in brackets’ or free ourselves from our usual
     approaches (Jacob 1987; Hitchcock and Hughes               ways of perceiving the world. What is left over from
     1995). Here we focus on three significant ‘tradi-           this reduction is our consciousness of which there
     tions’ in this style of research – phenomenology,          are three elements – the ‘I’ who thinks, the mental
     ethnomethodology and symbolic interactionism.              acts of this thinking subject, and the intentional
                            PHENOMENOLOGY, ETHNOMETHODOLOGY AND SYMBOLIC INTERACTIONISM                             23

objects of these mental acts. The aim, then, of           to treat practical activities, practical circumstances,

                                                                                                                         Chapter 1
this method of epoch´ , as Husserl called it, is the
                        e                                 and practical sociological reasonings as topics
dismembering of the constitution of objects in            of empirical study, and by paying to the most
such a way as to free us from all preconceptions          commonplace activities of daily life the attention
about the world (see Warnock 1970).                       usually accorded extraordinary events, seeks to learn
   Schutz was concerned with relating Husserl’s
                                                          about them as phenomena in their own right.
ideas to the issues of sociology and to the scientific
                                                                                                (Garfinkel 1967)
study of social behaviour. Of central concern
to him was the problem of understanding the             He maintains that students of the social world
meaning structure of the world of everyday life.        must doubt the reality of that world; and that in
The origins of meaning he thus sought in the            failing to view human behaviour more sceptically,
‘stream of consciousness’ – basically an unbroken       sociologists have created an ordered social reality
stream of lived experiences which have no               that bears little relationship to the real thing. He
meaning in themselves. One can impute meaning           thereby challenges the basic sociological concept
to them only retrospectively, by the process of         of order.
turning back on oneself and looking at what                Ethnomethodology, then, is concerned with
has been going on. In other words, meaning can          how people make sense of their everyday world.
be accounted for in this way by the concept of          More especially, it is directed at the mechanisms by
reflexivity. For Schutz, the attribution of meaning      which participants achieve and sustain interaction
reflexively is dependent on the people identifying       in a social encounter – the assumptions they make,
the purpose or goal they seek (see Burrell and          the conventions they utilize and the practices
Morgan 1979).                                           they adopt. Ethnomethodology thus seeks to
   According to Schutz, the way we understand           understand social accomplishments in their own
the behaviour of others is dependent on a               terms; it is concerned to understand them from
process of typification by means of which the            within (see Burrell and Morgan 1979).
observer makes use of concepts resembling ‘ideal           In identifying the taken-for-granted assump-
types’ to make sense of what people do. These           tions characterizing any social situation and the
concepts are derived from our experience of             ways in which the people involved make their
everyday life and it is through them, claims            activities rationally accountable, ethnomethodol-
Schutz, that we classify and organize our everyday      ogists use notions like ‘indexicality’ and ‘reflexiv-
world. As Burrell and Morgan (1979) observe, we         ity’. Indexicality refers to the ways in which actions
learn these typifications through our biographical       and statements are related to the social contexts
locations and social contexts. Our knowledge of         producing them; and to the way their meanings
the everyday world inheres in social order and this     are shared by the participants but not necessarily
world itself is socially ordered.                       stated explicitly. Indexical expressions are thus the
   The fund of everyday knowledge by means              designations imputed to a particular social occa-
of which we are able to typify other people’s           sion by the participants in order to locate the event
behaviour and come to terms with social reality         in the sphere of reality. Reflexivity, on the other
varies from situation to situation. We thus live in a   hand, refers to the way in which all accounts of
world of multiple realities, and social actors move     social settings – descriptions, analyses, criticisms,
within and between these with ease (Burrell and         etc. – and the social settings occasioning them are
Morgan 1979), abiding by the rules of the game          mutually interdependent.
for each of these worlds.                                  It is convenient to distinguish between two
   Like phenomenology, ethnomethodology is              types of ethnomethodologists: linguistic and
concerned with the world of everyday life. In           situational. The linguistic ethnomethodologists
the words of its proponent, Harold Garfinkel, it         focus upon the use of language and the ways
sets out                                                in which conversations in everyday life are

     structured. Their analyses make much use of              what makes them distinctively human and social.
     the unstated taken-for-granted meanings, the use         Interactionists therefore focus on the world of
     of indexical expressions and the way in which            subjective meanings and the symbols by which
     conversations convey much more than is actually          they are produced and represented. This means
     said. The situational ethnomethodologists cast           not making any prior assumptions about what is
     their view over a wider range of social activity         going on in an institution, and taking seriously,
     and seek to understand the ways in which                 indeed giving priority to, inmates’ own accounts.
     people negotiate the social contexts in which            Thus, if pupils appear preoccupied for too much of
     they find themselves. They are concerned to               the time – ‘being bored’, ‘mucking about’, ‘having
     understand how people make sense of and order            a laugh’, etc. the interactionist is keen to explore
     their environment. As part of their empirical            the properties and dimensions of these processes.
     method, ethnomethodologists may consciously                 Second, this attribution of meaning to objects
     and deliberately disrupt or question the ordered         through symbols is a continuous process. Action
     taken-for-granted elements in everyday situations        is not simply a consequence of psychological
     in order to reveal the underlying processes at work.     attributes such as drives, attitudes or personalities,
        The substance of ethnomethodology thus                or determined by external social facts such as
     largely comprises a set of specific techniques and        social structure or roles, but results from a
     approaches to be used in studying what Garfinkel          continuous process of meaning attribution which
     (1967) has described as the ‘awesome indexicality’       is always emerging in a state of flux and subject
     of everyday life. It is geared to empirical study, and   to change. The individual constructs, modifies,
     the stress which its practitioners place upon the        pieces together, weighs up the pros and cons and
     uniqueness of the situation encountered, projects        bargains.
     its essentially relativist standpoint. A commitment         Third, this process takes place in a social
     to the development of methodology and fieldwork           context. Individuals align their actions to those
     has occupied first place in the interests of its          of others. They do this by ‘taking the role of the
     adherents, so that related issues of ontology,           other’, by making indications to ‘themselves’ about
     epistemology and the nature of human beings have         the likely responses of ‘others’. They construct how
     received less attention than perhaps they deserve.       others wish or might act in certain circumstances,
        Essentially, the notion of symbolic inter-            and how they themselves might act. They might
     actionism derives from the work of Mead (1934).          try to ‘manage’ the impressions others have of
     Although subsequently to be associated with such         them, put on a ‘performance’, try to influence
     noted researchers as Blumer, Hughes, Becker and          others’ ‘definition of the situation’.
     Goffman, the term does not represent a unified               Instead of focusing on the individual, then, and
     perspective in that it does not embrace a common         his or her personality characteristics, or on how the
     set of assumptions and concepts accepted by all          social structure or social situation causes individual
     who subscribe to the approach. For our purposes,         behaviour, symbolic interactionists direct their
     however, it is possible to identify three basic          attention at the nature of interaction, the dynamic
     postulates. These have been set out by Woods             activities taking place between people. In focusing
     (1979) as follows. First, human beings act towards       on the interaction itself as a unit of study,
     things on the basis of the meanings they have            the symbolic interactionist creates a more active
     for them. Humans inhabit two different worlds:           image of the human being and rejects the image
     the ‘natural’ world wherein they are organisms           of the passive, determined organism. Individuals
     of drives and instincts and where the external           interact; societies are made up of interacting
     world exists independently of them, and the              individuals. People are constantly undergoing
     social world where the existence of symbols, like        change in interaction and society is changing
     language, enables them to give meaning to objects.       through interaction. Interaction implies human
     This attribution of meanings, this interpreting, is      beings acting in relation to each other, taking
                                    CRITICISMS OF THE NATURALISTIC AND INTERPRETIVE APPROACHES                       25

each other into account, acting, perceiving,          of their intentions, this, surely, cannot be said to

                                                                                                                          Chapter 1
interpreting, acting again. Hence, a more dynamic     comprise the purpose of a social science. As Rex
and active human being emerges rather than an         (1974) has observed:
actor merely responding to others. Woods (1983:
                                                        While patterns of social reactions and institutions
15–16) summarizes key emphases of symbolic
                                                        may be the product of the actors’ definitions of the
interaction thus:
                                                        situations there is also the possibility that those actors
   individuals as constructors of their own actions     might be falsely conscious and that sociologists have
   the various components of the self and how           an obligation to seek an objective perspective which
   they interact; the indications made to self,         is not necessarily that of any of the participating
   meanings attributed, interpretive mechanisms,        actors at all . . . . We need not be confined purely
   definitions of the situation; in short, the world     and simply to that . . . social reality which is made
   of subjective meanings, and the symbols by           available to us by participant actors themselves.
   which they are produced and represented                                                            (Rex 1974)
   the process of negotiation, by which meanings
                                                      While these more recent perspectives have
   are continually being constructed
                                                      presented models of people that are more in
   the social context in which they occur and
                                                      keeping with common experience, some argue that
   whence they derive
                                                      anti-positivists have gone too far in abandoning
   by taking the ‘role of the other’ – a dynamic
                                                      scientific procedures of verification and in giving
   concept involving the construction of how
                                                      up hope of discovering useful generalizations about
   others wish to or might act in a certain
                                                      behaviour (see Mead 1934). Are there not dangers
   circumstance, and how individuals themselves
                                                      in rejecting the approach of physics in favour
   might act – individuals align their actions to
                                                      of methods more akin to literature, biography
   those of others.
                                                      and journalism? Some specific criticisms of the
   A characteristic common to the phenomenolog-       methodologies are well directed, for example
ical, ethnomethodological and symbolic interac-       Argyle (1978) questions whether, if carefully
tionist perspectives, which makes them singularly     controlled interviews such as those used in social
attractive to the would-be educational researcher,    surveys are inaccurate, then the less controlled
is the way they fit naturally to the kind of con-      interviews carry even greater risks of inaccuracy.
centrated action found in classrooms and schools.     Indeed Bernstein (1974) suggests that subjective
Yet another shared characteristic is the manner       reports may be incomplete and misleading.
in which they are able to preserve the integrity         Bernstein’s criticism is directed at the overriding
of the situation where they are employed. Here        concern of phenomenologists and ethnomethodol-
the influence of the researcher in structuring,        ogists with the meanings of situations and the ways
analysing and interpreting the situation is present   in which these meanings are negotiated by the
to a much smaller degree than would be the            actors involved. What is overlooked about such
case with a more traditionally oriented research      negotiated meanings, observes Bernstein (1974),
approach.                                             is that the very process whereby one interprets
                                                      and defines a situation is itself a product of the
                                                      circumstances in which one is placed. One im-
Criticisms of the naturalistic and
                                                      portant factor in such circumstances that must be
interpretive approaches
                                                      considered is the power of others to impose their
Critics have wasted little time in pointing out       own definitions of situations upon participants.
what they regard as weaknesses in these newer         Doctors’ consulting rooms and headteachers’ stud-
qualitative perspectives. They argue that while it    ies are locations in which inequalities in power are
is undeniable that our understanding of the actions   regularly imposed upon unequal participants. The
of our fellow-beings necessarily requires knowledge   ability of certain individuals, groups, classes and

     authorities to persuade others to accept their def-     with technical and hermeneutic knowledge re-
     initions of situations demonstrates that while – as     spectively (Gage 1989). The paradigm of critical
     ethnomethodologists insist – social structure is a      educational research is heavily influenced by the
     consequence of the ways in which we perceive            early work of Habermas and, to a lesser ex-
     social relations, it is clearly more than this. Con-    tent, his predecessors in the Frankfurt School,
     ceiving of social structure as external to ourselves    most notably Adorno, Marcuse, Horkheimer and
     helps us take its self-evident effects upon our         Fromm. Here the expressed intention is delib-
     daily lives into our understanding of the social        erately political – the emancipation of individ-
     behaviour going on about us. Here is rehearsed the      uals and groups in an egalitarian society (see
     tension between agency and structure of social the-
     orists (Layder 1994); the danger of interactionist      9780415368780 – Chapter 1, file 1.4. ppt).
     and interpretive approaches is their relative ne-          Critical theory is explicitly prescriptive and nor-
     glect of the power of external – structural – forces    mative, entailing a view of what behaviour in a so-
     to shape behaviour and events. There is a risk          cial democracy should entail (Fay 1987; Morrison
     in interpretive approaches that they become             1995a). Its intention is not merely to give an ac-
     hermetically sealed from the world outside the          count of society and behaviour but to realize a
     participants’ theatre of activity – they put artifi-     society that is based on equality and democracy
     cial boundaries around subjects’ behaviour. Just        for all its members. Its purpose is not merely to
     as positivistic theories can be criticized for their    understand situations and phenomena but to
     macro-sociological persuasion, so interpretive and      change them. In particular it seeks to emancipate
     qualitative theories can be criticized for their nar-   the disempowered, to redress inequality and to
     rowly micro-sociological perspectives.                  promote individual freedoms within a democratic
                                                                In this enterprise critical theory identifies
     Critical theory and critical educational                the ‘false’ or ‘fragmented’ consciousness (Eagleton
     research                                                1991) that has brought an individual or social
     Positivist and interpretive paradigms are essen-        group to relative powerlessness or, indeed,
     tially concerned with understanding phenomena           power, and it questions the legitimacy of this.
     through two different lenses. Positivism strives        It holds up to the lights of legitimacy and
     for objectivity, measurability, predictability, con-    equality issues of repression, voice, ideology,
     trollability, patterning, the construction of laws      power, participation, representation, inclusion
     and rules of behaviour, and the ascription of           and interests. It argues that much behaviour
     causality; the interpretive paradigms strive to         (including research behaviour) is the outcome of
     understand and interpret the world in terms of          particular illegitimate, dominatory and repressive
     its actors. In the former observed phenomena are        factors, illegitimate in the sense that they do
     important; in the latter meanings and interpreta-       not operate in the general interest – one person’s
     tions are paramount. Habermas (1984: 109–10),           or group’s freedom and power is bought at the
     echoing Giddens (1976), describes this latter as a      price of another’s freedom and power. Hence
     ‘double hermeneutic’, where people strive to inter-     critical theory seeks to uncover the interests at
     pret and operate in an already interpreted world.       work in particular situations and to interrogate
     An emerging approach to educational research is         the legitimacy of those interests, identifying the
     the paradigm of critical educational research. This     extent to which they are legitimate in their
     regards the two previous paradigms as presenting        service of equality and democracy. Its intention is
     incomplete accounts of social behaviour by their        transformative: to transform society and individuals
     neglect of the political and ideological contexts       to social democracy. In this respect the purpose of
     of much educational research. Positivistic and          critical educational research is intensely practical,
     interpretive paradigms are seen as preoccupied          to bring about a more just, egalitarian society
                                                CRITICAL THEORY AND CRITICAL EDUCATIONAL RESEARCH                  27

in which individual and collective freedoms are           knowledge – is not neutral (see also Mannheim

                                                                                                                        Chapter 1
practised, and to eradicate the exercise and effects      1936). What counts as worthwhile knowledge is
of illegitimate power. The pedigree of critical           determined by the social and positional power
theory in Marxism, thus, is not difficult to               of the advocates of that knowledge. The link
discern. For critical theorists, researchers can no       here between objects of study and communi-
longer claim neutrality and ideological or political      ties of scholars echoes Kuhn’s (1962) notions
innocence.                                                of paradigms and paradigm shifts, where the
    Critical theory and critical educational              field of knowledge or paradigm is seen to be
research, then, have their substantive agenda – for       only as good as the evidence and the respect
example examining and interrogating: the re-              in which it is held by ‘authorities’. Knowledge
lationships between school and society – how              and definitions of knowledge reflect the inter-
schools perpetuate or reduce inequality; the so-          ests of the community of scholars who operate
cial construction of knowledge and curricula, who         in particular paradigms. Habermas (1972) con-
defines worthwhile knowledge, what ideological             structs the definition of worthwhile knowledge and
interests this serves, and how this reproduces in-        modes of understanding around three cognitive in-
equality in society; how power is produced and            terests (see
reproduced through education; whose interests are         9780415368780 – Chapter 1, file 1.5. ppt):
served by education and how legitimate these are
                                                             prediction and control
(e.g. the rich, white, middle-class males rather
                                                             understanding and interpretation
than poor, non-white females).
                                                             emancipation and freedom.
    The significance of critical theory for research is
immense, for it suggests that much social research        He names these the ‘technical’, ‘practical’ and
is comparatively trivial in that it accepts rather than   ‘emancipatory’ interests respectively. The techni-
questions given agendas for research, compounded          cal interest characterizes the scientific, positivist
by the funding for research, which underlines             method outlined earlier, with its emphasis on laws,
the political dimension of research sponsorship           rules, prediction and control of behaviour, with
(discussed later) (Norris 1990). Critical theorists       passive research objects – instrumental knowl-
would argue that the positivist and interpretive          edge. The ‘practical’ interest, an attenuation of the
paradigms are essentially technicist, seeking to          positivism of the scientific method, is exemplified
understand and render more efficient an existing           in the hermeneutic, interpretive methodologies
situation, rather than to question or transform it.       outlined in the qualitative approaches earlier (e.g.
    Habermas (1972) offers a useful tripartite con-       symbolic interactionism). Here research method-
ceptualization of interests that catches the three        ologies seek to clarify, understand and interpret the
paradigms of research in this chapter. He sug-            communications of ‘speaking and acting subjects’
gests that knowledge – and hence research knowl-          (Habermas 1974: 8).
edge – serves different interests. Interests, he ar-         Hermeneutics focuses on interaction and
gues, are socially constructed, and are ‘knowledge-       language; it seeks to understand situations through
constitutive’, because they shape and determine           the eyes of the participants, echoing the verstehen
what counts as the objects and types of knowledge.        approaches of Weber and premised on the view
Interests have an ideological function (Morrison          that reality is socially constructed (Berger and
1995a), for example a ‘technical interest’ (dis-          Luckmann 1967). Indeed Habermas (1988: 12)
cussed below) can have the effect of keeping the          suggests that sociology must understand social
empowered in their empowered position and the             facts in their cultural significance and as socially
disempowered in their powerlessness – i.e. rein-          determined. Hermeneutics involves recapturing
forcing and perpetuating the status quo. An ‘eman-        the meanings of interacting others, recovering and
cipatory interest’ (discussed below) threatens the        reconstructing the intentions of the other actors in a
status quo. In this view knowledge – and research         situation. Such an enterprise involves the analysis

     of meaning in a social context (Held 1980). Gadamer       perpetuate a system which keeps them either
     (1975: 273) argues that the hermeneutic sciences          empowered or disempowered (Geuss 1981), i.e.
     (e.g. qualitative approaches) involve the fusion          which suppresses a generalizable interest. Expla-
     of horizons between participants. Meanings rather         nations for situations might be other than those
     than phenomena take on significance here.                  ‘natural’, taken for granted, explanations that
        The emancipatory interest subsumes the pre-            the participants might offer or accept. Situations
     vious two paradigms; it requires them but goes            are not natural but problematic (Carr and Kem-
     beyond them (Habermas 1972: 211). It is con-              mis 1986). They are the outcomes or processes
     cerned with praxis – action that is informed by           wherein interests and powers are protected and
     reflection with the aim to emancipate (Kincheloe           suppressed, and one task of ideology critique is
     1991: 177). The twin intentions of this interest are      to expose this (Grundy 1987). The interests at
     to expose the operation of power and to bring about       work are uncovered by ideology critique, which,
     social justice as domination and repression act to        itself, is premised on reflective practice (Morrison
     prevent the full existential realization of individual    1995a; 1995b; 1996a). Habermas (1972: 230)
     and social freedoms (Habermas 1979: 14). The task         suggests that ideology critique through reflective
     of this knowledge-constitutive interest, indeed of        practice can be addressed in four stages:
     critical theory itself, is to restore to consciousness
     those suppressed, repressed and submerged deter-             Stage 1: a description and interpretation of
     minants of unfree behaviour with a view to their             the existing situation – a hermeneutic exercise
     dissolution (Habermas 1984: 194–5).                          that identifies and attempts to make sense of
        What we have in effect, then, in Habermas’s               the current situation (echoing the verstehen
     early work is an attempt to conceptualize three              approaches of the interpretive paradigm) (see
     research styles: the scientific, positivist style; the
     interpretive style; and the emancipatory, ideol-             9780415368780 – Chapter 1, file 1.6. ppt).
     ogy critical style. Not only does critical theory            Stage 2: a penetration of the reasons that
     have its own research agenda, but also it has its            brought the existing situation to the form
     own research methodologies, in particular ideol-             that it takes – the causes and purposes of
     ogy critique and action research. With regard to             a situation and an evaluation of their
     ideology critique, a particular reading of ideology          legitimacy, involving an analysis of interests
     is being adopted here, as the suppression of generaliz-      and ideologies at work in a situation, their
     able interests (Habermas 1976: 113), where systems,          power and legitimacy (both in micro- and
     groups and individuals operate in rationally inde-           macro-sociological terms). Habermas’s (1972)
     fensible ways because their power to act relies on           early work likens this to psychoanalysis as
     the disempowering of other groups, i.e. that their           a means for bringing into the consciousness
     principles of behaviour cannot be generalized.               of ‘patients’ those repressed, distorted and
        Ideology – the values and practices emanat-               oppressive conditions, experiences and factors
     ing from particular dominant groups – is the                 that have prevented them from a full, complete
     means by which powerful groups promote and                   and accurate understanding of their conditions,
     legitimize their particular – sectoral – interests at        situations and behaviour, and that, on such
     the expense of disempowered groups. Ideology                 exposure and examination, will be liberatory
     critique exposes the operation of ideology in                and emancipatory. Critique here reveals to
     many spheres of education, the working out of                individuals and groups how their views and
     vested interests under the mantle of the gen-                practices might be ideological distortions that,
     eral good. The task of ideology critique is to               in their effects, perpetuate a social order or
     uncover the vested interests at work which may               situation that works against their democratic
     be occurring consciously or subliminally, reveal-            freedoms, interests and empowerment (see
     ing to participants how they may be acting to                also Carr and Kemmis 1986: 138–9).
                                                        CRITICISMS OF APPROACHES FROM CRITICAL THEORY              29

   Stage 3: an agenda for altering the situation – in      that mandated change is addressed efficiently and

                                                                                                                        Chapter 1
   order for moves to an egalitarian society to be         effectively.
   furthered.                                                 Morrison (1995a) suggests that critical theory,
   Stage 4: an evaluation of the achievement of            because it has a practical intent to transform and
   the situation in practice.                              empower, can – and should – be examined and
                                                           perhaps tested empirically. For example, critical
In the world of education Habermas’s stages are            theory claims to be empowering; that is a testable
paralleled by Smyth (1989) who, too, denotes a             proposition. Indeed, in a departure from some of his
four-stage process:                                        earlier writing, in some of his later work Habermas
                                                           (1990) acknowledges this; he argues for the need
   description (what am I doing?)                          to find ‘counter examples’ (p. 6), to ‘critical
   information (what does it mean?)                        testing’ (p. 7) and empirical verification (p. 117).
   confrontation (how did I come to be like this?)         He acknowledges that his views have only
   reconstruction (how might I do things                   ‘hypothetical status’ (p. 32) that need to be
   differently?)                                           checked against specific cases (p. 9). One could
                                                           suggest, for instance, that the effectiveness of his
It can be seen that ideology critique here has both        critical theory can be examined by charting the
a reflective, theoretical and a practical side to it;       extent to which equality, freedom, democracy,
without reflection it is hollow and without practice        emancipation, empowerment have been realized
it is empty.                                               by dint of his theory; the extent to which
   As ideology is not mere theory but impacts              transformative practices have been addressed or
directly on practice (Eagleton 1991) there is              occurred as a result of his theory; the extent to
a strongly practical methodology implied by                which subscribers to his theory have been able to
critical theory, which articulates with action             assert their agency; the extent to which his theories
research (Callawaert 1999). Action research                have broken down the barriers of instrumental
(discussed in Chapter 14), as its name suggests,           rationality. The operationalization and testing (or
is about research that impacts on, and focuses on,         empirical investigation) of his theories clearly is
practice. In its espousal of practitioner research,        a major undertaking, and one which Habermas
for example teachers in schools, participant               has not done. In this respect critical theory, a
observers and curriculum developers, action                theory that strives to improve practical living,
research recognizes the significance of contexts            runs the risk of becoming merely contemplative
for practice – locational, ideological, historical,        (see
managerial, social. Furthermore it accords power to        9780415368780 – Chapter 1, file 1.7. ppt).
those who are operating in those contexts, for they
are both the engines of research and of practice. In
that sense the claim is made that action research          Criticisms of approaches from critical
is strongly empowering and emancipatory in that            theory
it gives practitioners a ‘voice’ (Carr and Kemmis
1986; Grundy 1987), participation in decision-             There are several criticisms that have been voiced
making, and control over their environment and             against critical approaches. Morrison (1995a)
professional lives. Whether the strength of the            suggests that there is an artificial separation
claims for empowerment are as strong as their              between Habermas’s three interests – they are
proponents would hold is another matter, for               drawn far more sharply (Hesse 1982; Bernstein
action research might be relatively powerless in           1983: 33). For example, one has to bring
the face of mandated changes in education. Here            hermeneutic knowledge to bear on positivist
action research might be more concerned with               science and vice versa in order to make
the intervening in existing practice to ensure             meaning of each other and in order to judge

     their own status. Further, the link between               theorists would argue that the call for researchers
     ideology critique and emancipation is neither clear       to be ideologically neutral is itself ideologically
     nor proven, nor a logical necessity (Morrison             saturated with laissez-faire values which allow the
     1995a: 67) – whether a person or society can              status quo to be reproduced, i.e. that the call
     become emancipated simply by the exercise                 for researchers to be neutral and disinterested is
     of ideology critique or action research is an             just as value laden as is the call for them to
     empirical rather than a logical matter (Morrison          intrude their own perspectives. The rights of the
     1995a; Wardekker and Miedama 1997). Indeed                researcher to move beyond disinterestedness are
     one can become emancipated by means other than            clearly contentious, though the safeguard here is
     ideology critique; emancipated societies do not           that the researcher’s is only one voice in the
     necessarily demonstrate or require an awareness           community of scholars (Kemmis 1982). Critical
     of ideology critique. Moreover, it could be argued        theorists as researchers have been hoisted by their
     that the rationalistic appeal of ideology critique        own petard, for if they are to become more than
     actually obstructs action designed to bring about         merely negative Jeremiahs and sceptics, berating
     emancipation. Roderick (1986: 65), for example,           a particular social order that is dominated by
     questions whether the espousal of ideology critique       scientism and instrumental rationality (Eagleton
     is itself as ideological as the approaches that it        1991; Wardekker and Miedama 1997), then they
     proscribes. Habermas, in his allegiance to the view       have to generate a positive agenda, but in so doing
     of the social construction of knowledge through           they are violating the traditional objectivity of
     ‘interests’, is inviting the charge of relativism.        researchers. Because their focus is on an ideological
        While the claim to there being three forms of          agenda, they themselves cannot avoid acting
     knowledge has the epistemological attraction of           ideologically (Morrison 1995a).
     simplicity, one has to question this very simplicity         Claims have been made for the power of action
     (e.g. Keat 1981: 67); there are a multitude of            research to empower participants as researchers
     interests and ways of understanding the world and         (e.g. Carr and Kemmis 1986; Grundy 1987). This
     it is simply artificial to reduce these to three.          might be over-optimistic in a world in which power
     Indeed it is unclear whether Habermas, in his             is often through statute; the reality of political
     three knowledge-constitutive interests, is dealing        power seldom extends to teachers. That teachers
     with a conceptual model, a political analysis, a set      might be able to exercise some power in schools
     of generalities, a set of transhistorical principles, a   but that this has little effect on the workings
     set of temporally specific observations, or a set of       of society at large was caught in Bernstein’s
     loosely defined slogans (Morrison 1995a: 71) that          (1970) famous comment that ‘education cannot
     survive only by dint of their ambiguity (Kolakowsi        compensate for society’. Giving action researchers
     1978). Lakomski (1999: 179–82) questions the              a small degree of power (to research their own
     acceptability of the consensus theory of truth on         situations) has little effect on the real locus
     which Habermas’s work is premised; she argues             of power and decision-making, which often lies
     that Habermas’s work is silent on social change,          outside the control of action researchers. Is action
     and is little more than speculation, a view echoed        research genuinely and full-bloodedly empowering
     by Fendler’s (1999) criticism of critical theory          and emancipatory? Where is the evidence?
     as inadequately problematizing subjectivity and
        More fundamental to a critique of this approach
                                                               Critical theory and curriculum research
     is the view that critical theory has a deliberate         For research methods, the tenets of critical theory
     political agenda, and that the task of the researcher     suggest their own substantive fields of enquiry
     is not to be an ideologue or to have an                   and their own methods (e.g. ideology critique and
     agenda, but to be dispassionate, disinterested and        action research). Beyond that the contribution to
     objective (Morrison 1995a). Of course, critical           this text on empirical research methods is perhaps
                                                       CRITICAL THEORY AND CURRICULUM RESEARCH                  31

limited by the fact that the agenda of critical        incorporated into a view of curricula as being rich,

                                                                                                                     Chapter 1
theory is highly particularistic, prescriptive and,    relational, recursive and rigorous (Doll 1993) with
as has been seen, problematical. Though it is          an emphasis on emergence, process epistemology and
an influential paradigm, it is influential in certain    constructivist psychology.
fields rather than in others. For example, its impact      Not all knowledge can be included in the cur-
on curriculum research has been far-reaching.          riculum; the curriculum is a selection of what is
   It has been argued for many years that the          deemed to be worthwhile knowledge. The justi-
most satisfactory account of the curriculum is         fication for that selection reveals the ideologies
given by a modernist, positivist reading of the        and power in decision-making in society and
development of education and society. This has its     through the curriculum. Curriculum is an ideologi-
curricular expression in Tyler’s (1949) famous and     cal selection from a range of possible knowledge.
influential rationale for the curriculum in terms of    This resonates with Habermas’s (1972) view that
four questions:                                        knowledge and its selection is neither neutral nor
1   What educational purposes should the school
                                                          Ideologies can be treated unpejoratively as
    seek to attain?
                                                       sets of beliefs or, more sharply, as sets of
2   What educational experiences can be
                                                       beliefs emanating from powerful groups in society,
    provided that are likely to attain these
                                                       designed to protect the interests of the dominant.
                                                       If curricula are value-based then why is it that
3   How can these educational experiences be
                                                       some values hold more sway than others? The link
    effectively organized?
                                                       between values and power is strong. This theme
4   How can we determine whether these
                                                       asks not only what knowledge is important but also
    purposes are being attained?
                                                       whose knowledge is important in curricula, what
Underlying this rationale is a view that the cur-      and whose interests such knowledge serves, and how
riculum is controlled (and controllable), ordered,     the curriculum and pedagogy serve (or do not
predetermined, uniform, predictable and largely        serve) differing interests. Knowledge is not neutral
behaviourist in outcome – all elements of the          (as was the tacit view in modernist curricula). The
positivist mentality that critical theory eschews.     curriculum is ideologically contestable terrain.
Tyler’s rationale resonates sympathetically with          The study of the sociology of knowledge
a modernist, scientific, managerialist mentality        indicates how the powerful might retain their
of society and education that regards ideology         power through curricula and how knowledge and
and power as unproblematic, indeed it claims           power are legitimized in curricula. The study
the putative political neutrality and objectivity      of the sociology of knowledge suggests that the
of positivism (Doll 1993); it ignores the advances     curriculum should be both subject to ideology
in psychology and psychopedagogy made by con-          critique and itself promote ideology critique in
structivism.                                           students. A research agenda for critical theorists,
   However, this view has been criticized for          then, is how the curriculum perpetuates the
precisely these sympathies. Doll (1993) argues         societal status quo and how can it (and should
that it represents a closed system of planning         it) promote equality in society.
and practice that sits uncomfortably with the             The notion of ideology critique engages the
notion of education as an opening process and          early writings of Habermas (1972), in particular his
with the view of postmodern society as open and        theory of three knowledge-constitutive interests.
diverse, multidimensional, fluid and with power         His technical interest (in control and predictability)
less monolithic and more problematical. This view      resonates with Tyler’s (1949) model of the
takes seriously the impact of chaos and complexity     curriculum and reveals itself in technicist,
theory and derives from them some important            instrumentalist and scientistic views of curricula
features for contemporary curricula. These are         that are to be ‘delivered’ to passive recipients – the

     curriculum is simply another commodity in a                  In the field of critical pedagogy the argument
     consumer society in which differential cultural           is advanced that educators must work with, and
     capital is inevitable. Habermas’s hermeneutic             on, the lived experience that students bring to
     interest (in understanding others’ perspectives           the pedagogical encounter rather than imposing
     and views) resonates with a process view                  a dominatory curriculum that reproduces social
     of the curriculum. His emancipatory interest              inequality. In this enterprise teachers are to trans-
     (in promoting social emancipation, equality,              form the experience of domination in students
     democracy, freedoms and individual and collective         and empower them to become ‘emancipated’ in
     empowerment) requires an exposure of the                  a full democracy. Students’ everyday experiences
     ideological interests at work in curricula in order       of oppression, of being ‘silenced’, of having their
     that teachers and students can take control of            cultures and ‘voices’ excluded from curricula and
     their own lives for the collective, egalitarian good.     decision-making are to be examined for the ideo-
     Habermas’s emancipatory interest denotes an               logical messages that are contained in such acts.
     inescapably political reading of the curriculum and       Raising awareness of such inequalities is an im-
     the purposes of education – the movement away             portant step to overcoming them. Teachers and
     from authoritarianism and elitism and towards             students together move forward in the progress
     social democracy.                                         towards ‘individual autonomy within a just soci-
        Habermas’s work underpins and informs much             ety’ (Masschelein 1991: 97). In place of centrally
     contemporary and recent curriculum theory                 prescribed and culturally biased curricula that stu-
     (e.g. Grundy 1987; Apple 1990; UNESCO 1996)               dents simply receive, critical pedagogy regards the
     and is a useful heuristic device for understanding        curriculum as a form of cultural politics in which
     the motives behind the heavy prescription of              participants in (rather than recipients of) curricula
     curriculum content in, for example, the United            question the cultural and dominatory messages
     Kingdom, New Zealand, Hong Kong and France.               contained in curricula and replace them with a
     For instance, one can argue that the National             ‘language of possibility’ and empowering, often
     Curriculum of England and Wales is heavy on the           community-related curricula. In this way curricula
     technical and hermeneutic interests but very light        serve the ‘socially critical’ rather than the cultur-
     on the emancipatory interest (Morrison 1995a)             ally and ideologically passive school.
     and that this (either deliberately or in its effects)        One can discern a utopian and generalized
     supports – if not contributes to – the reproduction       tenor in some of this work, and applying critical
     of social inequality. As Bernstein (1971: 47)             theory to education can be criticized for its
     argues: ‘how a society selects, classifies, distributes,   limited comments on practice. Indeed Miedama
     transmits and evaluates the educational knowledge         and Wardekker (1999: 68) go so far as to suggest
     it considers to be public, reflects both the               that critical pedagogy has had its day, and that
     distribution of power and the principles of social        it was a stillborn child and that critical theory
     control’.                                                 is a philosophy of science without a science
        Several writers on curriculum theory (e.g.             (p. 75)! Nevertheless it is an important field
     McLaren 1995; Leistyna et al. 1996) argue that            for it recognizes and makes much of the fact
     power is a central, defining concept in matters            that curricula and pedagogy are problematical and
     of the curriculum. Here considerable importance           political.
     is accorded to the political agenda of the
     curriculum, and the empowerment of individuals
     and societies is an inescapable consideration in
                                                               A summary of the three paradigms
     the curriculum. One means of developing student           Box 1.8 summarizes some of the broad differences
     and societal empowerment finds its expression              between the three approaches that we have made
     in Habermas’s (1972) emancipatory interest and            so far (see
     critical pedagogy.                                        9780415368780 – Chapter 1, file 1.8. ppt)
                                                             THE EMERGING PARADIGM OF COMPLEXITY THEORY                     33

Box 1.8

                                                                                                                                 Chapter 1
Differing approaches to the study of behaviour

  Normative                                Interpretive                           Critical
  Society and the social system            The individual                         Societies, groups and individuals
  Medium/large-scale research              Small-scale research                   Small-scale research
  Impersonal, anonymous forces             Human actions continuously             Political, ideological factors, power
  regulating behaviour                     recreating social life                 and interests shaping behaviour
  Model of natural sciences                Non-statistical                        Ideology critique and action research
  ‘Objectivity’                            ‘Subjectivity’                         Collectivity
  Research conducted ‘from the             Personal involvement of the            Participant researchers, researchers
  outside’                                 researcher                             and facilitators
  Generalizing from the specific            Interpreting the specific               Critiquing the specific
  Explaining behaviour/seeking causes      Understanding actions/meanings         Understanding, interrogating,
  Assuming the taken-for-granted           rather than causes                     critiquing, transforming actions and
  Macro-concepts: society,                 Investigating the taken-for-granted    interests
  institutions, norms, positions, roles,   Micro-concepts: individual             Interrogating and critiquing the
  expectations                             perspective, personal constructs,      taken for granted
  Structuralists                           negotiated meanings, definitions of     Macro- and micro-concepts: political
  Technical interest                       situations                             and ideological interests, operations
                                           Phenomenologists, symbolic             of power
                                           interactionists, ethnomethodologists   Critical theorists, action researchers,
                                           Practical interest                     practitioner researchers
                                                                                  Emancipatory interest

The emerging paradigm of complexity                                Chaos and complexity theories argue against
theory                                                          the linear, deterministic, patterned, universal-
                                                                izable, stable, atomized, modernistic, objective,
An emerging fourth paradigm in educational                      mechanist, controlled, closed systems of law-like
research is that of complexity theory (Morrison                 behaviour which may be operating in the labora-
2002a). Complexity theory looks at the world in                 tory but which do not operate in the social world
ways which break with simple cause-and-effect                   of education. These features of chaos and com-
models, linear predictability, and a dissection                 plexity theories seriously undermine the value of
approach to understanding phenomena, replacing                  experiments and positivist research in education
them with organic, non-linear and holistic                      (e.g. Gleick 1987; Waldrop 1992; Lewin 1993).
approaches (Santonus 1998: 3) in which relations                   Complexity theory suggests that phenomena
within interconnected networks are the order                    must be looked at holistically; to atomize
of the day (Youngblood 1997: 27; Wheatley                       phenomena into a restricted number of variables
1999: 10). Here key terms are feedback,                         and then to focus only on certain factors is
recursion, emergence, connectedness and self-                   to miss the necessary dynamic interaction of
organization. Out go the simplistic views of                    several parts. More fundamentally, complexity
linear causality, the ability to predict, control and           theory suggests that the conventional units of
manipulate, and in come uncertainty, networks                   analysis in educational research (as in other
and connection, self-organization, emergence over               fields) should move away from, for example,
time through feedback and the relationships of                  individuals, institutions, communities and systems
the internal and external environments, and                     (cf. Lemke 2001). These should merge, so
survival and development through adaptation and                 that the unit of analysis becomes a web
change.                                                         or ecosystem (Capra 1996: 301), focused on,

     and arising from, a specific topic or centre              and constructivist perspectives. Addressing com-
     of interest (a ‘strange attractor’). Individuals,        plexity theory’s argument for self-organization,
     families, students, classes, schools, communities        the call is for the teacher-as-researcher move-
     and societies exist in symbiosis; complexity theory      ment to be celebrated, and complexity theory
     tells us that their relationships are necessary,         suggests that research in education could concern
     not contingent, and analytic, not synthetic.             itself with the symbiosis of internal and exter-
     This is a challenging prospect for educational           nal researchers and research partnerships. Just as
     research, and complexity theory, a comparatively         complexity theory suggests that there are mul-
     new perspective in educational research, offers          tiple views of reality, so this accords not only
     considerable leverage into understanding societal,       with the need for several perspectives on a situ-
     community, individual, and institutional change;         ation (using multi-methods), but resonates with
     it provides the nexus between macro- and                 those tenets of critical research that argue for
     micro-research in understanding and promoting            different voices and views to be heard. Hetero-
     change.                                                  geneity is the watchword. Complexity theory not
        In addressing holism, complexity theory suggests      only provides a powerful challenge to conven-
     the need for case study methodology, action              tional approaches to educational research, but
     research, and participatory forms of research,           also suggests both a substantive agenda and a set
     premised in many ways on interactionist,                 of methodologies. It provides an emerging new
     qualitative accounts, i.e. looking at situations         paradigm for research (see http://www.routledge.
     through the eyes of as many participants or              com/textbooks/9780415368780 – Chapter 1, file
     stakeholders as possible. This enables multiple          1.1.doc).
     causality, multiple perspectives and multiple
     effects to be charted. Self-organization, a
     key feature of complexity theory, argues for             Feminist research
     participatory, collaborative and multi-perspectival
                                                              It is perhaps no mere coincidence that feminist
     approaches to educational research. This is not to
                                                              research should surface as a serious issue at the same
     deny ‘outsider’ research; it is to suggest that, if it
                                                              time as ideology-critical paradigms for research;
     is conducted, outsider research has to take in as        they are closely connected. Usher (1996: 124),
     many perspectives as possible.
                                                              although criticizing Habermas for his faith in
        In educational research terms, complexity
                                                              family life as a haven from a heartless, exploitative
     theory stands against simple linear methodologies
                                                              world, nevertheless sets out several principles of
     based on linear views of causality, arguing for          feminist research that resonate with the ideology
     multiple causality and multidirectional causes and
                                                              critique of the Frankfurt School:
     effects, as organisms (however defined: individuals,
     groups, communities) are networked and relate at            acknowledging the pervasive influence of
     a host of different levels and in a range of diverse        gender as a category of analysis and
     ways. No longer can one be certain that a simple            organization
     cause brings a simple or single effect, or that a           deconstructing traditional commitments to
     single effect is the result of a single cause, or that      truth, objectivity and neutrality
     the location of causes will be in single fields only,        adopting an approach to knowledge creation
     or that the location of effects will be in a limited        which recognizes that all theories are
     number of fields.                                            perspectival
        Complexity theory not only questions the val-            using a multiplicity of research methods
     ues of positivist research and experimentation, but         acknowledging the interdisciplinary nature of
     also underlines the importance of educational re-           feminist research
     search to catch the deliberate, intentional, agentic        involving the researcher and the people being
     actions of participants and to adopt interactionist         researched
                                                                                       FEMINIST RESEARCH       35

   deconstructing the theory–practice relation-          oppress women’ (p. 23). Gender, as Ezzy (2002:

                                                                                                                    Chapter 1
   ship.                                                 43) writes, is ‘a category of experience’.
                                                            Positivist research served a given set of power
Her suggestions build on earlier recognition of          relations, typically empowering the white, male-
the significance of addressing the ‘power issue’ in       dominated research community at the expense of
research (‘whose research’, ‘research for whom’,         other groups whose voices were silenced. Feminist
‘research in whose interests’) and the need to ad-       research seeks to demolish and replace this with
dress the emancipatory element of educational            a different substantive agenda – of empowerment,
research – that research should be empowering            voice, emancipation, equality and representation
to all participants. The paradigm of critical            for oppressed groups. In doing so, it recognizes
theory questioned the putative objective, neu-           the necessity for foregrounding issues of power,
tral, value-free, positivist, ‘scientific’ paradigm for   silencing and voicing, ideology critique and a
the splitting of theory and practice and for its         questioning of the legitimacy of research that does
reproduction of asymmetries of power (reproduc-          not emancipate hitherto disempowered groups.
ing power differentials in the research community        In feminist research, women’s consciousness of
and for treating participants/respondents instru-        oppression, exploitation and disempowerment
mentally – as objects).                                  becomes a focus for research – the paradigm of
   Robson (1993: 64) suggests seven sources of           ideology critique.
sexism in research:                                         Far from treating educational research as
                                                         objective and value-free, feminists argue that
   androcentricity: seeing the world through male        this is merely a smokescreen that serves the
   eyes and applying male research paradigms to          existing, disempowering status quo, and that the
   females                                               subject and value-laden nature of research must
   overgeneralization: when a study generalizes          be surfaced, exposed and engaged (Haig 1999:
   from males to females                                 223). Supposedly value-free, neutral research
   gender insensitivity: ignoring sex as a possible      perpetuates power differentials. Indeed Jayaratne
   variable                                              and Stewart (1991) question the traditional,
   double standards: using male criteria, measures       exploitative nature of much research in which
   and standards to judge the behaviour of women         the researchers receive all the rewards while
   and vice versa (e.g. in terms of social status)       the participants remain in their – typically
   sex appropriateness: e.g. that child-rearing is       powerless – situation, i.e. in which the status
   women’s responsibility                                quo of oppression, under-privilege and inequality
   familism: treating the family, rather than the        remain undisturbed. As Scott (1985: 80) writes:
   individual, as the unit of analysis                   ‘we may simply use other women’s experiences to
   sexual dichotomism: treating the sexes as distinct    further our own aims and careers’. Cresswell (1998:
   social groups when, in fact, they may share           83), too, suggests that feminist research strives
   characteristics.                                      to establish collaborative and non-exploitative
                                                         relationships. Indeed Scott (1985) questions how
Feminist research, too, challenges the legitimacy        ethical it is for a woman researcher to interview
of research that does not empower oppressed and          those who are less privileged and more exploited
otherwise invisible groups – women. Ezzy (2002:          than she herself is.
20) writes of the need to replace a traditional             Changing this situation entails taking seriously
masculine picture of science with an emancipatory        issues of reflexivity, the effects of the research
commitment to knowledge that stems from a                on the researched and the researchers, the
feminist perspective, since, ‘if women’s experience      breakdown of the positivist paradigm, and the
is analysed using only theories and observations         raising of consciousness of the purposes and
from the standpoint of men, the resulting theories       effects of the research. Ezzy (2002: 153) writes

     that ‘the personal experience of the researcher             There is a concern with the construction
     is an integral part of the research process’ and            and reproduction of gender and sexual
     reinforces the point that objectivity is a false claim      difference.
     by researchers.                                             Narrow disciplinary boundaries are rejected.
        Ribbens and Edwards (1997) suggest that it               The artificial subject/researcher dualism is
     is important to ask how researchers can produce             rejected.
     work with reference to theoretical perspectives             Positivism and objectivity as male mythology
     and formal traditions and requirements of public,           are rejected.
     academic knowledge while still remaining faithful           There is an increased use of qualitative,
     to the experiences and accounts of research                 introspective biographical research techniques.
     participants. Denzin (1989), Mies (1993), Haig              The gendered nature of social research and the
     (1999) and De Laine (2000) argue for several                development of anti-sexist research strategies
     principles in feminist research:                            are recognized.
                                                                 There is a review of the research process as
        The asymmetry of gender relations and                    consciousness and awareness raising and as
        representation must be studied reflexively as             fundamentally participatory.
        constituting a fundamental aspect of social life         The primacy of women’s personal subjective
        (which includes educational research).                   experience is recognized.
        Women’s issues, their history, biography and             Hierarchies in social research are rejected.
        biology, feature as a substantive agenda/focus           The vertical, hierarchical relationships of
        in research – moving beyond mere perspecti-              researchers, research community and research
        val/methodological issues to setting a research          objects, in which the research itself can
        agenda.                                                  become an instrument of domination and
        The raising of consciousness of oppression,              the reproduction and legitimation of power
        exploitation, empowerment, equality, voice               elites, have to be replaced by research that
        and representation is a methodological                   promotes the interests of dominated, oppressed,
        tool.                                                    exploited groups.
        The acceptability and notion of objectivity and          The equal status and reciprocal relationships
        objective research must be challenged.                   between subjects and researchers are recog-
        The substantive, value-laden dimensions                  nized.
        and purposes of feminist research must be                There is a need to change the status quo, not
        paramount.                                               merely to understand or interpret it.
        Research must empower women.                             The research must be a process of conscientiza-
        Research need not be undertaken only by                  tion, not research solely by experts for experts,
        academic experts.                                        but to empower oppressed participants.
        Collective research is necessary: women need
                                                              Indeed Webb et al. (2004) set out six principles
        to collectivize their own individual histories
                                                              for a feminist pedagogy in the teaching of research
        if they are to appropriate these histories for
        There is a commitment to revealing                       reformulating the professor–student relation-
        core processes and recurring features of                 ship (from hierarchy to equality and sharing)
        women’s oppression.                                      ensuring empowerment (for a participatory
        There is an insistence on the inseparability of          democracy)
        theory and practice.                                     building community (through collaborative
        There is an insistence on the connections                learning)
        between the private and the public, between              privileging the individual voice (not only the
        the domestic and the political.                          lecturer’s)
                                                                                       FEMINIST RESEARCH        37

   respecting diversity of personal experience            The use of textual analysis such as deconstruc-

                                                                                                                     Chapter 1
   (rooted, for example, in gender, race, ethnicity,      tion of documents and texts about women.
   class, sexual preference)                              The use of meta-analysis to synthesize findings
   challenging traditional views (e.g. the socio-         from individual studies (see Chapter 13).
   logy of knowledge).                                    A move away from numerical surveys and a
                                                          critical evaluation of them, including a critique
Gender shapes research agendas, the choice of             of question wording.
topics and foci, the choice of data collection
techniques and the relationships between re-           Edwards and Mauthner (2002: 15, 27) characterize
searchers and researched. Several methodologi-         feminist research as that which concerns a
cal principles flow from a ‘rationale’ for fem-         critique of dominatory and value-free research,
inist research (Denzin 1989; Mies 1993; Haig           the surfacing and rejection of exploitative power
1997, 1999; De Laine 2000):                            hierarchies between the researcher and the
                                                       participants, and the espousal of close – even
   The replacement of quantitative, positivist, ob-    intimate – relationships between the researcher
   jective research with qualitative, interpretive,    and the researched. Positivist research is rejected as
   ethnographic reflexive research, as objectivity      per se oppressive (Gillies and Alldred 2002: 34) and
   in quantitative research is a smokescreen for       inherently unable to abide by its own principle of
   masculine interests and agendas.                    objectivity; it is a flawed epistemology. Research,
   Collaborative, collectivist research undertaken     and its underpinning epistemologies, are rooted
   by collectives – often of women – combining         in, and inseparable from interests (Habermas
   researchers and researched in order to              1972).
   break subject–object and hierarchical, non-            The move is towards ‘participatory action re-
   reciprocal relationships.                           search’ in which empowerment and emancipation
   The appeal to alleged value-free, neutral, indif-   are promoted and which is an involved and col-
   ferent and impartial research has to be replaced    laborative process (e.g. De Laine 2000: 109 ff.).
   by conscious, deliberate partiality – through       Participation recognizes ‘power imbalances and
   researchers identifying with participants.          the need to engage oppressed people as agents of
   The use of ideology-critical approaches and         their own change’ (Ezzy 2002: 44), while action
   paradigms for research.                             research recognizes the value of ‘using research
   The spectator theory or contemplative theory        findings to inform intervention decisions’ (p. 44).
   of knowledge in which researchers research          As De Laine (2000: 16) writes: the call is ‘for
   from ivory towers has to be replaced by             more participation and less observation, of be-
   a participatory approach – perhaps action           ing with and for the other, not looking at’, with
   research – in which all participants (including     relations of reciprocity and equality rather than
   researchers) engage in the struggle for women’s     impersonality, exploitation and power/status dif-
   emancipation – a liberatory methodology.            ferentials between researcher and participants.
   The need to change the status quo is the               The relationship between the researcher and
   starting point for social research – if we want     participant, De Laine argues, must break a
   to know something we change it. (Mies (1993)        conventional patriarchy. The emphasis is on
   cites the Chinese saying that if you want to        partnerships between researchers and participants,
   know a pear then you must chew it!).                to the extent that researchers are themselves
   The extended use of triangulation and multiple      participants rather than outsiders and the
   methods (including visual techniques such as        participants shape the research process as co-
   video, photograph and film).                         researchers (De Laine 2000: 107), defining the
   The use of linguistic techniques such as            problem, the methods, the data collection and
   conversational analysis.                            analysis, interpretation and dissemination. The

     relationship between researchers and participants        at friendship between researchers and participants
     is one of equality, and outsider, objective, distant,    are disingenuous, with ‘purported solidarity’ being
     positivist research relations are off the agenda;        a fraud perpetrated by well-intentioned feminists.
     researchers are inextricably bound up in the                Duncombe and Jessop (2002: 111) ask a
     lives of those they research. That this may bring        very searching question when they question
     difficulties in participant and researcher reactivity     whether, if interviewees are persuaded to take
     is a matter to be engaged rather than built out of       part in an interview by virtue of the researcher’s
     the research.                                            demonstration of empathy and ‘rapport’, this
        Thapar-Bj¨ rkert and Henry (2004) argue that          is really giving informed consent. They suggest
     the conventional, one-sided and unidirectional           that informed consent, particularly in exploratory
     view of the researcher as powerful and the research      interviews, has to be continually renegotiated and
     participants as less powerful, with the researcher       care has to be taken by the interviewer not to be
     exploiting and manipulating the researched, could        too intrusive. Personal testimonies, oral narratives
     be a construction by western white researchers.          and long interviews also figure highly in feminist
     They report research that indicates that power           approaches (De Laine 2000: 110; Thapar-Bj¨ rkerto
     is exercised by the researched as well as the            and Henry 2004), not least in those that touch
     researchers, and is a much more fluid, shifting and       on sensitive issues. These, it is argued (Ezzy 2002:
     negotiated matter than conventionally suggested,         45), enable women’s voices to be heard, to be
     being dispersed through both the researcher and          close to lived experiences, and avoid unwarranted
     the researched. Indeed they show how the research        assumptions about people’s experiences.
     participants can, and do, exercise considerable             The drive towards collective, egalitarian and
     power over the researchers both before, during           emancipatory qualitative research is seen as neces-
     and after the research process. They provide a           sary if women are to avoid colluding in their own
     fascinating example of interviewing women in             oppression by undertaking positivist, uninvolved,
     their homes in India, where, far from the home           dispassionate, objective research. Mies (1993: 67)
     being a location of oppression, it was a site of their   argues that for women to undertake this latter form
     power and control.                                       of research puts them into a schizophrenic position
        With regard to methods of data collection,            of having to adopt methods which contribute to
     Oakley (1981) suggests that ‘interviewing women’         their own subjugation and repression by ignoring
     in the standardized, impersonal style which              their experience (however vicarious) of oppres-
     expects a response to a prescribed agenda and            sion and by forcing them to abide by the ‘rules
     set of questions may be a ‘contradiction in              of the game’ of the competitive, male-dominated
     terms’, as it implies an exploitative relationship.      academic world. In this view, argue Roman and
     Rather, the subject–object relationship should be        Apple (1990: 59), it is not enough for women sim-
     replaced by a guided dialogue. She criticizes the        ply to embrace ethnographic forms of research, as
     conventional notion of ‘rapport’ in conducting           this does not necessarily challenge the existing and
     interviews (Oakley 1981: 35), arguing that they are      constituting forces of oppression or asymmetries of
     instrumental, non-reciprocal and hierarchical, all       power. Ethnographic research, they argue, has to
     of which are masculine traits. Rapport in this sense,    be accompanied by ideology critique; indeed they
     she argues, is not genuine in that the researcher        argue that the transformative, empowering, eman-
     is using it for scientific rather than human ends         cipatory potential of research is a critical standard
     (Oakley 1981: 55). Here researchers are ‘faking          for evaluating that piece of research.
     friendship’ for their own ends (Duncombe and                This latter point resonates with the call
     Jessop 2002: 108), equating ‘doing rapport’ with         by Lather (1991) for researchers to be concerned
     trust, and, thereby, operating a very ‘detached’         with the political consequences of their research
     form of friendship (p. 110). Similarly Thapar-           (e.g. consequential validity), not only the
     Bj¨ rkert and Henry (2004) suggest that attempts         conduct of the research and data analysis itself.
                                                                                     FEMINIST RESEARCH        39

Research must lead to change and improvement,         solve many ethical problems in research, as these

                                                                                                                   Chapter 1
particularly, in this context, for women (Gillies     are endemic in any form of fieldwork. She argues
and Alldred 2002: 32). Research is a political        that some feminist researchers may not wish to
activity with a political agenda (Gillies and         seek either less participation or more detachment,
Alldred 2002: 33; see also Lather 1991). Research     and that more detachment and less participation
and action – praxis – must combine ‘knowledge         are not solutions to ethical dilemmas and ‘morally
for’ as well as ‘knowledge what’ (Ezzy 2002:          responsible fieldwork’ as these, too, bring their
47). As Marx reminds us in his Theses on              own ethical dilemmas, e.g. the risk of threat. She
Feuerbach: ‘the philosophers have only interpreted    reports work (p. 113) that suggests that close
the world, in various ways; the point, however,       relationships between researchers and participants
is to change it’. Gillies and Alldred (2002: 45),     may be construed as just as exploitative, if more
however, point out that ‘many feminists have          disguised, as conventional researcher roles, and
agonized over whether politicizing participants       that they may bring considerable problems if data
is necessarily helpful’, as it raises awareness of    that were revealed in an intimate account between
constraints on their actions without being able       friends (researcher and participant) are then used
to offer solutions or to challenge their structural   in public research. The researcher is caught in a
causes. Research, thus politicized but unable to      dilemma: if she is a true friend then this imposes
change conditions, may actually be disempowering      constraints on the researcher, and yet if she is
and, indeed, patronizing in its simplistic call       only pretending to be a friend, or limiting that
for enlightenment and emancipation. It could          friendship, then this provokes questions of honesty
render women more vulnerable than before.             and personal integrity. Are research friendships
Emancipation is a struggle.                           real, ephemeral, or impression management used
   Several of these views of feminist research        to gather data?
and methodology are contested by other feminist          De Laine (2000: 115) suggests that it may be
researchers. For example, Jayaratne (1993: 109)       misguided to privilege qualitative research for its
argues for ‘fitness for purpose’, suggesting that      claim to non-exploitative relationships. While she
exclusive focus on qualitative methodologies          acknowledges that quantitative approaches may
might not be appropriate either for the research      perpetuate power differentials and exploitation,
purposes or, indeed, for advancing the feminist       there is no guarantee that qualitative research
agenda (see also Scott 1985: 82-3). Jayaratne         will not do the same, only in a more disguised
refutes the argument that quantitative methods        way. Qualitative approaches too, she suggests, can
are unsuitable for feminists because they neglect     create and perpetuate unequal relations, not least
the emotions of the people under study. Indeed she    simply because the researcher is in the field qua
argues for beating quantitative research on its own   researcher rather than a friend; if it were not for
grounds (Jayaratne 1993: 121), suggesting the need    the research then the researcher would not be
for feminist quantitative data and methodologies      present. Stacey (1988) suggests that the intimacy
in order to counter sexist quantitative data in       advocated for feminist ethnography may render
the social sciences. She suggests that feminist       exploitative relationships more rather than less
researchers can accomplish this without ‘selling      likely. We refer readers to Chapter 5 on sensitive
out’ to the positivist, male-dominated academic       educational research for a further discussion of
research community. Oakley (1998) suggests that       these issues.
the separation of women from quantitative                Gillies and Alldred (2002: 43-6) suggest that
methodology may have the unintended effect of         action research, an area strongly supported in
perpetuating women as the ‘other’, and, thereby,      some quarters of feminist researchers, is, itself,
discriminating against them.                          problematic. It risks being an intervention in
   De Laine (2000: 112) argues that shifting from     people’s lives (i.e. a potential abuse of power), and
quantitative to qualitative techniques may not        the researcher typically plays a significant, if not

     central, role in initiating, facilitating, crystallizing   rather than to be built out of the research
     and developing the meanings involved in, or                in the interests of objectivity (Edwards and
     stemming from, the research, i.e. the researcher           Mauthner 2002: 19). Emotions should not be
     is the one exercising power and influence.                  seen as disruptive of research or as irrelevant
        Ezzy (2002: 44) reports that, just as there is          (De Laine 2000: 151–2), but central to it,
     no single feminist methodology, both quantitative          just as they are central to human life. Indeed
     and qualitative methods are entirely legitimate.           emotional responses are essential in establishing
     Indeed, Kelly (1978) argues that a feminist                the veracity of inquiries and data, and the
     commitment should enter research at the stages of          ‘feminist communitarian model’ which De Laine
     formulating the research topic and interpreting the        (2000: 212–13) outlines values connectedness
     results, but it should be left out during the stages       at several levels: emotions, emotionality and
     of data collection and conduct of the research.            personal expressiveness, empathy. The egalitarian
        Thapar-Bj¨ rkert and Henry (2004) indicate              feminism that De Laine (2000: 108) and others
     that the researcher being an outsider might bring          advocate suggests a community of insiders in the
     more advantages than if she were an insider. For           same culture, in which empathy, reciprocity and
     example, being a white female researching non-             egalitarianism are hallmarks.
     white females may not be a handicap, as many                  Swantz (1996: 134) argues that there may be
     non-white women might disclose information to              some self-deception by the researcher in adopting
     white women that they would not disclose to a              a dual role as a researcher and one who shares
     non-white person. Similarly, having interviewers           the situation and interests of the participants.
     and interviewees of the same racial and ethnic             She questions the extent to which the researcher
     background does not mean that non-hierarchical             may be able to be genuinely involved with the
     relationships will still not be present. They also         participants in other than a peripheral way and
     report that the categories of ‘insider’ and ‘outsider’     whether, simply because the researcher may have
     were much more fuzzy than exclusive. Researchers           ‘superior knowledge’, a covert power differential
     are both ‘subject’ and ‘object’, and those being           may exist. De Laine (2000: 114) suggests that such
     researched are both ‘observed’ and ‘observers’.            superior knowledge may stem from the researcher’s
        De Laine (2000: 110) suggests that there                own background in anthropology or ethnography,
     is a division among feminists between those                or simply more education. The primary purpose
     who advocate closeness in relationships between            of the researcher is research, and that is different
     researchers and subjects – a human research-               from the primary purpose of the participants.
     ing fellow humans – and those who advocate                    Further, the researcher’s desire for identification
     ‘respectful distance’ between researchers and those        and solidarity with her research subjects may be
     being studied. Close relationships may turn into           pious but unrealistic optimism, not least because
     quasi-therapeutic situations rather than research          she may not share the same race, ethnicity,
     (Duncombe and Jessop 2002: 111), yet it may                background, life chances, experiences or colour
     be important to establish closeness in reaching            as those being researched. Indeed Gillies and
     deeper issues. Further, one has to question how far        Alldred (2002: 39–40) raise the question of how
     close relationships lead to reciprocal and mutual          far researchers can, or should, try to represent
     disclosure (p. 120). The debate is open: should the        groups to which they themselves do not belong,
     researcher share, be close and be prepared for more        not least those groups without power or voice,
     intimate social relations – a ‘feminist ethic of care’     as this, itself, is a form of colonization and
     (p. 111) – or keep those cool, outsider relations          oppression. Affinity, they argue (p. 40), is no
     which might objectify those being researched? It           authoritative basis for representative research.
     is a moral as well as a methodological matter.             Even the notion of affinity becomes suspect when
        The issue runs deep: the suggestion is that             it overlooks, or underplays, the significance of
     emotions and feelings are integral to the research,        difference, thereby homogenizing groups and their
                                                                             RESEARCH AND EVALUATION           41

particular experiences. In response to this, some       case of ‘categorically funded’ and commissioned

                                                                                                                    Chapter 1
feminist researchers (p. 40) suggest that researchers   research – research which is funded by policy-
only have the warrant to confine themselves to           makers (e.g. governments, fund-awarding bodies)
their own immediate communities, though this is         under any number of different headings that
a contentious issue. There is value in speaking for     those policy-makers devise (Burgess 1993). On
others, not least for those who are silenced and        the one hand, this is laudable, for it targets
marginalized, and in not speaking for others for        research directly towards policy; on the other
fear of oppression and colonization. One has to         hand, it is dangerous in that it enables others
question the acceptability and appropriateness of,      to set the research agenda. Research ceases to
and fidelity to, the feminist ethic, if one represents   become open-ended, pure research, and, instead,
and uses others’ stories (p. 41).                       becomes the evaluation of given initiatives. Less
   An example of a feminist approach to research        politically charged, much research is evaluative,
is the Girls Into Science and Technology (GIST)         and indeed there are many similarities between
action research project. This took place over three     research and evaluation. The two overlap but
years, involving 2,000 students and their teachers      possess important differences. The problem of
in ten coeducational, comprehensive schools in          trying to identify differences between evaluation
one area of the United Kingdom, eight schools           and research is compounded because not only do
serving as the bases of the ‘action’, the remaining     they share several of the same methodological
two acting as ‘controls’. Several publications have     characteristics but also one branch of research is
documented the methodologies and findings of             called evaluative research or applied research. This
the GIST study (Kelly 1986; 1989a; 1989b; Kelly         is often kept separate from ‘blue skies’ research
and Smail 1986; Whyte 1986), described by               in that the latter is open-ended, exploratory,
its co-director as ‘simultaneous-integrated action      contributes something original to the substantive
research’ (Kelly 1987) (i.e. integrating action         field and extends the frontiers of knowledge and
and research). Kelly is open about the feminist         theory whereas in the former the theory is given
orientation of the GIST project team, seeking           rather than interrogated or tested. One can detect
deliberately to change girls’ option choices and        many similarities between the two in that they
career aspirations, because the researchers saw         both use methodologies and methods of social
that girls were disadvantaged by traditional sex-       science research generally, covering, for example
stereotypes. The researchers’ actions, she suggests,    (see
were a small attempt to ameliorate women’s              9780415368780 – Chapter 1, file 1.9. ppt), the
subordinate social position (Kelly 1987).               following:

Research and evaluation
                                                           the need to clarify the purposes of the
The preceding discussion has suggested that                investigation
research and politics are inextricably bound               the need to operationalize purposes and areas of
together. This can be taken further, as researchers        investigation
in education will be advised to pay serious                the need to address principles of research design
consideration to the politics of their research            that include:
enterprise and the ways in which politics can                 formulating operational questions
steer research. For example, one can detect                   deciding appropriate methodologies
a trend in educational research towards more                  deciding which instruments to use for data
evaluative research, where, for example, a                    collection
researcher’s task is to evaluate the effectiveness            deciding on the sample for the investigation
(often of the implementation) of given policies               addressing reliability and validity in the
and projects. This is particularly true in the                investigation and instrumentation

           addressing ethical issues in conducting the       set out some of the differences between eval-
           investigation                                     uation and research. For example Smith and
           deciding on data analysis techniques              Glass (1987) offer eight main differences (see
           deciding on reporting and interpreting results.
                                                             9780415368780 – Chapter 1, file 1.11. ppt):
     Indeed Norris (1990) argues that evaluation
     applies research methods to shed light on a                The intents and purposes of the investigation: the
     problem of action (Norris 1990: 97); he suggests           researcher wants to advance the frontiers of
     that evaluation can be viewed as an extension of           knowledge of phenomena, to contribute to
     research, because it shares its methodologies and          theory and to be able to make generalizations;
     methods, and because evaluators and researchers            the evaluator is less interested in contributing
     possess similar skills in conducting investigations        to theory or the general body of knowledge.
     (see                   Evaluation is more parochial than universal
     9780415368780 – Chapter 1, file 1.10. ppt). In              (Smith and Glass 1987: 33–4).
     many senses the eight features outlined above              The scope of the investigation: evaluation studies
     embrace many elements of the scientific method,             tend to be more comprehensive than research
     which Smith and Glass (1987) set out in seven              in the number and variety of aspects of a
     steps:                                                     programme that are being studied (p. 34).
     1   A theory about the phenomenon exists.                  Values in the investigation: research aspires to
     2   A research problem within the theory is                value neutrality, evaluations must represent
         detected and a research question is devised.           multiple sets of values and include data on
     3   A research hypothesis is deduced (often about          these values.
         the relationship between constructs).                  The origins of the study: research has its origins
     4   A research design is developed, operationalizing       and motivation in the researcher’s curiosity
         the research question and stating the null             and desire to know (p. 34). The researcher is
         hypothesis.                                            answerable to colleagues and scientists (i.e. the
     5   The research is conducted.                             research community) whereas the evaluator
     6   The null hypothesis is tested based on the             is answerable to the ‘client’. The researcher
         data gathered.                                         is autonomous whereas the evaluator is
     7   The original theory is revised or supported            answerable to clients and stakeholders. The
         based on the results of the hypothesis testing.        researcher is motivated by a search for
                                                                knowledge, the evaluator is motivated by the
     Indeed, if steps 1 and 7 were removed then there           need to solve problems, allocate resources and
     would be nothing to distinguish between research           make decisions. Research studies are public,
     and evaluation. Both researchers and evaluators            evaluations are for a restricted audience.
     pose questions and hypotheses, select samples,             The uses of the study: the research is used to
     manipulate and measure variables, compute statis-          further knowledge, evaluations are used to
     tics and data, and state conclusions. Never-               inform decisions.
     theless there are important differences between            The timeliness of the study: evaluations must be
     evaluation and research that are not always ob-            timely, research need not be. Evaluators’ time
     vious simply by looking at publications. Publica-          scales are given, researchers’ time scales need
     tions do not always make clear the background              not be given.
     events that gave rise to the investigation, nor            Criteria for judging the study: evaluations are
     do they always make clear the uses of the ma-              judged by the criteria of utility and credibility,
     terial that they report, nor do they always make           research is judged methodologically and by the
     clear what the dissemination rights (Sanday 1993)          contribution that it makes to the field (i.e.
     are and who holds them. Several commentators               internal and external validity).
                                                                                RESEARCH AND EVALUATION                43

   The agendas of the study: an evaluator’s agenda          Disciplinary base: the researcher can afford to

                                                                                                                            Chapter 1
   is given, a researcher’s agenda is his or her own.       pursue inquiry within one discipline and the
                                                            evaluator cannot.
  Norris (1990) reports an earlier piece of
work by Glass and Worthen (1971) in which                   A clue to some of the differences between
they identified eleven main differences between           evaluation and research can be seen in the
evaluation and research:                                 definition of evaluation. Most definitions of
                                                         evaluation include reference to several key
   The motivation of the inquirer: research is           features:
   pursued largely to satisfy curiosity, evaluation
                                                            answering specific, given questions
   is undertaken to contribute to the solution of
                                                            gathering information
   a problem.
                                                            making judgements
   The objectives of the research: research and
                                                            taking decisions
   evaluation seek different ends. Research seeks
                                                            addressing the politics of a situation (Morrison
   conclusions, evaluation leads to decisions.
                                                            1993: 2).
   Laws versus description: research is the quest for
   laws (nomothetic), evaluation merely seeks to         (See
   describe a particular thing (idiographic).            9780415368780 – Chapter 1, file 1.12. ppt.) Mor-
   The role of explanation: proper and useful evalu-     rison (1993: 2) provides one definition of evalu-
   ation can be conducted without producing an           ation as: the provision of information about specified
   explanation of why the product or project is          issues upon which judgements are based and from
   good or bad or of how it operates to produce its      which decisions for action are taken. This view
   effects.                                              echoes MacDonald (1987) in his comments that
   The autonomy of the inquiry: evaluation is            the evaluator
   undertaken at the behest of a client, while
   researchers set their own problems.                     is faced with competing interest groups, with diver-
   Properties of the phenomena that are assessed:          gent definitions of the situation and conflicting infor-
   evaluation seeks to assess social utility directly,     mational needs . . . . He has to decide which decision-
   research may yield evidence of social utility but       makers he will serve, what information will be of most
   often only indirectly.                                  use, when it is needed and how it can be obtained . . . .
   Universality of the phenomena studied:                  The resolution of these issues commits the evaluator
   researchers work with constructs having a               to a political stance, an attitude to the government of
   currency and scope of application that make             education. No such commitment is required of the re-
   the objects of evaluation seem parochial by             searcher. He stands outside the political process, and
   comparison.                                             values his detachment from it. For him the production
   Salience of the value question: in evaluation value     of new knowledge and its social use are separated. The
   questions are central and usually determine             evaluator is embroiled in the action, built into a polit-
   what information is sought.                             ical process which concerns the distribution of power,
   Investigative techniques: while there may be            i.e. the allocation of resources and the determination
   legitimate differences between research and             of goals, roles and tasks . . . . When evaluation data
   evaluation methods, there are far more                  influences power relationships the evaluator is com-
   similarities than differences with regard to            pelled to weight carefully the consequences of his task
   techniques and procedures for judging validity.         specification . . . . The researcher is free to select his
   Criteria for assessing the activity: the two most       questions, and to seek answers to them. The evalua-
   important criteria for judging the adequacy of          tor, on the other hand, must never fall into the error
   research are internal and external validity, for        of answering questions which no one but he is asking.
   evaluation they are utility and credibility.                                              (MacDonald 1987: 42)

     MacDonald (1987) argues that evaluation is an                 (e.g. the UK Economic and Social Research
     inherently political enterprise. His much-used                Council) a move towards sponsoring policy-
     threefold typification of evaluations as autocratic,           oriented projects rather than the ‘blue-skies’
     bureaucratic and democratic is premised on a                  research mentioned earlier. Indeed Burgess (1993:
     political reading of evaluation (see also Chelinsky           1) argues that ‘researchers are little more than
     and Mulhauser (1993: 54) who refer to ‘the                    contract workers . . . research in education must
     inescapability of politics’ in the world of                   become policy relevant . . . research must come
     evaluation). MacDonald (1987: 101), noting that               closer to the requirement of practitioners’.
     ‘educational research is becoming more evaluative                This view is reinforced by several chapters in
     in character’, argues for research to be kept out             the collection edited by Anderson and Biddle
     of politics and for evaluation to square up to the            (1991) which show that research and politics go
     political issues at stake:                                    together uncomfortably because researchers have
                                                                   different agendas and longer time scales than
       The danger therefore of conceptualizing evaluation as
                                                                   politicians and try to address the complexity of
       a branch of research is that evaluators become trapped
                                                                   situations, whereas politicians, anxious for short-
       in the restrictive tentacles of research respectability.
                                                                   term survival want telescoped time scales, simple
       Purity may be substituted for utility, trivial proofs for
                                                                   remedies and research that will be consonant with
       clumsy attempts to grasp complex significance. How
                                                                   their political agendas. Indeed James (1993) argues
       much more productive it would be to define research
       as a branch of evaluation, a branch whose task it is
       to solve the technological problems encountered by            the power of research-based evaluation to provide
       the evaluator.                                                evidence on which rational decisions can be expected
                                       (MacDonald 1987: 43)          to be made is quite limited. Policy-makers will
                                                                     always find reasons to ignore, or be highly selective
     However, the truth of the matter is far more
                                                                     of, evaluation findings if the information does not
     blurred than these distinctions suggest. Two
                                                                     support the particular political agenda operating at
     principal causes of this blurring lie in the
                                                                     the time when decisions have to be made.
     funding and the politics of both evaluation and
                                                                                                        (James 1993: 135)
     research. For example, the view of research as
     uncontaminated by everyday life is na¨ve and                  The politicization of research has resulted in
     simplistic; Norris (1990: 99) argues that such an             funding bodies awarding research grants for
     antiseptic view of research ignores the social                categorical research that specify time scales and
     context of educational research, some of which                the terms of reference. Burgess’s (1993) view also
     is located in the hierarchies of universities and             points to the constraints under which research
     research communities and the funding support                  is undertaken; if it is not concerned with policy
     provided for some research projects but not all               issues then research tends not to be funded. One
     by governments. His point has a pedigree that                 could support Burgess’s view that research must
     reaches back to Kuhn (1962), and is commenting                have some impact on policy-making.
     on the politics of research funding and research                 Not only is research becoming a political issue,
     utilization. Since the early 1980s one can detect             but also this extends to the use being made of
     a huge rise in ‘categorical’ funding of projects,             evaluation studies. It was argued above that evalua-
     i.e. defined, given projects (often by government              tions are designed to provide useful data to inform
     or research sponsors) for which bids have to                  decision-making. However, as evaluation has be-
     be placed. This may seem unsurprising if one                  come more politicized so its uses (or non-uses) have
     is discussing research grants by government                   become more politicized. Indeed Norris (1990)
     bodies, which are deliberately policy-oriented,               shows how politics frequently overrides evalu-
     though one can also detect in projects that have              ation or research evidence. Norris (1990: 135)
     been granted by non-governmental organizations                writes that the announcement of the decision
                                                                               RESEARCH AND EVALUATION           45

to extend the TVEI project was made with-                    community, whereby individuals come to

                                                                                                                      Chapter 1
out any evaluation reports having been received              hold divergent public and private opinions,
from evaluation teams in Leeds or the National               or offer criticisms in general rather than
Foundation for Educational Research. (The Tech-              in particular, or quietly develop ‘academic’
nical and Vocational Education Initiative (TVEI)             critiques which are at variance with their
was a 1980s UK government-funded project fre-                contractual evaluation activities, alternating
quently targeted to lower-attaining students.) This          between ‘critical’ and ‘conformative’ selves.
echoes James (1993) where she writes:
                                                             The argument so far has been confined to
  The classic definition of the role of evaluation as      large-scale projects that are influenced by and
  providing information for decision-makers . . . is a    may or may not influence political decision-
  fiction if this is taken to mean that policy-makers      making. However, the argument need not remain
  who commission evaluations are expected to make         there. Morrison (1993), for example, indicates
  rational decisions based on the best (valid and         how evaluations might influence the ‘micro-
  reliable) information available to them.                politics of the school’. Hoyle (1986), for example,
                                      (James 1993: 119)   asks whether evaluation data are used to bring
                                                          resources into, or take resources out of, a
  Where evaluations are commissioned and
                                                          department or faculty. The issue does not relate
have heavily political implications, Stronach and
                                                          only to evaluations, for school-based research, far
Morris (1994) argue that the response to this is
                                                          from the emancipatory claims for it made by action
that evaluations become more ‘conformative’ (see
                                                          researchers (e.g. Carr and Kemmis 1986; Grundy
                                                          1987), is often concerned more with finding out
9780415368780 – Chapter 1, file 1.13. ppt),
                                                          the most successful ways of organization, planning,
possessing several characteristics:
                                                          teaching and assessment of a given agenda rather
                                                          than setting agendas and following one’s own
   Being short-term, taking project goals as given
                                                          research agendas. This is problem-solving rather
   and supporting their realization.
                                                          than problem-setting. That evaluation and research
   Ignoring the evaluation of longer-term learning
                                                          are being drawn together by politics at both macro-
   outcomes, or anticipated economic or social
                                                          level and micro-level is evidence of a growing
   consequences of the programme.
                                                          interventionism by politics into education, thus
   Giving undue weight to the perceptions of
                                                          reinforcing the hegemony of the government in
   programme participants who are responsible
                                                          power. Several points have been made here:
   for the successful development and implemen-
   tation of the programme; as a result, tending to          There is considerable overlap between
   ‘over-report’ change.                                     evaluation and research.
   Neglecting and ‘under-reporting’ the views of             There are some conceptual differences between
   classroom practitioners, and programme critics.           evaluation and research, though, in practice,
   Adopting an atheoretical approach, and                    there is considerable blurring of the edges of
   generally regarding the aggregation of opinion            the differences between the two.
   as the determination of overall significance.              The funding and control of research and
   Involving a tight contractual relationship with           research agendas reflect the persuasions of
   the programme sponsors that either disbars                political decision-makers.
   public reporting, or encourages self-censorship           Evaluative research has increased in response
   in order to protect future funding prospects.             to categorical funding of research projects.
   Undertaking various forms of implicit advocacy            The attention being given to, and utilization of,
   for the programme in its reporting style.                 evaluation varies according to the consonance
   Creating and reinforcing a professional                   between the findings and their political
   schizophrenia in the research and evaluation              attractiveness to political decision-makers.

     In this sense the views expressed earlier               piece of research does not feed simplistically
     by MacDonald (1987) are now little more than an         or directly into a specific piece of policy-
     historical relic; there is very considerable blurring   making. Rather, research generates a range
     of the edges between evaluation and research            of different types of knowledge – concepts,
     because of the political intrusion into, and use        propositions, explanations, theories, strategies,
     of, these two types of study. One response to           evidence, methodologies (Caplan 1991). These
     this can be seen in Burgess’s (1993) view that a        feed subtly and often indirectly into the
     researcher needs to be able to meet the sponsor’s       decision-making process, providing, for example,
     requirements for evaluation while also generating       direct inputs, general guidance, a scientific
     research data (engaging the issues of the need to       gloss, orienting perspectives, generalizations and
     negotiate ownership of the data and intellectual        new insights. Basic and applied research have
     property rights).                                       significant parts to play in this process.
                                                                The degree of influence exerted by research
                                                             depends on careful dissemination; too little and
     Research, politics and policy-making                    its message is ignored, too much and data
     The preceding discussion has suggested that             overload confounds decision-makers and makes
     there is an inescapable political dimension to          them cynical – the syndrome of the boy who
     educational research, both in the macro- and            cried wolf (Knott and Wildavsky 1991). Hence
     micro-political senses. In the macro-political sense    researchers must give care to utilization by policy-
     this manifests itself in funding arrangements,          makers (Weiss 1991a), reduce jargon, provide
     where awards are made provided that the research        summaries, and improve links between the two
     is ‘policy-related’ (Burgess 1993) – guiding policy     cultures of researchers and policy-makers (Cook
     decisions, improving quality in areas of concern        1991) and, further, to the educational community.
     identified by policy-makers, facilitating the            Researchers must cultivate ways of influencing
     implementation of policy decisions, evaluating          policy, particularly when policy-makers can
     the effects of the implementation of policy.            simply ignore research findings, commission
     Burgess notes a shift here from a situation where       their own research (Cohen and Garet 1991) or
     the researcher specifies the topic of research           underfund research into social problems (Coleman
     and towards the sponsor specifying the focus of         1991; Thomas 1991). Researchers must recognize
     research. The issue of sponsoring research reaches      their links with the power groups who decide
     beyond simply commissioning research towards            policy. Research utilization takes many forms
     the dissemination (or not) of research – who will       depending on its location in the process of
     receive or have access to the findings and how           policy-making, e.g. in research and development,
     the findings will be used and reported. This, in         problem solving, interactive and tactical models
     turn, raises the fundamental issue of who owns          (Weiss 1991b). Researchers will have to judge
     and controls data, and who controls the release         the most appropriate forms of utilization of their
     of research findings. Unfavourable reports might         research (Alkin et al. 1991).
     be withheld for a time, suppressed or selectively          The impact of research on policy-making
     released! Research can be brought into the service      depends on its degree of consonance with the
     of wider educational purposes – the politics of a       political agendas of governments (Thomas 1991)
     local education authority, or indeed the politics of    and policy-makers anxious for their own political
     government agencies.                                    survival (Cook 1991) and the promotion of their
        Though research and politics intertwine,             social programmes. Research is used if it is
     the relationships between educational research,         politically acceptable. That the impact of research
     politics and policy-making are complex because          on policy is intensely and inescapably political
     research designs strive to address a complex            is a truism (Horowitz and Katz 1991; Kamin
     social reality (Anderson and Biddle 1991); a            1991; Selleck 1991; Wineburg 1991). Research
                                                                         METHODS AND METHODOLOGY               47

too easily becomes simply an ‘affirmatory text’         worthwhile research knowledge, what constitutes

                                                                                                                    Chapter 1
which ‘exonerates the system’ (Wineburg 1991)          acceptable focuses and methodologies of research
and is used by those who seek to hear in it only       and how the findings will be used.
echoes of their own voices and wishes (Kogan and
Atkin 1991).
   There is a significant tension between               Methods and methodology
researchers and policy-makers. The two parties
                                                       We return to our principal concern, methods and
have different, and often conflicting, interests,
                                                       methodology in educational research. By methods,
agendas, audiences, time scales, terminology,
                                                       we mean that range of approaches used in educa-
and concern for topicality (Levin 1991). These
                                                       tional research to gather data which are to be
have huge implications for research styles.
                                                       used as a basis for inference and interpretation,
Policy-makers anxious for the quick fix of
                                                       for explanation and prediction. Traditionally, the
superficial facts, short-term solutions and simple
                                                       word refers to those techniques associated with the
remedies for complex and generalized social
                                                       positivistic model – eliciting responses to prede-
problems (Cartwright 1991; Cook 1991) – the
                                                       termined questions, recording measurements, de-
Simple Impact model (Biddle and Anderson
                                                       scribing phenomena and performing experiments.
1991; Weiss 1991a; 1991b) – find positivist
                                                       For our purposes, we will extend the meaning to
methodologies attractive, often debasing the
                                                       include not only the methods of normative re-
data through illegitimate summary. Moreover,
                                                       search but also those associated with interpretive
policy-makers find much research uncertain in
                                                       paradigms – participant observation, role-playing,
its effects (Cohen and Garet 1991; Kerlinger
                                                       non-directive interviewing, episodes and accounts.
1991), dealing in a Weltanschauung rather
                                                       Although methods may also be taken to include
than specifics, and being too complex in
                                                       the more specific features of the scientific enter-
its designs and of limited applicability (Finn
                                                       prise such as forming concepts and hypotheses,
1991). This, reply the researchers, misrepresents
                                                       building models and theories, and sampling pro-
the nature of their work (Shavelson and
                                                       cedures, we will limit ourselves principally to the
Berliner 1991) and belies the complex reality
                                                       more general techniques which researchers use.
which they are trying to investigate (Blalock
                                                          If methods refer to techniques and procedures
1991). Capturing social complexity and serving
                                                       used in the process of data-gathering, the aim of
political utility can run counter to each
                                                       methodology then is to describe approaches to,
                                                       kinds and paradigms of research (Kaplan 1973).
   The issue of the connection between research
                                                       Kaplan suggests that the aim of methodology is
and politics – power and decision-making – is
                                                       to help us to understand, in the broadest possible
complex. On another dimension, the notion that
                                                       terms, not the products of scientific inquiry but
research is inherently a political act because it is
                                                       the process itself.
part of the political processes of society has not
                                                          We, for our part, will attempt to present
been lost on researchers. Usher and Scott (1996:
                                                       normative and interpretive perspectives in a
176) argue that positivist research has allowed a
                                                       complementary light and will try to lessen the
traditional conception of society to be preserved
                                                       tension that is sometimes generated between
relatively unchallenged – the white, male, middle-
                                                       them. Merton and Kendall (1946)7 express the
class researcher – to the relative exclusion of
                                                       same sentiment:
‘others’ as legitimate knowers. That this reaches
into epistemological debate is evidenced in the          Social scientists have come to abandon the spurious
issues of who defines the ‘traditions of knowledge’       choice between qualitative and quantitative data:
and the disciplines of knowledge; the social             they are concerned rather with that combination of
construction of knowledge has to take into account       both which makes use of the most valuable features
the differential power of groups to define what is        of each. The problem becomes one of determining at

       which points they should adopt the one, and at which   we have in mind the systematic and scholarly
       the other, approach.                                   application of the principles of a science of
                                (Merton and Kendall 1946)     behaviour to the problems of people within
                                                              their social contexts and when we use the
        The term research itself may take on a range of       term educational research, we likewise have in
     meanings and thereby be legitimately applied to a        mind the application of these same principles
     variety of contexts from, say, an investigation          to the problems of teaching and learning within
     into the techniques of Dutch painters of the             the formal educational framework and to the
     seventeenth century to the problem of finding             clarification of issues having direct or indirect
     more efficient means of improving traffic flow in           bearing on these concepts.
     major city centres. For our purposes, however,              The particular value of scientific research in
     we will restrict its usages to those activities          education is that it will enable educators to
     and undertakings aimed at developing a science           develop the kind of sound knowledge base that
     of behaviour, the word science itself implying           characterizes other professions and disciplines; and
     both normative and interpretive perspectives.            one that will ensure education a maturity and sense
     Accordingly, when we speak of social research,           of progression it at present lacks.
Part Two

Planning educational research

The planning of educational research is not            readers through them systematically. In addition,
an arbitrary matter; the research itself is an         a new chapter on sensitive educational research
inescapably ethical enterprise. We place ethical       is included here, taking sensitivity not only in
issues at a very early point in the book to signal     terms of content, but also in terms of process,
this. The research community and those using           purpose, outcome and usage. This new chapter
the findings have a right to expect that research       also makes the point that often access itself is
be conducted rigorously, scrupulously and in an        a sensitive matter, and this could be the major
ethically defensible manner. All this necessitates     issue to be faced in planning research. This part
careful planning, and this part introduces some        sets out a range of planning possibilities so that
key planning issues. In planning research, we need     the eventual selection of sampling procedures,
to consider the issues of sampling, reliability and    versions of reliability and validity are made on the
validity at the very outset, and this part addresses   basis of fitness for purpose, and so that sensitivities
these. These are complex issues, and we take           in research are anticipated and addressed.
2        The ethics of educational and social research

Introduction                                           should not be, forced into a procrustean sys-
                                                       tem of ethics. When it comes to the reso-
The awareness of ethical concerns in research
                                                       lution of a specific moral problem, each sit-
is reflected in the growth of relevant literature
                                                       uation frequently offers a spectrum of possi-
and in the appearance of regulatory codes of
                                                       bilities (see
research practice formulated by various agencies
                                                       9780415368780 – Chapter 2, file 2.1. ppt).
and professional bodies.1 A major ethical dilemma
                                                          In this chapter we review seriatim several issues
is that which requires researchers to strike a
                                                       in the ethical field. These can constitute a set
balance between the demands placed on them
                                                       of initial considerations that researchers should
as professional scientists in pursuit of truth,
                                                       address in planning research:
and their subjects’ rights and values potentially
threatened by the research. This is known as              informed consent
the ‘costs/benefits ratio’, the essence of which           gaining access to and acceptance in the
is outlined by Frankfort-Nachmias and Nachmias            research setting
(1992) in Box 2.1, and is a concept we return             the nature of ethics in social research generally
to later in the chapter. Ethical problems for             sources of tension in the ethical debate,
researchers can multiply surprisingly when they           including non-maleficence, beneficence and
move from the general to the particular, and from         human dignity, absolutist and relativist ethics
the abstract to the concrete.                             problems and dilemmas confronting the re-
   Ethical issues may stem from the kinds of              searcher, including matters of privacy, ano-
problems investigated by social scientists and the        nymity, confidentiality, betrayal and deception
methods they use to obtain valid and reliable             ethical problems endemic in particular research
data. This means that each stage in the research          methods
sequence raises ethical issues. They may arise            ethics and evaluative research
from the nature of the research project itself            regulatory ethical frameworks, guidelines and
(ethnic differences in intelligence, for example);        codes of practice for research
the context for the research (a remand home);             personal codes of practice
the procedures to be adopted (producing high              sponsored research
levels of anxiety); methods of data collection            responsibilities to the research community.
(covert observation); the nature of the participants
(emotionally disturbed adolescents); the type of         While many of these issues concern procedural
data collected (highly personal and sensitive          ethics, we have to recall that ethics concern right
information); and what is to be done with the          and wrong, good and bad, and so procedural
data (publishing in a manner that may cause            ethics are not enough; one has to consider
participants embarrassment).                           how the research purposes, contents, methods,
   In this chapter we present a conspectus of the      reporting and outcomes abide by ethical principles
main issues that may confront researchers. Each        and practices. Before this, however, we examine
research undertaking is an event sui generis, and      another fundamental concept which, along
the conduct of researchers cannot be, indeed           with the costs/benefits ratio, contributes to the

     Box 2.1                                                       control over what happens (e.g. in drug research);
     The costs/benefits ratio                                       such informed consent requires full information
                                                                   about the possible consequences and dangers.
       The costs/benefits ratio is a fundamental concept               The principle of informed consent arises
       expressing the primary ethical dilemma in social            from the subject’s right to freedom and self-
       research. In planning their proposed research, social
                                                                   determination. Being free is a condition of living in
       scientists have to consider the likely social benefits
       of their endeavours against the personal costs to the       a democracy, and when restrictions and limitations
       individuals taking part. Possible benefits accruing from     are placed on that freedom they must be justified
       the research may take the form of crucial findings           and consented to, as in research. Consent thus pro-
       leading to significant advances in theoretical and applied   tects and respects the right of self-determination
       knowledge. Failure to do the research may cost society
                                                                   and places some of the responsibility on the partic-
       the advantages of the research findings and ultimately
       the opportunity to improve the human condition. The         ipant should anything go wrong in the research. As
       costs to participants may include affronts to dignity,      part of the right to self-determination, the subject
       embarrassment, loss of trust in social relations, loss      has the right to refuse to take part, or to with-
       of autonomy and self-determination, and lowered self-       draw once the research has begun (see Frankfort-
       esteem. On the other hand, the benefits to participants
                                                                   Nachmias and Nachmias 1992). Thus informed
       could take the form of satisfaction in having made
       a contribution to science and a greater personal            consent implies informed refusal.
       understanding of the research area under scrutiny.             Informed consent has been defined by Diener
       The process of balancing benefits against possible costs     and Crandall (1978) as ‘the procedures in which
       is chiefly a subjective one and not at all easy. There       individuals choose whether to participate in an
       are few or no absolutes and researchers have to make
                                                                   investigation after being informed of facts that
       decisions about research content and procedures in
       accordance with professional and personal values. This      would be likely to influence their decisions’. This
       costs/benefits ratio is the basic dilemma residual in a      definition involves four elements: competence,
       great deal of social research.                              voluntarism, full information and comprehension.
                                                                      Competence implies that responsible, mature
     Source: adapted from Frankfort-Nachmias and Nachmias          individuals will make correct decisions if they
     1992                                                          are given the relevant information. It is
                                                                   incumbent on researchers to ensure they do
                                                                   not engage individuals incapable of making such
     bedrock of ethical procedure – that of informed               decisions because of immaturity or some form of
     consent.                                                      psychological impairment.
                                                                      Voluntarism entails applying the principle
                                                                   of informed consent and thus ensuring that
     Informed consent                                              participants freely choose to take part (or not)
     Much social research necessitates obtaining the               in the research and guarantees that exposure to
     consent and cooperation of subjects who are to                risks is undertaken knowingly and voluntarily.
     assist in investigations and of significant others             This element can be problematical, especially in
     in the institutions or organizations providing                the field of medical research where unknowing
     the research facilities (see http://www.routledge.            patients are used as guinea-pigs.
     com/textbooks/9780415368780 – Chapter 2, file                     Full information implies that consent is fully
     2.2. ppt). While some cultures may not be stringent           informed, though in practice it is often impossible
     about informed consent, in others there are                   for researchers to inform subjects on everything,
     strict protocols for informed consent. Frankfort-             e.g. on the statistical treatment of data; and, as
     Nachmias and Nachmias (1992) suggest that                     we shall see below, on those occasions when the
     informed consent is particularly important if                 researchers themselves do not know everything
     participants are going to be exposed to any stress,           about the investigation. In such circumstances,
     pain, invasion of privacy, or if they are going to lose       the strategy of reasonably informed consent has to
                                                                                          INFORMED CONSENT          53

be applied. Box 2.2 illustrates a set of guidelines        or contents, e.g. the Milgram experiments, see

                                                                                                                         Chapter 2
used in the United States that are based on the            Chapter 21), she argues that it may actually
idea of reasonably informed consent.2                      confuse the respondents.
   Comprehension refers to the fact that participants         It must also be remembered that there are
fully understand the nature of the research project,       some research methods where it is impossible
even when procedures are complicated and entail            to seek informed consent. Covert observation,
risks. Suggestions have been made to ensure                for example, as used in Patrick’s (1973)
that subjects fully comprehend the situation               study of a Glasgow gang (Chapter 11), or
they are putting themselves into, e.g. by using            experimental techniques involving deception,
highly educated subjects, by engaging a consultant         as in Milgram’s (1974) obedience-to-authority
to explain difficulties, or by building into the            experiments (Chapter 21), would, by their very
research scheme a time lag between the request             nature, rule out the option. And, of course, there
for participation and decision time.                       may be occasions when problems arise even though
   If these four elements are present, researchers         consent has been obtained. Burgess (1989), for
can be assured that subjects’ rights will have been        example, cites his own research in which teachers
given appropriate consideration. As Frankfort-             had been informed that research was taking place
Nachmias and Nachmias (1992) note, however,                but in which it was not possible to specify exactly
informed consent may not always be necessary (e.g.         what data would be collected or how they would
deception may be justified), but that, as a general         be used. It could be said, in this particular case,
rule, the greater the risk, the more important it is       that individuals were not fully informed, that
to gain informed consent.                                  consent had not been obtained, and that privacy
   Ruane (2005: 21) also raises the question of ‘how       had been violated. As a general rule, however,
much information is enough’; she argues that this          informed consent is an important principle. It is
may be an unknown, not necessarily deliberately            this principle that will form the basis of an implicit
withheld. Further, just as providing information           contractual relationship between the researcher
may bias the results (i.e. it is important for the         and the researched and will serve as a foundation
integrity of the research not to disclose its purposes     on which subsequent ethical considerations can
                                                           be structured.
                                                              From the remarks on informed consent so far, we
Box 2.2                                                    may appear to be assuming relationships between
Guidelines for reasonably informed consent                 peers – researcher and teachers, for example, or
                                                           research professor and postgraduate students – and
  1 A fair explanation of the procedures to be followed    this assumption would seem to underpin many of
    and their purposes.                                    the discussions of an ethical nature in the research
  2 A description of the attendant discomforts and risks   literature generally. However, much educational
    reasonably to be expected.                             research involves children who cannot be regarded
  3 A description of the benefits reasonably to be
                                                           as being on equal terms with the researcher and
  4 A disclosure of appropriate alternative procedures     it is important to keep this in mind at all stages
    that might be advantageous to the participants.        in the research process, including the point where
  5 An offer to answer any inquiries concerning the        informed consent is sought. In this connection we
    procedures.                                            refer to the important work of Fine and Sandstrom
  6 An instruction that the person is free to withdraw
    consent and to discontinue participation in the
                                                           (1988), whose ethnographic and participant
    project at any time without prejudice to the           observational studies of children and young people
    participant.                                           focus, among other issues, on this asymmetry with
                                                           respect to the problems of obtaining informed
Source: US Department of Health, Education and Welfare     consent from their young subjects and explaining
et al. 1971                                                the research in a comprehensible fashion. As a

     guiding principle, they advise that, while it is       rapport with the group, those who refused initially
     desirable to lessen the power differential between     may be approached again, perhaps in private.
     children and adult researchers, the difference            Two particular groups of children require
     will remain and its elimination may be ethically       special mention: very young children, and those
     inadvisable.                                           not capable of making a decision. Researchers
         There are other aspects of the problem of          intending to work with pre-school or nursery
     informed consent (or refusal) in relation to young,    children may dismiss the idea of seeking informed
     or very young, children. Seeking informed consent      consent from their would-be subjects because
     with regard to minors involves two stages. First,      of their age, but Fine and Sandstrom (1988)
     researchers consult and seek permission from those     would recommend otherwise. Even though such
     adults responsible for the prospective subjects,       children would not understand what research was,
     and second, they approach the young people             the authors advise that the children be given
     themselves. The adults in question will be, for        some explanation. For example, one to the effect
     example, parents, teachers, tutors, psychiatrists,     that an adult will be watching and playing with
     youth leaders, or team coaches, depending on the       them might be sufficient to provide a measure of
     research context. The point of the research will       informed consent consistent with the children’s
     be explained, questions invited, and permission to     understanding. As Fine and Sandstrom comment:
     proceed to the next stage sought. Objections, for
                                                              Our feeling is that children should be told as much
     whatever reason, will be duly respected. Obtaining
                                                              as possible, even if some of them cannot understand
     approval from relevant adults may be more difficult
                                                              the full explanation. Their age should not diminish
     than in the case of the children, but, being
                                                              their rights, although their level of understanding
     sensitive to children’s welfare, it is vital that
                                                              must be taken into account in the explanations that
     researchers secure such approval. It may be useful
                                                              are shared with them.
     if, in seeking the consent of children, researchers
                                                                                        (Fine and Sandstrom 1988)
     bear in mind the provisory comments below.
         While seeking children’s permission and               The second group consists of those children who
     cooperation is an automatic part of quantitative       are to be used in a research project and who may
     research (a child cannot unknowingly complete a        not meet Diener and Crandall’s (1978) criterion of
     simple questionnaire), the importance of informed      ‘competence’ (a group of psychologically impaired
     consent in qualitative research is not always          children, for example – the issue of ‘advocacy’
     recognized. Speaking of participant observation,       applies here). In such circumstances there may
     for example, Fine and Sandstrom (1988) say that        be institutional or local authority guidelines to
     researchers must provide a credible and meaningful     follow. In the absence of these, the requirements
     explanation of their research intentions, especially   of informed consent would be met by obtaining the
     in situations where they have little authority, and    permission of headteachers acting in loco parentis or
     that children must be given a real and legitimate      who have had delegated to them the responsibility
     opportunity to say that they do not want to take       for providing informed consent by the parents.
     part. The authors advise that where subjects do           Two cautions: first, where an extreme form of
     refuse, they should not be questioned, their actions   research is planned, parents would have to be fully
     should not be recorded, and they should not be         informed in advance and their consent obtained;
     included in any book or article (even under a          and second, whatever the nature of the research
     pseudonym). Where they form part of a group,           and whoever is involved, should a child show
     they may be included as part of a collectivity.        signs of discomfort or stress, the research should
     Fine and Sandstrom (1988) consider that such           be terminated immediately. For further discussion
     rejections are sometimes a result of mistrust of the   on the care that needs to be exercised in working
     researcher. They suggest that at a later date, when    with children we refer readers to Graue and Walsh
     the researcher has been able to establish greater      (1998); Greig and Taylor (1998); Holmes (1998).
                                                                               ACCESS AND ACCEPTANCE            55

  Informed consent requires an explanation and        and acceptance in particular at this point because

                                                                                                                     Chapter 2
description of several factors, including, for        it offers the best opportunity for researchers to
example:                                              present their credentials as serious investigators
                                                      and establish their own ethical position with
   the purposes, contents and procedures of the
                                                      respect to their proposed research.
                                                         Investigators cannot expect access to a nursery,
   any foreseeable risks and negative outcomes,
                                                      school, college or university as a matter of
   discomfort or consequences and how they will
                                                      right. They have to demonstrate that they are
   be handled
                                                      worthy, as researchers and human beings, of being
   benefits that might derive from the research
                                                      accorded the facilities needed to carry out their
   incentives to participate and rewards from
                                                      investigations. The advice of Bell (1991: 37) is
                                                      to gain permission early on, with fully informed
   right to voluntary non-participation, with-
                                                      consent gained, and indicating to participants the
   drawal and rejoining the project
                                                      possible benefits of the research.
   rights and obligations to confidentiality and
                                                         The first stage thus involves the gaining of
   non-disclosure of the research, participants and
                                                      official permission to undertake one’s research in
                                                      the target community. This will mean contacting,
   disclosure of any alternative procedures that
                                                      in person or in writing, an appropriate official
   may be advantageous
                                                      and/or the chairperson of the governors if one is
   opportunities for participants to ask questions
                                                      to work in a school, along with the headteacher or
   about any aspect of the research
                                                      principal. At a later point, significant figures who
   signed contracts for participation.
                                                      will be responsible for, or assist in, the organization
There are many more issues, and researchers will      and administration of the research will also need to
need to decide what to include in informed            be contacted – the deputy head or senior teacher,
consent. Not least among these is the issue of        for instance, and most certainly the class teacher
volunteering. Participants may feel coerced to        if children are to be used in the research. Since
volunteer (e.g. by a school principal), or may        the researcher’s potential for intrusion and perhaps
not wish to offend a researcher by refusing to        disruption is considerable, amicable relations with
participate, or may succumb to peer pressure          the class teacher in particular should be fostered
to volunteer (or not to volunteer), or may            as expeditiously as possible. If the investigation
wish to volunteer for reasons other than the          involves teachers as participants, propositions
researcher’s (e.g. to malign a school principal or    may have to be put to the stakeholders and
senior colleagues, to gain resources for his or her   conditions negotiated. Where the research is to
department, or to gain approval from colleagues).     take place in another kind of institution, e.g. a
Researchers have to ensure that volunteers have       youth club or detention centre, the approach will
real freedom of choice if informed consent is to be   be similar, although the organizational structure
fulfilled.                                             will be different.
                                                         Achieving goodwill and cooperation is espe-
                                                      cially important where the proposed research
Access and acceptance                                 extends over a period of time: days, perhaps,
The relevance of the principle of informed            in the case of an ethnographic study; months
consent becomes apparent at the initial stage         (or perhaps years) where longitudinal research
of the research project – that of access to the       is involved. Access does not present quite such
institution or organization where the research        a problem when, for example, a one-off survey
is to be conducted, and acceptance by those           requires respondents to give up half-an-hour of
whose permission one needs before embarking           their time or when a researcher is normally a
on the task. We highlight this stage of access        member of the organization where the research

     is taking place (an insider), though in the case              be observed, which subjects are to be interviewed,
     of the latter, it is generally unwise to take co-             observational needs, the time involved, the degree
     operation for granted. Where research procedures              of disruption envisaged; arrangements to guarantee
     are extensive and complicated, however, or where              confidentiality with respect to data (if this is
     the design is developmental or longitudinal, or               necessary), the role of feedback and how findings
     where researchers are not normally based in the               can best be disseminated, the overall timetable
     target community, the problems of access are more             within which the research is to be encompassed,
     involved and require greater preparation. Box 2.3             and finally, whether assistance will be required
     gives a flavour of the kinds of accessibility problems         in the organization and administration of the
     that can be experienced (Foster 1989).                        research.
        Having identified the official and significant                   By such planning and foresight, both researchers
     figures whose permission must be sought, and                   and institutions will have a good idea of the
     before actually meeting them, researchers will                demands likely to be made on both subjects (be
     need to clarify in their own minds the precise                they children or teachers) and organizations. It is
     nature and scope of their research. It is desirable           also a good opportunity to anticipate and resolve
     that they have a total picture of what it all entails,        likely problems, especially those of a practical kind.
     even if the overall scheme is a provisional one               A long, complicated questionnaire, for example,
     (though we have to bear in mind that this may                 may place undue demands on the comprehension
     cause difficulties later). In this respect researchers         skills and attention spans of a particular class
     could, for instance, identify the aims of the                 of 13 year olds, or a relatively inexperienced
     research; its practical applications, if any, the             teacher could feel threatened by sustained research
     design, methods and procedures to be used, the                scrutiny. Once this kind of information has been
     nature and size of samples or groups, what tests are          sorted out and clarified, researchers will be in a
     to be administered and how, what activities are to            stronger position to discuss their proposed plans
                                                                   in an informed, open and frank manner (though
                                                                   not necessarily too open, as we shall see) and may
                                                                   thereby more readily gain permission, acceptance
     Box 2.3                                                       and support. It must be remembered that hosts
     Close encounters of a researcher kind                         will have perceptions of researchers and their
                                                                   intentions and that these need to be positive.
       My first entry into a staffroom at the college was the       Researchers can best influence such perceptions by
       occasion of some shuffling and shifting of books and
       chairs so that I could be given a comfortable seat while
                                                                   presenting themselves as competent, trustworthy
       the tutor talked to me from a standing position. As time    and accommodating.
       progressed my presence was almost taken for granted            Once this preliminary information has been
       and later, when events threatened the security of the       collected, researchers are duly prepared for
       tutors, I was ignored. No one enquired as to whether        the next stage: making actual contact in
       they could assist me and my own enquiries were met
       with cursory answers and confused looks, followed
                                                                   person, perhaps after an introductory letter, with
       by the immediate disappearance of the individuals           appropriate people in the organization with a view
       concerned, bearing a pile of papers. I learned not          to negotiating access. If the research is university-
       to make too many enquiries. Unfortunately, when             based, they will have the support of their university
       individuals feel insecure, when their world is threatened   and supervisor. Festinger and Katz (1966) consider
       with change that is beyond their control, they are likely
       to respond in an unpredictable manner to persons
                                                                   that there is real economy in going to the very
       within their midst whose role is unclear, and the role      top of the organization or system in question to
       of the researcher is rarely understood by those not         obtain assent and cooperation. This is particularly
       engaged in research.                                        so where the structure is clearly hierarchical and
                                                                   where lower levels are always dependent on their
     Source: Foster 1989: 194                                      superiors. They consider it likely that the nature
                                                                                     ACCESS AND ACCEPTANCE          57

of the research will be referred to the top of the       Box 2.4

                                                                                                                         Chapter 2
organization sooner or later, and that there is a        Conditions and guarantees proffered for a school-
much better chance for a favourable decision if          based research project
leaders are consulted at the outset. It may also be
the case that heads will be more open-minded than          1 All participants must be given the chance to remain
those lower down, who, because of their insecurity,          anonymous.
                                                           2 All data must be given strict confidentiality.
may be less cooperative.                                   3 Interviewees should have the chance to verify
   Festinger and Katz (1996) also warn against               statements at the stage of drafting the report
using the easiest entrances into the organization            (respondent validation).
when seeking permission. Researchers may                   4 Participants should be given a copy of the final
perhaps seek to come in as allies of individuals             report.
                                                           5 Permission for publication must be gained from the
or groups who have a special interest to exploit             participants.
and who see research as a means to their ends,             6 If possible, the research report should be of benefit
rather than entering the situation in the common             to the school and participants.
interests of all parties, with findings equally
available to all groups and persons (Festinger and       Source: adapted from Bell 1991
Katz 1966). Investigators should thus seek as broad
a basis for their support as possible. Other potential
problems may be circumvented by making use
of accepted channels of communication in the             level with one or two examples of items that
institution or organization. Festinger and Katz          are not crucial to the study as a whole. As most
(1966) caution that if information is limited to         research entails some risks, especially where field
a single channel then the study risks becoming           studies are concerned, and as the presence of an
identified with the interests that are associated         observer scrutinizing various aspects of community
with that channel.                                       or school life may not be relished by all in the
   Following contact, there will be a negotiation        group, investigators must at all times manifest a
process. At this point researchers will give as          sensitive appreciation of their hosts’ and subjects’
much information about the aims, nature and              position and reassure anyone who feels threatened
procedures of the research as is appropriate. This is    by the work. Such reassurance could take the form
very important: information that may prejudice           of a statement of conditions and guarantees given
the results of the investigation may have to             by researchers at this negotiation stage. By way of
be withheld. Aronson and Carlsmith (1969), for           illustration, Box 2.4 contains conditions laid down
instance, note that one cannot imagine researchers       for the Open University students’ school-based
who are studying the effects of group pressure           research project.
on conformity announcing their intentions in                Ethical considerations pervade the whole pro-
advance. On the other hand, researchers may find          cess of research; these will be no more so than at
themselves on dangerous ground if they go to the         the stage of access and acceptance, where appro-
extreme of maintaining a ‘conspiracy of silence’,        priateness of topic, design, methods, guarantees of
because, as Festinger and Katz note, such a stance is    confidentiality, analysis and dissemination of find-
hard to keep up if the research is extensive and lasts   ings must be negotiated with relative openness,
over several days or weeks, and trying to preserve       sensitivity, honesty, accuracy and scientific im-
secrecy might lead to an increase in the spread and      partiality. There can be no rigid rules in this
wildness of rumours (Festinger and Katz 1966).           context. It will be a case of formulating and
If researchers do not want their potential hosts         abiding by one’s own situated ethics. These will
and/or subjects to know too much about specific           determine what is acceptable and what is not ac-
hypotheses and objectives, then a simple way out is      ceptable. As Hitchcock and Hughes (1995) say in
to present an explicit statement at a fairly general     this regard:

       Individual circumstances must be the final arbiter. As         confidences betrayed, and the social scientist
       far as possible it is better if the teacher can discuss the   may only infrequently be confronted with an
       research with all parties involved. On other occasions        unresolvable ethical dilemma. Where research is
       it may be better for the teacher to develop a pilot study     ethically sensitive, however, many factors may
       and uncover some of the problems in advance of the            need to be taken into account and these may
       research proper. If it appears that the research is going     vary from situation to situation, for example:
       to come into conflict with aspects of school policy,           the age of those being researched; whether the
       management styles, or individual personalities, it is         subject matter of the research is a sensitive area;
       better to confront the issues head on, consult relevant       whether the aims of the research are in any
       parties, and make rearrangements in the research              way subversive (vis-` -vis subjects, teachers, or
       design where possible or necessary.                           institution); the extent to which the researcher
                              (Hitchcock and Hughes l995: 41)        and researched can participate and collaborate in
                                                                     planning the research; how the data are to be
     Where a pilot study is not feasible it may be
                                                                     processed, interpreted, and used. Laing (1967: 53)
     possible to arrange one or two scouting forays
                                                                     offers an interesting, cautionary view of data where
     to assess possible problems and risks. By way of
                                                                     he writes that they are ‘not so much given as taken
     summary, we refer the reader to Box 2.5.
                                                                     out of a constantly elusive matrix of happenings.
                                                                     We should speak of capta rather than data’.
     The field of ethics
     Whatever the specific nature of their work, social               Sources of tension
     researchers must take into account the effects of
                                                                     Non-maleficence, beneficence and human
     the research on participants, and act in such a
     way as to preserve their dignity as human beings:
     responsibility to participants. Such is ethical                 The first tension, as expressed by Aronson and
     behaviour. Indeed, ethics has been defined as ‘a                 Carlsmith (1969), is that which exists between two
     matter of principled sensitivity to the rights of               sets of related values held by society: a belief in the
     others, and that ‘while truth is good, respect for              value of free scientific inquiry in pursuit of truth
     human dignity is better’ (Cavan 1977: 810).                     and knowledge, and a belief in the dignity of indi-
        Kimmel (1988) has pointed out that it is                     viduals and their right to those considerations that
     important we recognize that the distinction                     follow from it. It is this polarity that we referred
     between ethical and unethical behaviour is not                  to earlier as the costs/benefits ratio and by which
     dichotomous, even though the normative code                     ‘greater consideration must be given to the risks to
     of prescribed (‘ought’) and proscribed (‘ought                  physical, psychological, humane, proprietary and
     not’) behaviours, as represented by the ethical                 cultural values than to the potential contribu-
     standards of a profession, seem to imply that it                tion of research to knowledge’ (Social Sciences
     is. Judgements about whether behaviour conflicts                 and Humanities Research Council of Canada
     with professional values lie on a continuum that                1981), i.e. the issue of ‘non-maleficence’ (where
     ranges from the clearly ethical to the clearly                  no harm is wished upon subjects or occurs) (see
     unethical. The point here is that ethical principles  
     are not absolute, generally speaking, though some               9780415368780 – Chapter 2, file 2.3. ppt).
     maintain that they are as we shall see shortly, but                Non-maleficence (do not harm) is enshrined
     must be interpreted in the light of the research                in the Hippocratic oath, in which the principle
     context and of other values at stake.                           of primum non nocere (first of all, do no harm) is
        Of course, a considerable amount of research                 held as a guiding precept. So also with research.
     does not cause pain or indignity to the participants,           At first sight this seems uncontentious; of course
     self-esteem is not necessarily undermined nor                   we do not wish to bring harm to our research
                                                                                                 SOURCES OF TENSION          59

Box 2.5

                                                                                                                                  Chapter 2
Negotiating access checklist

   1 Clear official channels by formally requesting permission to carry out your investigation as soon as you
     have an agreed project outline.
     Some LEAs insist that requests to carry out research are channelled through the LEA office. Check what is required in
     your area.
   2 Speak to the people who will be asked to cooperate.
     Getting the LEA or head’s permission is one thing, but you need to have the support of the people who will be asked
     to give interviews or complete questionnaires.
   3 Submit the project outline to the head, if you are carrying out a study in your or another educational
     List people you would like to interview or to whom you wish to send questionnaires and state conditions under which
     the study will be conducted.
   4 Decide what you mean by anonymity and confidentiality.
     Remember that if you are writing about ‘the head of English’ and there is only one head of English in the school, the
     person concerned is immediately recognizable.
   5 Decide whether participants will receive a copy of the report and/or see drafts or interview transcripts.
     There are cost and time implications. Think carefully before you make promises.
   6 Inform participants what is to be done with the information they provide.
     Your eyes and those of the examiner only? Shown to the head, the LEA etc.?
   7 Prepare an outline of intentions and conditions under which the study will be carried out to hand to the
     Even if you explain the purpose of the study the conditions and the guarantees, participants may forget.
   8 Be honest about the purpose of the study and about the conditions of the research.
     If you say an interview will last ten minutes, you will break faith if it lasts an hour. If you are conducting the
     investigation as part of a degree or diploma course, say so.
   9 Remember that people who agree to help are doing you a favour.
     Make sure you return papers and books in good order and on time. Letters of thanks should be sent, no matter how
     busy you are.
  10 Never assume ‘it will be all right’. Negotiating access is an important stage in your investigation.
     If you are an inside researcher, you will have to live with your mistakes, so take care.

Source: adapted from Bell 1991

subjects. However, what constitutes ‘harm’ is                   with very serious weaknesses, such that their
unclear: one person’s harm is a society’s benefit,               contracts should be terminated in the interests
and whether a little harm for a few is tolerable                of the students?
in the interests of a major benefit for all, or even                When researchers are confronted with dilemmas
for the person concerned, throws into relief the                such as these (though they are likely to occur
tension involved here. The question is whether                  much less in education than in social psychology
the end justifies the means. As a general principle              or medicine), it is generally considered that they
we would advocate the application of primum                     resolve them in a manner that avoids the extremes
non nocere and, indeed, ethics regulatory boards,               of, on the one hand, giving up the idea of research
for example in universities perusing research                   and, on the other, ignoring the rights of the
proposals (discussed later), are guided heavily by              subjects. At all times, the welfare of subjects should
this principle. However, there could be tensions                be kept in mind, even if it involves compromising
here. What do you do if you discover that the                   the impact of the research. Researchers should
headteacher has a serious alcohol problem or is                 never lose sight of the obligations they owe to those
having an affair with a parent? What do you                     who are helping, and should constantly be alert
do if your research shows teachers in the school                to alternative techniques should the ones they

     are employing at the time prove controversial.          that a selfish approach to the benefits of the
     Indeed, this polarity between the research and          research by the researcher is unethical.
     the researched is reflected in the principles of            This latter point requires researchers to do more
     the American Psychological Association which,           than pay lip service to the notion of treating
     as Zechmeister and Shaughnessy (1992) show,             research participants as subjects rather than
     attempts to strike a balance between the rights of      as objects to be used instrumentally – research
     investigators to seek an understanding of human         fodder, so to speak – imbuing them with self-
     behaviour, and the rights and welfare of individuals    esteem and respect. One can treat people with
     who participate in the research. In the final            respect but still the research may make no
     reckoning, the decision to go ahead with a research     material difference to their lives. While it is
     project rests on a subjective evaluation of the costs   surely impossible to argue against treating people
     both to the individual and society.                     with dignity and respect, it raises the issue of the
        The corollary of non-maleficence is benefi-            obligations and commitments of the researcher.
     cence: what benefits will the research bring, and        Let us say that the researcher has been working
     to whom? Many would-be participants could be            closely in a school for one or two years; surely
     persuaded to take part in research if it is made        that researcher has an obligation to improve the
     clear that it will, or may, bring personal, educa-      lives of those being researched, rather than simply
     tional and social benefits. For example, it may lead     gathering data instrumentally? To do the latter
     to the improvement of learning, increased fund-         would be inhumane and deeply disrespectful. The
     ing and resources for a particular curriculum area,     issue is tension ridden: is the research for people
     improved approaches to the teaching of a subject,       and issues or about people and issues? We have
     increased self-esteem for students, or additional       to be clear about our answer to the question
     teachers in a school. While it is sometimes worth       ‘what will this research do for the participants
     including a statement of potential benefit when          and the wider community, not just for the
     contacting schools and individuals, it may also be      researcher?’
     an actual requirement for ethics regulatory boards         Bailey (1994: 457) suggests that there are several
     or sponsors.                                            approaches that can be used to avoid harming
        The recipients of the benefit also have to be         research subjects, including:
     factored into the discussion here. A researcher
     may gain promotion, publications, a degree,                using computer simulations
     research sponsorship and celebrity from a piece            finding a situation in which the negative effects
     of research. However, the research might still             of harm already exist, i.e. where the research
     leave the participants untouched, underprivileged,         does not have the responsibility for having
     living and working in squalid and under-                   produced these conditions
     resourced conditions, under-supported, and with            applying only a very low level of potential
     no material, educational or other improvements             harm, or for only a short period of time, so that
     brought to the quality of their lives and work.            any effects are minimal
     On the one hand, it could be argued that                   obtaining informed consent (providing details
     research that did not lead to such benefits is              of the potential negative effects and securing
     unethical; on the other hand, it could be that the         participants’ consent)
     research helps to place the issue on the agenda            justifying the research on the grounds that
     of decision-makers and that, in the long run, it           the small amount of harm caused is much less
     could contribute to a groundswell of opinion that,         than the harm caused by the existing situation
     itself, brings change. While it may be fanciful            (which the research is trying to improve)
     to believe that a single piece of research will            using samples rather than complete popula-
     automatically lead to improvement, the ethical             tions, so that fewer people are exposed to the
     question raised here – who benefits? – suggests             harm
                                                                                              VOICES OF EXPERIENCE            61

    maintaining the privacy of participants                        By this absolute principle, the Stanford Prison

                                                                                                                                   Chapter 2
    through the use of aggregated or anonymised                 Experiment must be regarded as unethical because
    data.                                                       the participants suffered considerably.
                                                                   In absolutist principles – ‘duty ethics of prin-
While some of these are uncontentious, others in                ciples’ (Edwards and Mauthner 2002: 20), a
this list are debatable, and researchers will need to           deontological model – research is governed by uni-
be able to justify the decision that they reach.                versal precepts such as justice, honesty and respect
                                                                (among others). In the ‘utilitarian ethics of con-
Absolutist and relativist ethics                                sequences’ (p. 20) ethical research is judged in
                                                                terms of its consequences, e.g. increased knowl-
The second source of tension in this context is                 edge, benefit for many.
that generated by the competing absolutist and                     Those who hold a relativist position would argue
relativist positions. The absolutist view holds that            that there can be no absolute guidelines and that
clear, set principles should guide the researchers              ethical considerations will arise from the very
in their work and that these should determine                   nature of the particular research being pursued
what ought and what ought not to be done                        at the time: situation determines behaviour.
(see Box 2.6). To have taken a wholly absolutist                This underlines the significance of ‘situated
stance, for example, in the case of the Stanford                ethics’ (Simons and Usher 2000), where overall
Prison Experiment (see Chapter 21), where the                   guidelines may offer little help when confronted
researchers studied interpersonal dynamics in a                 with a very specific situation.
simulated prison, would have meant that the                        There are some contexts, however, where
experiment should not have taken place at all                   neither the absolutist nor the relativist position
or that it should have been terminated well before              is clear cut. Writing of the application of the
the sixth day. Zimbardo (1984) has stated that the              principle of informed consent with respect to life
absolutist ethical position, in which it is unjustified          history studies, Plummer (1983) says:
to induce any human suffering, would bring about
                                                                  Both sides have a weakness. If, for instance, as the
the end of much psychological or medical research,
                                                                  absolutists usually insist, there should be informed
regardless of its possible benefits to society.
                                                                  consent, it may leave relatively privileged groups
                                                                  under-researched (since they will say ‘no’) and
                                                                  underprivileged groups over-researched (they have
Box 2.6
                                                                  nothing to lose and say ‘yes’ in hope). If the individual
Absolute ethical principles in social research
                                                                  conscience is the guide, as the relativists insist,
                                                                  the door is wide open for the unscrupulous–even
  Ethics embody individual and communal codes of
  conduct based upon a set of explicit or implicit principles
  and which may be abstract and impersonal or concrete                                                  (Plummer 1983)
  and personal. Ethics can be ‘absolute’ and ‘relative’.
  When behaviour is guided by absolute ethical standards,
                                                                He suggests that broad guidelines laid down by
  a higher-order moral principle is invoked which does          professional bodies which offer the researcher
  not vary with regard to the situation in hand. Such           room for personal ethical choice are a way out
  absolutist ethics permit no degree of freedom for             of the problem. We consider these later in this
  ends to justify means or for any beneficial or positive        chapter.
  outcomes to justify occasions where the principle is
  suspended, altered or diluted, i.e. there are no special
  or extenuating circumstances which can be considered          Voices of experience
  as justifying a departure from, or modification to, the
  ethical standard.                                             Whatever the ethical stance one assumes and
                                                                no matter what forethought one brings to
Source: adapted from Zimbardo 1984                              bear on one’s work, there will always be

     unknown, unforeseen problems and difficulties             such be the case. This means that whatever form
     lying in wait (Kimmel 1988). Baumrind (1964),            the written account takes, be it a report, article,
     for example, warns of the possible failure on            chapter or thesis, and no matter the readership for
     the researchers’ part to perceive a positive             which it is intended, its authors must acknowledge
     indebtedness to their subjects for their services,       and thank all who helped in the research, even
     perhaps, she suggests, because the detachment            to the extent of identifying by name those whose
     which investigators bring to their task prevents         contribution was significant. This can be done in
     appreciation of subjects as individuals. This kind       a foreword, introduction or footnote. All this is
     of omission can be averted if the experimenters          really a question of commonsensical ethics.
     are prepared to spend a few minutes with subjects           Ethical problems in educational research can
     afterwards in order to thank them for their              often result from thoughtlessness, oversight or
     participation, answer their questions, reassure          taking matters for granted. Again, researchers
     them that they did well, and generally talk to           engaged in sponsored research may feel they
     them for a time. If the research involves subjects       do not have to deal with ethical issues,
     in a failure experience, isolation or loss of self-      believing their sponsors to have them in hand.
     esteem, for example, researchers must ensure that        Likewise, each researcher in a collaborative
     the subjects do not leave the situation more             venture may take it for granted, wrongly, that
     humiliated, insecure and alienated than when             colleagues have the relevant ethical questions in
     they arrived. From the subject’s point of view,          mind, consequently appropriate precautions go by
     procedures which involve loss of dignity, injury         default. A student whose research is part of a course
     to self-esteem, or affect trust in rational authority    requirement and who is motivated wholly by self-
     are probably most harmful in the long run and            interest, or academic researchers with professional
     may require the most carefully organized ways            advancement in mind, may overlook the ‘oughts’
     of recompensing the subject in some way if the           and ‘ought nots’.
     researcher chooses to carry on with those methods.          A related issue here is that it is unethical for
        With particularly sensitive areas, participants       the researcher to be incompetent in the area
     need to be fully informed of the dangers of serious      of research. Competence may require training
     after-effects. There is reason to believe that at        (Ticehurst and Veal 2000: 55). Indeed an ethical
     least some of the obedient subjects in Milgram’s         piece of research must demonstrate rigour in the
     (1963) experiments (see Chapter 21) came away            design, conduct, analysis and reporting of the
     from the experience with a lower self-esteem,            research (Morrison 1996b).
     having to live with the realization that they were          An ethical dilemma that is frequently discussed
     willing to yield to destructive authority to the         is in the experiment. Gorard (2001: 146) sum-
     point of inflicting extreme pain on a fellow human        marizes the issue as being that the design is
     being (Kelman 1967). It follows that researchers         discriminatory, in that the control group is being
     need to reflect attitudes of compassion, respect,         denied access to a potentially better treatment
     gratitude and common sense without being too             (e.g. curriculum, teaching style). Of course, the
     effusive. Subjects clearly have a right to expect that   response to this is that, in a genuine experiment,
     the researchers with whom they are interacting           we do not know which treatment is better, and
     have some concern for the welfare of participants.       that, indeed, this is the point of the experiment.
        Further, the subject’s sensibilities need also to
     be taken into account when the researcher comes          Ethical dilemmas
     to write up the research. It is unacceptable for
     researchers to show scant regard for subjects’           Robson (1993: 33) raises ten questionable prac-
     feelings at the report stage. A related and not          tices in social research:
     insignificant issue concerns the formal recognition          involving people without their knowledge or
     of those who have assisted in the investigation, if         consent
                                                                                           ETHICAL DILEMMAS          63

   coercing them to participate                            extends to all information relating to a person’s phys-

                                                                                                                          Chapter 2
   withholding information about the true nature           ical and mental condition, personal circumstances
   of the research                                         and social relationships which is not already in the
   deceiving participants in other ways                    public domain. It gives to the individual or collectiv-
   inducing them to commit acts diminishing                ity the freedom to decide for themselves when and
   their self-esteem                                       where, in what circumstances and to what extent
   violating rights of self-determination (e.g. in         their personal attitudes, opinions, habits, eccentrici-
   studies seeking to promote individual change)           ties, doubts and fears are to be communicated to or
   exposing participants to physical or mental
                                                           withheld from others.
                                                                                (Social Sciences and Humanities
   invading their privacy
                                                                              Research Council of Canada 1981)
   withholding benefits from some participants
   (e.g. in comparison groups)                           In the context of research, therefore, ‘right to
   not treating participants fairly, or with             privacy’ may easily be violated during the course
   consideration, or with respect.                       of an investigation or denied after it has been
                                                         completed. At either point the participant is
Interestingly, Robson (1993) calls these ‘question-
able practices’ rather than areas to be proscribed,
                                                            Privacy has been considered from three different
indicating that they are not black and white, right
                                                         perspectives by Diener and Crandall (1978). These
or wrong matters. They constitute the problem of
                                                         are the sensitivity of the information being given,
ethical dilemmas.
                                                         the setting being observed, and dissemination
   At the beginning of this chapter, we spoke of
                                                         of information. Sensitivity of information refers
the costs/benefits ratio. Frankfort-Nachmias and
                                                         to how personal or potentially threatening the
Nachmias (1992) express this as a conflict between
                                                         information is that is being collected by the
two rights: the rights to conduct research in
                                                         researcher. Certain kinds of information are
order to gain knowledge versus the rights of
                                                         more personal than others and may be more
participants to self-determination, privacy and
                                                         threatening. According to a report by the
dignity. This constitutes the fundamental ethical
                                                         American Psychological Association (1973) for
dilemma of the social scientist for whom there
                                                         example, ‘Religious preferences, sexual practices,
are no absolute right or wrong answers. Which
                                                         income, racial prejudices, and other personal
proposition is favoured, or how a balance between
                                                         attributes such as intelligence, honesty, and
the two is struck will depend very much on the
                                                         courage are more sensitive items than ‘‘name,
background, experience, and personal values of
                                                         rank and serial number’’ ’. Thus, the greater the
the individual researcher. We examine here other
                                                         sensitivity of the information, the more safeguards
dilemmas that may confront investigators once
                                                         are called for to protect the privacy of the
they have come to some accommodation with this
fundamental dilemma and decided to proceed with
                                                            The setting being observed may vary from
their research.                                          very private to completely public. The home,
                                                         for example, is considered one of the most
                                                         private settings and intrusions into people’s homes
                                                         without their consent are forbidden by law.
For the most part, individual ‘right to privacy’         Dissemination of information concerns the ability
is usually contrasted with public ‘right to              to match personal information with the identity
know’ (Pring 1984) and this has been defined              of the research participants. Indeed, personal data
in the Ethical Guidelines for the Institutional Review   are defined at law as those data which uniquely
Committee for Research with Human Subjects as that       identify the individual providing them. When such
which                                                    information is publicized with names through the

     media, for example, privacy is seriously violated.     supplier. A participant or subject is therefore con-
     The more people there are who can learn about          sidered anonymous when the researcher or another
     the information, the more concern there must be        person cannot identify the participant or sub-
     about privacy (see Diener and Crandall 1978).          ject from the information provided. Where this
        As is the case with most rights, privacy can        situation holds, a participant’s privacy is guar-
     be voluntarily relinquished. Research participants     anteed, no matter how personal or sensitive the
     may choose to give up their right to privacy either    information is. Thus a respondent completing a
     by allowing a researcher access to sensitive topics    questionnaire that bears absolutely no identify-
     or settings or by agreeing that the research report    ing marks – names, addresses, occupational details
     may identify them by name. The latter case at least    or coding symbols – is ensured complete and total
     would be an occasion where informed consent            anonymity. A subject agreeing to a face-to-face in-
     would need to be sought.                               terview, on the other hand, can in no way expect
        Generally speaking, if researchers intend to        anonymity. At most, the interviewer can promise
     probe into the private aspects or affairs of           confidentiality. Non-traceability is an important
     individuals, their intentions should be made clear     matter, and this extends to aggregating data in
     and explicit and informed consent should be            some cases, so that an individual’s response is
     sought from those who are to be observed or            unknowable.
     scrutinized in private contexts. Other methods            The principal means of ensuring anonymity,
     to protect participants are anonymity and              then, is not using the names of the participants or
     confidentiality and our examination of these            any other personal means of identification. Further
     follows.                                               ways of achieving anonymity have been listed
        Privacy is more than simple confidentiality          by Frankfort-Nachmias and Nachmias (1992), for
     (discussed below). The right to privacy means          example, the use of aliases, the use of codes for
     that a person has the right not to take part in        identifying people (to keep the information on
     the research, not to answer questions, not to be       individuals separate from access to them) and the
     interviewed, not to have their home intruded into,     use of password-protected files.
     not to answer telephone calls or emails, and to           These may work satisfactorily in most situations,
     engage in private behaviour in their own private       but as Raffe and his colleagues (1989) have
     place without fear of being observed. It is freedom    shown, there is sometimes the difficulty of
     from as well as freedom for. This is frequently an     maintaining an assurance of anonymity when,
     issue with intrusive journalism. Hence researchers     for example, combining data may uniquely
     may have an obligation to inform participants of       identify an individual or institution or when
     their rights to refuse to take part in any or all of   there is access to incoming returns by support
     the research, to obtain permission to conduct the      staff. Plummer (1983), likewise, refers to life
     research, to limit the time needed for participation   studies in which names have been changed,
     and to limit the observation to public behaviour.      places shifted, and fictional events added to
                                                            prevent acquaintances of subjects discovering their
                                                            identity. Although one can go a long way down
                                                            this path, there is no absolute guarantee of total
     Frankfort-Nachmias and Nachmias (1992) under-          anonymity as far as life studies are concerned.
     line the need for confidentiality of participants’      In experimental research the experimenter is
     identities, and that any violations of this should     interested in ‘human’ behaviour rather than in
     be made with the agreement of the participants.        the behaviour of specific individuals (Aronson and
     The essence of anonymity is that information pro-      Carlsmith 1969). Consequently the researcher has
     vided by participants should in no way reveal          absolutely no interest in linking the person as
     their identity. The obverse of this is, as we saw      a unique, named individual to actual behaviour,
     earlier, personal data that uniquely identify their    and the research data can be transferred to coded,
                                                                                           ETHICAL DILEMMAS         65

unnamed data sheets. As they comment, ‘the very              deletion of identifiers (for example, deleting

                                                                                                                         Chapter 2
impersonality of the process is a great advantage            the names, addresses or other means of identi-
ethically because it eliminates some of the negative         fication from the data released on individuals)
consequences of the invasion of privacy’ (Aronson            crude report categories (for example, releasing
and Carlsmith 1969).                                         the year of birth rather than the specific date,
                                                             profession but not the speciality within that
                                                             profession, general information rather than
The second way of protecting a participant’s right           micro-aggregation (that is, the construction of
to privacy is through the promise of confidentiality.         ‘average persons’ from data on individuals and
This means that although researchers know who                the release of these data, rather than data on
has provided the information or are able to identify         individuals)
participants from the information given, they will           error inoculation (deliberately introducing
in no way make the connection known publicly;                errors into individual records while leaving
the boundaries surrounding the shared secret will            the aggregate data unchanged).
be protected. The essence of the matter is the
                                                          Cooper and Schindler (2001: 117) suggest that
extent to which investigators keep faith with
                                                          confidentiality can be protected by obtaining
those who have helped them. It is generally at
                                                          signed statements indicating non-disclosure of
the access stage or at the point where researchers
                                                          the research, restricting access to data which
collect their data that they make their position
                                                          identify respondents, seeking the approval of
clear to the hosts and/or subjects. They will thus
                                                          the respondents before any disclosure about
be quite explicit in explaining to subjects what
                                                          respondents takes place, non-disclosure of data
the meaning and limits of confidentiality are in
                                                          (e.g. subsets that may be able to be combined to
relation to the particular research project. On the
                                                          identify an individual).
whole, the more sensitive, intimate or discrediting
the information, the greater is the obligation on
the researcher’s part to make sure that guarantees
of confidentiality are carried out in spirit and letter.
Promises must be kept.                                    The term ‘betrayal’ is usually applied to those
   Kimmel (1988) notes that some potential                occasions where data disclosed in confidence
respondents in research on sensitive topics               are revealed publicly in such a way as to cause
will refuse to cooperate when an assurance of             embarrassment, anxiety or perhaps suffering to the
confidentiality is weak, vague, not understood,            subject or participant disclosing the information.
or thought likely to be breached. He concludes            It is a breach of trust, in contrast to confidentiality,
that the usefulness of data in sensitive research         and is often a consequence of selfish motives
areas may be seriously affected by the researcher’s       of either a personal or professional nature. As
inability to provide a credible promise of                Plummer (1983) comments, ‘in sociology, there
confidentiality. Assurances do not appear to affect        is something slightly awry when a sociologist can
cooperation rates in innocuous studies perhaps            enter a group and a person’s life for a lengthy
because, as Kimmel suggests, there is expectation         period, learn their most closely guarded secrets,
on the part of most potential respondents that            and then expose all in a critical light to the
confidentiality will be protected.                         public’ (see
   A number of techniques have been developed             9780415368780 – Chapter 2, file 2.4. ppt).
to allow public access to data and information               One of the research methods that is perhaps
without confidentiality being betrayed. These              most vulnerable to betrayal is action research.
have been listed by Frankfort-Nachmias and                As Kelly (1989a) notes, this can produce several
Nachmias (1992) as follows:                               ethical problems. She says that if we treat teachers

     as collaborators in our day-to-day interactions,           Indeed she uses the word ‘betrayal’ in her concern
     it may seem like betrayal of trust if these                that she might be betraying the trust of the women
     interactions are recorded and used as evidence.            with whom she had worked for three years, not
     This is particularly the case where the evidence           least because they were in a far worse economic
     is negative. One way out, Kelly (1989a) suggests,          and personal state than she herself was (Finch
     could be to submit reports and evaluations of              1985: 118).
     teachers’ reactions to the teachers involved for
     comment, to get them to assess their own
     changing attitudes. She warns, however, that this
     might work well with teachers who have become              The use of deception in social psychological
     converts, but is more problematic where teachers           and sociological research has attracted a certain
     remain indifferent or hostile to the aims of the           amount of adverse publicity. Deception may lie
     research project. How does one write an honest but         in not telling people that they are being re-
     critical report of teachers’ attitudes, she asks, if one   searched (in some people’s eyes this is tantamount
     hopes to continue to work with those involved?             to spying), not telling the truth, telling lies, or com-
        Similarly Morrison (2006) considers the case of         promising the truth. It may also lie in using people
     a school that is under-performing, poorly managed          in a degrading or dehumanizing way (e.g. as a rat in
     or badly led. Does not the consumer, indeed the            an experiment). In social psychological research,
     state, have a right or a duty respectively to know         the term is applied to that kind of experimental
     or address this, such action typically involving the       situation where the researcher knowingly conceals
     exposure to the public of a school’s shortcomings,         the true purpose and conditions of the research, or
     and will this not damage individuals working in the        else positively misinforms the subjects, or exposes
     school, the principal and the teachers? What ‘fidu-         them to unduly painful, stressful or embarrassing
     ciary trust’ (Mitchell 1993) not to harm individ-          experiences, without the subjects having knowl-
     uals (the ethical issue of ‘non-maleficence’) does          edge of what is going on. The deception lies in
     the researcher have to the school or to the public,        not telling the whole truth. Bailey (1994: 463)
     and how can these two potentially contradictory            gives a clear example here, where respondents
     demands be reconciled? Should the researcher               may be asked to complete a postal questionnaire,
     expose the school’s weaknesses, which almost cer-          and believe that they are being asked for informa-
     tainly could damage individuals but which may be           tion about length and type of postage, whereas,
     in the public interest, or, in the interests of primum     in fact, the study is designed to compare different
     non nocere, remain silent? The issue hinges on trust:      kinds of questionnaire. He reports that 88 per cent
     the pursuit of truth and the pursuit of trust may run      of studies from a sample of 457 studies used de-
     counter to each other (Kelly 1985: 147); indeed            ception (see
     Kelly herself writes that ‘I do not think we have yet      9780415368780 – Chapter 2, file 2.5. ppt).
     found a satisfactory way of resolving this dilemma’.          Advocates of the method feel that if a deception
        Finch (1985) raises ethical issues in the conse-        experiment is the only way to discover something
     quences of reporting. In her research she worried          of real importance, the truth so discovered is
     that her reporting                                         worth the lies told in the process, so long as
                                                                no harm comes to the subject (see Aronson et al.
       could well mean that I was further reinforcing those
                                                                1990). Deception may be justified on the grounds
       assumptions deeply embedded in our culture and
                                                                that the research serves the public good, and that
       political life that working class women (especially      the deception prevents any bias from entering
       the urban poor) are inadequate mothers and too           the research, and also that it may protect the
       incompetent to be able to organize facilities that       confidentiality of a third party (for example, a
       most normal women could manage.                          sponsor). The problem from the researcher’s point
                                         (Finch 1985: 117)      of view is: ‘What is the proper balance between the
                                                                                             ETHICAL DILEMMAS        67

interests of science and the thoughtful, humane                  Kelman (1967) has suggested three ways of

                                                                                                                          Chapter 2
treatment of people who, innocently, provide the              dealing with the problem of deception. First, it is
data?’ In other words, the problem again hinges               important that we increase our active awareness
on the costs/benefits ratio.                                   that it exists as a problem. It is crucial that
   The pervasiveness of the issue of deception                we always ask ourselves the question whether
becomes even more apparent when we remember                   deception is necessary and justified. We must be
that it is even built into many of our measurement            wary of the tendency to dismiss the question as
devices, since it is important to keep the                    irrelevant and to accept deception as a matter of
respondent ignorant of the personality and                    course. Active awareness is thus in itself part of
attitude dimensions that we wish to investigate.              the solution, for it makes the use of deception
There are many problems that cannot be                        a focus for discussion, deliberation, investigation
investigated without deception and, although                  and choice.
there is some evidence that most subjects accept                 The second way of approaching the problem
without resentment the fact of having been                    concerns counteracting and minimizing the
duped once they understand the necessity for it               negative effects of deception. For example, subjects
(e.g. the Milgram (1974) obedience-to-authority               must be selected in a way that will exclude
experiment: see Chapter 21), it is important to               individuals who are especially vulnerable; any
keep in the forefront of one’s mind the question              potentially harmful manipulation must be kept
of whether the amount and type of deception is                to a moderate level of intensity; researchers must
justified by the significance of the study and the              be sensitive to danger signals in the reactions of
unavailability of alternative procedures.                     subjects and be prepared to deal with crises when
   The use of deception resulting in particularly             they arise; and at the conclusion of the research,
harmful consequences would be another occasion                they must take time not only to reassure subjects,
where ethical considerations would need to be                 but also to help them work through their feelings
given priority. An example here would be the                  about the experience to whatever degree may be
study by Campbell et al. (1964) which created                 required. The principle that subjects ought not to
extremely stressful conditions by using drugs to              leave the research situation with greater anxiety or
induce temporary interruption of breathing (see               lower levels of self-esteem than they came with is
Box 2.7).                                                     a good one to follow (the issue of non-maleficence
                                                              again). Desirably, subjects should be enriched by
                                                              the experience and should leave it with the feeling
Box 2.7                                                       that they have learned something.
An extreme case of deception                                     The primary way of counteracting negative
                                                              effects of research employing deception is to
  In an experiment designed to study the establishment of     ensure that adequate feedback is provided at
  a conditioned response in a situation that is traumatic
  but not painful, Campbell et al. (1964) induced – through
                                                              the end of the research or research session.
  the use of a drug – a temporary interruption of             Feedback must be kept inviolable and in no
  respiration in their subjects. The subjects’ reports        circumstances should subjects be given false
  confirmed that this was a ‘horrific’ experience for           feedback or be misled into thinking they are
  them. All the subjects thought they were dying. The         receiving feedback when the researcher is in fact
  subjects, male alcoholic patients who had volunteered
  for the experiment when they were told that it was
                                                              introducing another experimental manipulation.
  connected with a possible therapy for alcoholism, were      Debriefing may include the following (Cooper and
  not warned in advance about the effect of the drug,         Schindler 2001: 116):
  since this information would have reduced the traumatic
  impact of the experience.                                      explaining any deception and the reasons for it
                                                                 describing the purposes, hypotheses, objectives
Source: adapted from Kelman 1967                                 and methods of the research

        sharing the results after the research                     lesser nature occur. Thus, for example, the general
        ensuring follow-up psychological or medical                description given of research may leave out some
        attention after the research.                              key issues; indeed, to tell the subject what it
                                                                   is you are looking for may bias the outcome
     Even here, however, there are dangers. As                     quite substantially. Further, different accounts
     Aronson and Carlsmith (1969) say:                             of the research may have to be presented to
                                                                   different groups. He quotes an instance from
       debriefing a subject is not simply a matter of exposing
                                                                   his own research, a study of sexual minorities,
       him to the truth. There is nothing magically curative
                                                                   which required various levels of release – for the
       about the truth; indeed . . . if harshly presented, the
                                                                   subjects, for colleagues, for general enquiries, and
       truth can be more harmful than no explanation at all.
                                                                   for outside friends. None of these accounts actually
       There are vast differences in how this is accomplished,
                                                                   lied, they merely emphasized a different aspect of
       and it is precisely these differences that are of crucial
                                                                   the research.
       importance in determining whether or not a subject
                                                                      In the social sciences, the dilemma of deception
       is uncomfortable when he leaves the experimental
                                                                   has played an important part in experimental
                                                                   social psychology where subjects are not told the
                           (Aronson and Carlsmith 1969: 31)
                                                                   true nature of the experiment. Another area where
     They consider that the one essential aspect of the            it is used is that of sociology, where researchers
     debriefing process is that researchers communicate             conceal their identities and ‘con’ their way into
     their own sincerity as scientists seeking the truth           alien groups – the overt/covert debate (Mitchell
     and their own discomfort about the fact that they             1993). Covert, or secret participation, refers to
     found it necessary to resort to deception in order to         that kind of research where researchers spend an
     uncover the truth. As they say, ‘No amount of pos-            extended period of time in particular research
     texperimental gentleness is as effective in relieving         settings, concealing the fact that they are
     a subject’s discomfort as an honest accounting of             researchers and pretending to play some other
     the experimenter’s own discomfort in the situa-               role.
     tion’ (Aronson and Carlsmith 1969: 31–2).                        Bulmer (1982) notes that there are no simple
        The third way of dealing with the problem of               and universally agreed answers to the ethical issues
     deception is to ensure that new procedures and                that covert research produces. Erikson (1967), for
     novel techniques are developed. It is a question              example, suggests that sociologists have responsi-
     of tapping one’s own creativity in the quest for              bilities to their subjects and that secret research
     alternative methods. It has been suggested that               can injure other people in ways that cannot be an-
     role-playing, or ‘as-if’ experiments, could prove a           ticipated or compensated for afterwards, and that
     worthwhile avenue to explore – the ‘role-playing              sociologists have responsibilities towards fellow
     versus deception’ debate is raised in Chapter 21.             sociologists. Douglas (1976), by contrast, argues
     By this method, as we shall see, the subject is               that covert observation is necessary, useful and
     asked to behave as if he or she were a particular             revealing. Bulmer (1982), too, concludes that the
     person in a particular situation. Whatever form               most compelling argument in favour of covert re-
     they take, however, new approaches will involve               search is that it has produced good social science
     a radically different set of assumptions about the            which would not have been possible without the
     role of the subject in this type of research. They            method. It would be churlish, he adds, not to rec-
     require us to use subjects’ motivations rather than           ognize that the use of covert methods has advanced
     bypassing them. They may even call for increas-               our understanding of society.
     ing the sophistication of potential subjects, rather             Kimmel (1988) claims that few researchers feel
     than maintaining their naivety.                               that they can do without deception entirely, since
        Plummer (1983) informs us that even in an                  the adoption of an overtly conservative approach
     unlikely area like life history, deceptions of a              could render the study of important research hardly
                                                             ETHICS AND RESEARCH METHODS IN EDUCATION               69

worth the effort. A study of racial prejudice,               others?’ ‘How much can the researcher tell the

                                                                                                                         Chapter 2
for example, accurately labelled as such, would              pupils about a particular piece of research?’
certainly affect the behaviour of the subjects taking        ‘When is a casual conversation part of the
part. Deception studies, he considers, differ so             research data and when is it not?’ ‘Is gossip
greatly that even the harshest critics would be hard         legitimate data and can the researcher ethically use
pressed to state unequivocally that all deception            material that has been passed on in confidence?’
has potentially harmful effects on participants or           As Hitchcock and Hughes (1989) conclude, the
is wrong.                                                    list of questions is endless yet they can be related
                                                             to the nature of both the research technique
                                                             involved and the social organization of the setting
Ethics and research methods in education                     being investigated. The key to the successful
Ethical problems arising from research methods               resolution of such questions lies in establishing
used in educational contexts occur passim                    good relations. This will involve the development
in Burgess’s (1989) edited collection, The Ethics            of a sense of rapport between researchers and their
of Educational Research and in Simons and                    subjects that will lead to feelings of trust and
Usher’s (2000) edited volume, Situated Ethics                confidence.
in Educational Research. Every contribution in                  Fine and Sandstrom (1988) discuss in some
these reflects the reality of the day-to-day                  detail the ethical and practical aspects of doing
problems, issues and dilemmas that the educational           fieldwork with children. In particular they show
researcher and beginning researchers are likely              how the ethical implications of participant
to encounter. These two books show that the                  observation research differ with the age of
issues thrown up by the complexities of research             the children. Another feature of qualitative
methods in educational institutions and their                methods in this connection has been identified
ethical consequences are probably among the                  by Finch (1985: 116–17) who comments on the
least anticipated, particularly among the more               possible acute political and ethical dilemmas
inexperienced researchers, not least the socio-              arising from how data are used, both by the
political dimension of research. Newcomers to the            researcher and others, and that researchers
field need to be aware of those kinds of research             have a duty of trust placed in them by the
which, by their nature, lead from one problem                participants to use privileged data appropriately,
to another. Indeed, the researcher will frequently           not least for improvement of the condition of the
find that methodological and ethical issues are               participants.
inextricably interwoven in much of the research                 Kelly (1989a) suggests that the area in
we have designated as qualitative or interpretative.         qualitative research where one’s ethical antennae
As Hitchcock and Hughes (1989) note:                         need to be especially sensitive is that of action
                                                             research, and it is here that researchers, be
  Doing participant observation or interviewing one’s
                                                             they teachers or outsiders, must show particular
  peers raises ethical problems that are directly related
                                                             awareness of the traps that lie in wait. These
  to the nature of the research technique employed.
                                                             difficulties have been summed up by Hopkins
  The degree of openness or closure of the nature of
                                                             (1985: 135) when he suggests that, as the
  the research and its aims is one that directly faces the
                                                             researcher’s actions are deeply embedded in the
  teacher researcher.
                                                             organization, it is important to work within these,
                         (Hitchcock and Hughes 1989)
                                                             and this throws into relief issues of confidentiality
They go on to pose the kinds of question                     and personal respect.
that may arise in such a situation. ‘Where for                  Box 2.8 presents a set of principles specially
the researcher does formal observation end and               formulated for action researchers by Kemmis
informal observation begin?’ ‘Is it justifiable to            and McTaggart (1981) and quoted by Hopkins
be open with some teachers and closed with                   (1985).

     Box 2.8
     Ethical principles for the guidance of action researchers

           Observe protocol: take care to ensure that the relevant persons, committees and authorities have been consulted,
           informed and that the necessary permission and approval have been obtained.
           Involve participants: encourage others who have a stake in the improvement you envisage to shape and form the work.
           Negotiate with those affected: not everyone will want to be directly involved; your work should take account of the
           responsibilities and wishes of others.
           Report progress: keep the work visible and remain open to suggestions so that unforeseen and unseen ramifications can
           be taken account of; colleagues must have the opportunity to lodge a protest to you.
           Obtain explicit authorizations: this applies where you wish to observe your professional colleagues, and where you wish to
           examine documentation.
           Negotiate descriptions of people’s work: always allow those described to challenge your accounts on the grounds of
           fairness, relevance and accuracy.
           Negotiate accounts of others’ points of view (e.g. in accounts of communication): always allow those involved in interviews,
           meetings and written exchanges to require amendments which enhance fairness, relevance and accuracy.
           Obtain explicit authorization before using quotations: this includes verbatim transcripts, attributed observations, excerpts of
           audio and video recordings, judgements, conclusions or recommendations in reports (written or to meetings).
           Negotiate reports for various levels of release: remember that different audiences require different kinds of reports; what is
           appropriate for an informal verbal report to a faculty meeting may not be appropriate for a staff meeting, a report to
           council, a journal article, a newspaper, a newsletter to parents; be conservative if you cannot control distribution.
           Accept responsibility for maintaining confidentiality.
           Retain the right to report your work: provided that those involved are satisfied with the fairness, accuracy and relevance of
           accounts which pertain to them, and that the accounts do not unnecessarily expose or embarrass those involved, then
           accounts should not be subject to veto or be sheltered by prohibitions of confidentiality.
           Make your principles of procedure binding and known: all of the people involved in your action research project must agree
           to the principles before the work begins; others must be aware of their rights in the process.

     Source: adapted from Kemmis and McTaggart (1981) and quoted in Hopkins (1985: 134–6)

     Ethics and evaluative research                                        second principle, that of equal respect, demands
                                                                           that we respect the equal worth of all people. This
     Strike (1990), discussing the ethics of educational                   requires us to treat people as ends rather than
     evaluation, offers two broad principles which may                     means, to regard them as free and rational, and
     form the basis of further considerations in the                       to accept that they are entitled to the same basic
     field of evaluation. These are the principle of                        rights as others.
     benefit maximization and the principle of equal                           Strike (1990) lists the following ethical prin-
     respect. The former, the principle of benefit                          ciples which he regards as particularly important
     maximization, holds that the best decision is                         to evaluative research and which may be seen
     the one that results in the greatest benefit for                       in the light of the two broad principles outlined
     most people. It is pragmatic in the sense that                        above:
     it judges the rightness of our actions by their
     consequences or, as Strike (1990) says, the best                          Due process: evaluative procedures must
     action is the one with the best results. The                              ensure that judgements are reasonable: that
     principle of utilitarianism requires us to identify                       known and accepted standards are consistently
     the particular benefits we wish to maximize, to                            applied from case to case, that evidence is
     identify a suitable population for maximization,                          reasonable and that there are systematic and
     specify what is to count as maximization, and fully                       reasonable procedures for collecting and testing
     understand the consequences of our actions. The                           evidence.
                                              RESEARCH AND REGULATION: ETHICAL CODES AND REVIEW                  71

   Privacy: this involves a right to control             (these can constitute a major hurdle for those

                                                                                                                      Chapter 2
   information about oneself, and protects people        planning to undertake research), ethical codes of
   from unwarranted interference in their affairs.       the professional bodies and associations as well
   In evaluation, it requires that procedures are        as the personal ethics of individual researchers
   not overtly intrusive and that such evaluation        are all important regulatory mechanisms. All
   pertains only to those aspects of a teacher’s         investigators, from undergraduates pursuing a
   activity that are job related. It also protects the   course-based research project to professional
   confidentiality of evaluation information.             researchers striving at the frontiers of knowledge,
   Equality: in the context of evaluation, this can      must take cognizance of the ethical codes
   best be understood as a prohibition against           and regulations governing their practice. Failure
   making decisions on irrelevant grounds, such          to meet these responsibilities on the part of
   as race, religion, gender, ethnicity or sexual        researchers is perceived as undermining the whole
   orientation.                                          scientific process and may lead to legal and
   Public perspicuity: this principle requires           financial penalties and liabilities for individuals
   openness to the public concerning evaluative          and institutions.
   procedures, their purposes and their results.            Professional societies and associations have
   Humaneness: this principle requires that              formulated codes of practice which express the
   consideration is shown to the feelings and            consensus of values within a particular group and
   sensitivities of those in evaluative contexts.        which help individual researchers in indicating
   Client benefit: this principle requires that           what is desirable and what is to be avoided.
   evaluative decisions are made in a way that           Of course, this does not solve all problems,
   respects the interests of students, parents and       for there are few absolutes and in consequence
   the public, in preference to those of educational     ethical principles may be open to a wide
   institutions and their staff. This extends to         range of interpretations. The establishment of
   treating participants as subjects rather than as      comprehensive regulatory mechanisms is well
   ‘research fodder’.                                    founded in the United Kingdom, but it is perhaps
   Academic freedom: this requires that an atmo-         in the field of information and data – how they
   sphere of intellectual openness is maintained         are stored and the uses to which they are put, for
   in the classroom for both teachers and students.      example – that educational researchers are likely
   Evaluation should not be conducted in a way           to find growing interest. This category would
   that chills this environment.                         include, for instance, statistical data, data used
   Respect for autonomy: teachers are entitled           as the basis for evaluation, curricular records,
   to reasonable discretion in, and to exercise          written records, transcripts, data sheets, personal
   reasonable judgement about, their work.               documents, research data, computer files, and
   Evaluations should not be conducted so                audio and video recordings.
   as to unreasonably restrict discretion and               As information technology establishes itself
   judgement.                                            in a centre-stage position and as society has
                                                         become increasingly dependent on information,
Strike (1990) develops these principles in a more        the concept of information is important not only
extended and systematic form in his contribution.        for what it is, but for what it can do. Numerous
                                                         writers have pointed out the connection between
                                                         information and power, for example Harris et al.’s
Research and regulation: ethical codes                   (1992) comments on the power over individuals
and review                                               through the control of personal information and its
Ethical regulation exists at several levels:             relationship to power of professionalism in which
legislation, ethics review committees to oversee         submission to expert knowledge is required. Data
research in universities and other institutions          misuse, therefore, or disclosure at the wrong time or

     to the wrong client or organ, can result in the most We advise readers to consult these
     unfortunate consequences for an individual, group        in detail.
     or institution. And matters are greatly exacerbated         The British Psychological Society’s Code of
     if it is the wrong information, or incomplete, or        Conduct, Ethical Principles and Guidelines (2005)
     deliberately misleading.                                 includes, among many others, sections on
        In an increasingly information-rich world, it is      competence, obtaining consent, confidentiality
     essential that safeguards be established to protect      and personal conduct. Its section on Ethical
     it from misuse or abuse. The UK Data Protection          Principles for Conducting Research with Human
     Acts of 1984 and 1998 are designed to achieve            Participants first discusses deception, debriefing,
     such an end. These cover the principles of data          risk and implementation (pp. 6–7) and then
     protection, the responsibilities of data users, and      moves to eleven main sections: introduction;
     the rights of data subjects. Data held for ‘historical   general principles, including the guiding precept
     and research’ purposes are exempted from the             that ‘the essential principle is that the
     principle which gives individuals the right of           investigation should be considered from the
     access to personal data about themselves, provided       standpoint of all the participants; foreseeable
     the data are not made available in a form which          threats to their psychological well-being, health,
     identifies individuals. Research data also have           values or dignity should be eliminated’ (p. 8);
     partial exemption from two further principles, with      consent; deception; debriefing; withdrawal from
     the effect that such data may be held indefinitely        the investigation; confidentiality; protection of
     and the use of the data for research purposes need       participants; observational research; giving advice;
     not be disclosed at the time of data collection.         and colleagues. Interestingly it does not insist
        Of the two most important principles which do         on informed consent, rather expressing it
     concern research data, one states that personal          as ‘wherever possible, the investigator should
     data (i.e. data that uniquely identify the person        inform all participants of the objectives of the
     supplying them) shall be held only for specified          investigation’ (para. 3.1). Similarly it does not
     and lawful purposes. The second principle states         proscribe deception, indicating that ‘it may be
     that appropriate security measures shall be taken        impossible to study some psychological processes
     against unauthorized access to, or alteration,           without withholding information about the true
     disclosure or destruction of personal data and           object of the study or deliberately misleading
     against accidental loss or destruction of personal       the participants’ (para. 4.3). However, it says
     data.                                                    that these need to be rigorously justified, and
        Most institutions of higher education have            alternatives must have been explored and found
     their own ethics committees, and these usually           to be unavailable.
     have their own codes of ethics against which                The American Psychological Association’s
     they evaluate research proposals. In addition,           Ethical Principles and Code of Conduct (2002)
     some important codes of practice and guide-              states five general principles: beneficence and non-
     lines are published by research associations, for        maleficence, fidelity and responsibility, integrity,
     example the British Educational Research Asso-           justice, and respect for people’s rights and
     ciation (, the British Psy-        dignity. These principles then become the
     chological Society (, the          basis for ten sections of ‘ethical standards’:
     British Sociological Association (http://www.            resolving ethical issues; competence; human, the Social Research Associ-              relations (including ‘avoiding harm’ ‘exploitative
     ation (, the Ameri-            relationships’ and ‘informed consent’); privacy
     can Educational Research Association (http://            and confidentiality; advertising and other public, the American Psychological As-            statements; record keeping and fees; education and
     sociation ( and the Amer-             training; research and publication; assessment; and
     ican Sociological Association (http://www.               therapy.
                                             RESEARCH AND REGULATION: ETHICAL CODES AND REVIEW                  73

   The American Sociological Association’s Code         (para. B3)), rights of withdrawal, exploitation for

                                                                                                                     Chapter 2
of Ethics and Policies and Procedures of the            personal gain, sensitivity to local circumstances
ASA Committee on Professional Ethics (1999) has         (e.g. culture, religion, gender), avoidance of neg-
five general principles: professional competence;        ative consequences, dissemination, anonymity;
integrity; professional and scientific responsibility;   intellectual ownership; editing, reviewing and
respect for people’s rights, dignity and diversity;     appraising research; sponsors, policy-makers and
and social responsibility. These are then devolved      other users of research; and students and student
onto twenty ethical standards, including non-           researchers.
exploitation, confidentiality, informed consent,            Web sites of these research associations’ ethical
deception, offering inducements and many others.        principles and guidelines can be found either on
   The Statement of Ethical Practice for the            the home page of each association or as follows:
British Sociological Association (2002) includes
                                                        American Educational Research Association:
sections on: professional integrity; relations with
and responsibilities towards research participants;
                                                        AERA/Ethical Standards/Ethical
relationships with research participants; covert
research; anonymity, privacy and confidentiality;
                                                        American Psychological Association: http://www.
relations with and responsibilities towards sponsors
and/or funders; carrying obligations, roles and
                                                        American Sociological Association: http://www.
rights; pre-empting outcomes and negotiations
about research; and obligations to sponsors and/or
                                                        British Educational Research Association: http://
funders during the research process.
   The Social Research Association’s Ethical
                                                        British Psychological Society: http://www.bps.
Guidelines (2003) draws on European law
( and indicates four
                                                        document-download$.cfm?file uuid=
levels of obligations: to society; to funders and
employers; to colleagues; and to subjects (includ-
ing avoiding undue intrusion, obtaining informed
                                                        British Sociological Association: http://www.
consent, modifications to informed consent, pro-
                                               site/user doc/
tecting the interests of subjects, enabling partici-
pation, maintaining confidentiality of records, and
preventing disclosure of identities).
                                                        Social Research Association: http://www.thesra.
   The British Educational Research Associa-
tion’s Ethical Guidelines (2000) are devolved onto:
responsibilities to the research profession; respon-       The difficulty and yet the strength with
sibility to the participants (including working with    ethical codes is that they cannot and do not
children, informed consent, rights to withdrawal);      provide specific advice for what to do in specific
responsibility to the public; relationships with        situations. Ultimately, it is researchers themselves,
funding agencies; publication; intellectual owner-      their integrity and conscience, informed by an
ship; relationship with host institution. Similarly,    acute awareness of ethical issues, underpinned
the Ethical Standards of the American Educational Re-   by guideline codes and regulated practice, which
search Association (2000) includes: responsibilities    should decide what to do in a specific situation,
to the field; research populations, educational          and this should be justified, justifiable, thought
institutions, and the public (including working         through and defensible.
with children), informed consent, confidentiality,          There is a certain degree of homogeneity
honesty (‘deception is discouraged’ and ‘should         between the codes and guidelines cited above.
be used only when clearly necessary’, after which       While they are helpful in providing guidance,
the reasons for the deception should be explained       they cannot tell the researcher what to do in every

     unique circumstance. The issue here is that ethics              how to conduct the research
     are ‘situated’ (Simons and Usher 2000). Indeed                  what results the researcher should look for and
     the authors state at the outset that                            what findings should be suppressed
                                                                     what should and should not be reported
       while ethics has traditionally been seen as a set of
                                                                     to conceal who the sponsor is
       general principles invariantly and validly applied to
                                                                     what are the purposes of the research.
       all situations . . . on the contrary, ethical principles
       are mediated within different research practices and       On the other hand, sponsors do have the right to
       thus take on different significances in relation to         remain confidential; they may have the right to
       those practices.                                           non-disclosure of who they are, and the purposes
                                  (Simons and Usher 2000: 1)      and findings of the research.
                                                                     While sponsored research is usually contractual
     The authors state that this implies that situated
                                                                  between the researcher and the sponsor, and
     ethics are ‘immune to universalization’, because
                                                                  between the researcher and the participants, and
       researchers cannot avoid weighing up conflicting            while the research may be for the sponsor alone
       considerations and dilemmas which are located in           and not for the public, this does not privilege the
       the specificities of the research situation and where       sponsor in dictating how the research should be
       there is a need to make ethical decisions but where        conducted and what it should find; in short, ‘fixing’
       those decisions cannot be reached by appeal to             the study.
       unambiguous and univalent principles or codes.                Of course the researcher’s responsibilities may
                                (Simons and Usher 2000: 2)        lie only in conducting the study and providing the
                                                                  sponsor with a report; what happens to the report
     Indeed, it was observed earlier that many ethical
                                                                  after that (e.g. whether it is released completely,
     codes and guidelines themselves avoid univalency
                                                                  selectively or not at all to the public or other
     and unambiguity, arguing, for example, that
                                                                  parties within the sponsor’s organization) is a
     deception, covert research and the lack of
                                                                  matter for the sponsor. However, this does not
     informed consent may be justified. The need for
                                                                  absolve the researcher from decisions about the
     polyvalency (multiple interpretations of what is
                                                                  conduct of the study, and the researcher must
     worthwhile, acceptable and valuable) and situated
                                                                  retain the right to conduct the study as she or
     ethics, Simons and Usher (2000: 11) argue, arises
                                                                  he thinks fit, informed by, but not decided by,
     from the practicality of conducting research, the
                                                                  the sponsor. The researcher’s integrity must be
     need for sensitivity to socio-political contexts and
                                                                  absolute. It is often the case that researchers will
     to be fair to disadvantaged groups, and to take
                                                                  negotiate publication rights with the sponsor in
     account of the diversity and uniqueness of different
                                                                  advance of the research and what confidentiality
     research practices. What this suggests, then, is
                                                                  the researcher must respect.
     that, while codes and guidelines may be useful
                                                                     The sponsor has a right to expect high quality,
     in raising issues and orienting researchers, they
                                                                  rigorous and usable research. The researcher
     cannot decide what should and should not be
                                                                  should not succumb to pressure to
     done in a specific situation; that is for individual
     researchers and their informed conscience to                    betray the confidentiality of the respondents
     decide.                                                         tamper with data, their analysis or presentation
                                                                     to meet a particular objective
                                                                     present selective and unrepresentative data and
     Sponsored research
     Sponsored research does not absolve the researcher              make recommendations that do not arise from
     from ethical behaviour. For example, it may be                  the data themselves
     considered unethical for the sponsor to tell the                use the data for non-negotiated personal
     researcher:                                                     interests, agendas, purposes and advancement
                                                                                               CONCLUSION        75

   conduct a study in which personal research              protect their reputation

                                                                                                                      Chapter 2
   objectives influence the nature, contents and            enable further research to be conducted
   conduct of the research.                                expect them to behave ethically
                                                           ensure that they adhere to correct and agreed
The researcher has obligations to the sponsor, but
not to doctor or compromise the research.
                                                           protect the anonymity and confidentiality of
                                                           sponsors if so agreed.
Responsibilities to the research
community                                               The researcher is a member of a research
                                                        community, and this brings ethical responsibili-
The researcher has responsibilities to the research     ties.
community, for example not to jeopardize the
reputation of the research community (e.g. the
university) or spoil the opportunities for further      Conclusion
research. Thus, a novice researcher working for
                                                        In this chapter we have attempted to acquaint
a higher degree may approach a school directly,
                                                        readers with some of the ethical difficulties they
using a clumsy approach, with inadequate data col-
                                                        are likely to experience in the conduct of such
lection instruments and a poor research design, and
                                                        research. It is not possible to identify all potential
then proceed to publicize the results as though they
                                                        ethical questions or adjudicate on what is correct
are valid and reliable. This researcher does not de-
                                                        researcher behaviour.3 It is hoped that these
serve the degree; at the very least he or she should
                                                        pages will have induced in readers a certain
have sought and gained advice from the supervisor,
                                                        disposition that will enable them to approach their
modified the research as necessary, gained approval
                                                        own projects with a greater awareness and fuller
for the research, made suitably sensitive overtures
                                                        understanding of the ethical dilemmas and moral
to the school, and agreed rights of disclosure. Not
                                                        issues lurking in the interstices of the research
to do so puts the researcher’s institution at risk
                                                        process. However inexperienced in these matters
of being denied further access, of damaging the
                                                        researchers are, they bring to social research a sense
reputation of the institution, and, if word spreads,
                                                        of rightness (Huizinga 1949) on which they can
of being publicly vilified and denied the opportu-
                                                        construct a set of rational principles appropriate
nity for further research to be conducted. In this
                                                        to their own circumstances and based on personal,
case the novice researcher has behaved uneth-
                                                        professional, and societal values (we stress the
ically (see
                                                        word ‘rational’ since reason is a prime ingredient
9780415368780 – Chapter 2, file 2.6. ppt).
                                                        of ethical thinking and it is the combination of
   Further, what responsibility to the research
                                                        reason and a sense of rightness that researchers
community does the researcher have? If a negative
                                                        must keep faith with if they are to bring a rich
research report is released will schools retrench,
                                                        ethical quality to their work).
preventing future research in schools from being
                                                           Although no code of practice can anticipate or
undertaken? Negative research data, such as
                                                        resolve all problems, there is a six-fold advantage
reported evidence on deliberate grade inflation by
                                                        in fashioning a personal code of ethical practice.
schools in order to preserve reputation (Morrison
                                                        First, such a code establishes one as a member of
and Tang 2002), may not endear researchers to
                                                        the wider scientific community having a shared
                                                        interest in its values and concerns. Second, a code
   The researcher has a responsibility to colleagues
                                                        of ethical practice makes researchers aware of
                                                        their obligations to their subjects and also to those
   protect their safety (e.g. in conducting sensitive   problem areas where there is a general consensus
   research or research in dangerous locations)         about what is acceptable and what is not. In this
   protect their well-being                             sense it has a clarificatory value. Third, when one’s

     professional behaviour is guided by a principled                   a short ethical code, by way of example. It must
     code of ethics, then it is possible to consider that               be stressed, however, that bespoke items, i.e. those
     there may be alternative ways of doing the same                    designed to meet the needs of a specific project, are
     thing, ways that are more ethical or less unethical                preferable to standard ones. The items in Box 2.9
     should one be confronted by a moral challenge.                     are illustrative, and in no way exhaustive.
     Fourth, a balanced code can be an important                           In more detail, one can suggest that further
     organizing factor in researchers’ perceptions of the               considerations have to be borne in mind in
     research situation, and as such may assist them in                 planning, conducting and reporting educational
     their need to anticipate and prepare. Fifth, a code                research (Box 2.10).
     of practice validated by their own sense of rightness                 Box 2.10 raises issues and suggestions, not
     will help researchers to develop an intuitive                      solutions or decisions. These will have to be
     sensitivity that will be particularly helpful to them              decided by each researcher in respect of the
     in dealing with the unknown and the unexpected,                    particular situation he or she faces. For a summary
     especially where the more fluidic methods such                      of ethical principles for social research see the
     as ethnography and participant observation are                     accompanying web site (http://www.routledge.
     concerned. And sixth, a code of practice will bring                com/textbooks/9780415368780 – Chapter 2, file
     discipline to researchers’ awareness. Box 2.9 gives                2.1.doc).

     Box 2.9
     An ethical code: an illustration

        1   It is important for the researcher to reveal fully his or her identity and background.
        2   The purpose and procedures of the research should be fully explained to the subjects at the outset.
        3   The research and its ethical consequences should be seen from the subjects’ and institution’s point of view.
        4   Possible controversial findings need to be anticipated and, where they ensue, handled with great sensitivity.
        5   The research should be as objective as possible: this will require careful thought being given to the design, conduct and
            reporting of research.
        6   Informed consent should be sought from all participants: all agreements reached at this stage should be honoured.
        7   Sometimes it is desirable to obtain informed consent in writing.
        8   Subjects should have the option to refuse to take part and know this, and the right to terminate their involvement at
            any time and know this also.
        9   Arrangements should be made during initial contacts to provide feedback for participants who request it: this may take
                                    e    e
            the form of a written r´ sum´ of findings.
       10   The dignity, privacy and interests of the participants should be respected and protected at all times.
       11   Deceit should be used only when absolutely necessary.
       12   When ethical dilemmas arise, the researcher may need to consult other researchers or teachers.

     Source: adapted from Reynolds 1979
                                                                                                                    CONCLUSION        77

Box 2.10

                                                                                                                                           Chapter 2
Ethical principles for educational research (to be agreed before the research commences)

  Responsibility to research
     The researcher should be competent and aware of what is involved in conducting research.
     The research must be conducted rigorously and with the correct procedures – avoid misuse of procedures at all stages.
     Report procedures accurately and publicly (rigour).
     Don’t jeopardize future research(ers).
     Report clearly and make data available for checking.
     Tell the truth: do not tell lies or falsify data, avoid being unfairly selective (e.g. to support a case), do not misrepresent
     Maintain the integrity and autonomy of the research, e.g. avoid censorship of, or interference with, the research by
     sponsors or those who give permission for the research to be undertaken.

  Responsibility to participants and audience(s)
     Gain fully informed consent where appropriate (usually in writing), in order to respect self-determination and
     autonomy; provide information on all aspects of the research and its possible consequences.
     Decide whether, and how, overt or covert research is required/justified.
     Decide whether, and how, deception is required/justified; be honest or justify dishonesty.
     Ensure non-maleficence (no harm, hurt or suffering to be caused to participants and those who might be affected by the
     research); be humane.
     Ensure beneficence (the research will bring benefit to the participants or will contribute to the welfare of participants).
     Ensure that participants do not leave the research worse off than when they started it.
     Respect people’s rights and dignity and interests, and be respectful: research participants are subjects, not objects to be
     exploited. Treat people as subjects, not objects.
     Agree individuals’ rights to privacy.
     Ensure participants have the right to withdraw at any time.
     Inform participants who will have access to the data/report, i.e. the audiences of the research, how public it will be,
     when it will become public, and how it will be disseminated; negotiate levels of release, i.e. who see which parts of the
     Ensure anonymity/confidentiality/non-traceability; if these are not possible then tell participants in advance.
     Indicate how anonymity will be addressed (e.g. by confidentiality, aggregation of data).
     Inform participants how data will be collected and how files/questionnaires/audio data/video data/computer files will be
     stored during the research and destroyed after use.
     Ensure sensitivity to people (e.g. age, ethnicity, gender, culture, religion, language, socio-economic status).
     Gain permission from all relevant parties (e.g. parents/guardians, school, principals etc.) for access.
     Respect vulnerability (e.g. in interviewing children or those without power).
     Agree respondent validation.
     Agree ownership of the data (and when ownership passes from participants to researcher).
     Allow time for review.
     Avoid causing unnecessary offence. Thank the participants.
     Ensure that participants and sponsors have the right to dissent or distance themselves from the research.
     Demonstrate social responsibility and obligations.
     Consider indemnification, liabilities and disclaimers.
     Don’t abuse your position or power as a researcher.
     Don’t use dangerous methods.
3         Planning educational research

Introduction                                               the researcher is in a very positive position to
There is no single blueprint for planning research.        undertake the research. The setting up of the
Research design is governed by the notion                  research is a balancing act, for it requires the
of ‘fitness for purpose’. The purposes of the               harmonizing of planned possibilities with workable,
research determine the methodology and design              coherent practice, i.e. the resolution of the difference
of the research. For example, if the purpose of            between what could be done/what one would
the research is to map the field, or to make                like to do and what will actually work/what
generalizable comments then a survey approach              one can actually do, for, at the end of the
might be desirable, using some form of stratified           day, research has to work. In planning research
sample; if the effects of a specific intervention are       there are two phases – a divergent phase and a
to be evaluated then an experimental or action             convergent phase. The divergent phase will open
research model may be appropriate; if an in-depth          up a range of possible options facing the researcher,
study of a particular situation or group is important      while the convergent phase will sift through these
then an ethnographic model might be suitable.              possibilities, see which ones are desirable, which
   That said, it is possible, nevertheless, to identify    ones are compatible with each other, which ones
a set of issues that researchers need to address,          will actually work in the situation, and move
regardless of the specifics of their research. This         towards an action plan that can realistically
chapter addresses this set of issues, to indicate          operate. This can be approached through the
those matters that need to be addressed in practice        establishment of a framework of planning
so that an area of research interest can become            issues (see
practicable and feasible. This chapter indicates           9780415368780 – Chapter 3, file 3.1. ppt).
how research might be operationalized, i.e. how a
general set of research aims and purposes can be
                                                           A framework for planning research
translated into a practical, researchable topic.           Clearly, the set of issues that constitute a
   To change the ‘rules of the game’ in midstream          framework for planning research will need to
once the research has commenced is a sure recipe           be interpreted differently for different styles of
for problems. The terms of the research and the            research, nevertheless it is useful to indicate what
mechanism of its operation must be ironed out              those issues might be (see Box 3.1).
in advance if it is to be credible, legitimate and            A possible sequence of consideration is shown
practicable. Once they have been decided upon,             in the diagram.

Preparatory issues    →     Methodology         →     Sampling and         →     Piloting   →     Timing and
                                                      instrumentation                             sequencing
Constraints,          →     Approaches,         →     Reliability          →                →
purposes, foci,             reliability and           and validity,
ethics, research            validity                  pre-piloting
question, politics
                                                                          A FRAMEWORK FOR PLANNING RESEARCH                  79

Box 3.1

                                                                                                                                  Chapter 3
The elements of research design

   1   A clear statement of the problem/need that has given rise to the research.
   2   Constraints on the research (e.g. access, time, people, politics).
   3   The general aims and purposes of the research.
   4   The intended outcomes of the research: what the research will do and what is the ‘deliverable’ outcome.
   5   How to operationalize research aims and purposes.
   6   Generating research questions (specific, concrete questions to which concrete answers can be given) and hypotheses
       (if appropriate).
   7   The foci of the research.
   8   Identifying and setting in order the priorities for the research.
   9   Approaching the research design.
  10   Focusing the research.
  11   Research methodology (approaches and research styles, e.g. survey; experimental; ethnographic/naturalistic;
       longitudinal; cross-sectional; historical; correlational; ex post facto).
  12   Ethical issues and ownership of the research (e.g. informed consent; overt and covert research; anonymity;
       confidentiality; non-traceability; non-maleficence; beneficence; right to refuse/withdraw; respondent validation;
       research subjects; social responsibility; honesty and deception).
  13   Politics of the research: who is the researcher; researching one’s own institution; power and interests; advantage;
       insider and outsider research.
  14   Audiences of the research.
  15   Instrumentation, e.g. questionnaires; interviews; observation; tests; field notes; accounts; documents; personal
       constructs; role-play.
  16   Sampling: size/access/representativeness; type: probability: random, systematic, stratified, cluster, stage,
       multi-phase; non-probability: convenience, quota, purposive, dimensional, snowball.
  17   Piloting: technical matters: clarity, layout and appearance, timing, length, threat, ease/difficulty, intrusiveness;
       questions: validity, elimination of ambiguities, types of questions (e.g. multiple choice, open-ended, closed),
       response categories, identifying redundancies; pre-piloting: generating categories, grouping and classification.
  18   Time frames and sequence (what will happen, when and with whom).
  19   Resources required.
  20   Validity: construct; content; concurrent; face; ecological; internal; external.
  21   Reliability: consistency (replicability); equivalence (inter-rater, equivalent forms), predictability; precision;
       accuracy; honesty; authenticity; richness; dependability; depth; overcoming Hawthorne and halo effects;
       triangulation: time; space; theoretical; investigator; instruments.
  22   Data analysis.
  23   Verifying and validating the data.
  24   Reporting and writing up the research.

   Clearly this need not be the actual sequence;                    Orienting decisions are those decisions which
for example it may be necessary to consider access               set the boundaries or the constraints on the
to a possible sample at the very outset of the re-               research. For example, let us say that the overriding
search (see                  feature of the research is that it has to be
9780415368780 – Chapter 3, file 3.2. ppt).                        completed within six months; this will exert
   These issues can be arranged into four main                   an influence on the enterprise. On the one
areas (Morrison 1993):                                           hand, it will ‘focus the mind’, requiring priorities
                                                                 to be settled and data to be provided in a
   orienting decisions                                           relatively short time. On the other hand, this
   research design and methodology                               may reduce the variety of possibilities available to
   data analysis                                                 the researcher. Hence questions of time scale will
   presenting and reporting the results.                         affect:

        the research questions which might be                   the number of foci which can be covered in the
        answered feasibly and fairly (for example, some         time (for example, in uncovering relevant data,
        research questions might require a long data            some foci might be costly in researcher’s time)
        collection period)                                      the size and nature of the reporting (for
        the number of data collection instruments used          example, the number of written reports
        (for example, there might be only enough time           produced, the costs of convening meetings).
        for a few instruments to be used)
                                                                Certain time scales permit certain types
        the sources (people) to whom the researcher
                                                             of research, thus a short time scale permits
        might go (for example, there might only be
                                                             answers to short-term issues, while long-term or
        enough time to interview a handful of people)
                                                             large questions might require a long-term data
        the number of foci which can be covered in
                                                             collection period to cover a range of foci. Costs in
        the time (for example, for some foci it will take
                                                             terms of time, resources and people might affect the
        a long time to gather relevant data)
                                                             choice of data collection instruments. Time and
        the size and nature of the reporting (there
                                                             cost will require the researcher to determine, for
        might only be time to produce one interim
                                                             example, what will be the minimum representative
                                                             sample of teachers or students in a school, as
                                                             interviews are time-consuming and questionnaires
     By clarifying the time scale a valuable note of
                                                             are expensive to produce. These are only two
     realism is injected into the research, which enables
                                                             examples of the real constraints on the research
     questions of practicability to be answered.
                                                             which must be addressed. Planning the research
        Let us take another example. Suppose the
                                                             early on will enable the researcher to identify
     overriding feature of the research is that the costs
                                                             the boundaries within which the research must
     in terms of time, people and materials for carrying
                                                             operate and what the constraints are on it.
     it out are to be negligible. This, too, will exert an
                                                                Let us take another important set of questions:
     effect on the research. On the one hand, it will
                                                             is the research feasible? Can it actually be done?
     inject a sense of realism into proposals, identifying
                                                             Will the researchers have the necessary access to
     what is and what is not manageable. On the
                                                             the schools, institutions and people? This issue
     other hand, it will reduce, again, the variety of
                                                             becomes a major feature if the research is in any
     possibilities which are available to the researcher.
                                                             way sensitive (see Chapter 5).
     Questions of cost will affect:
                                                                With these preliminary comments, let us turn to
                                                             the four main areas of the framework for planning
        the research questions which might be feasibly
        and fairly answered (for example, some
        research questions might require interviewing,
        which is costly in time both to administer           Orienting decisions
        and transcribe, or expensive commercially
                                                             Decisions in this field are strategic; they set the
        produced data collection instruments (e.g.
                                                             general nature of the research, and there are several
        tests) and costly computer services, which may
                                                             questions that researchers may need to consider:
        include purchasing software)
        the number of data collection instruments               Who wants the research?
        used (for example, some data collection                 Who will receive the research/who is it for?
        instruments, e.g. postal questionnaires, are            Who are the possible/likely audiences of the
        costly for reprographics and postage)                   research?
        the people to whom the researcher might go              What powers do the recipients of the research
        (for example, if teachers are to be released from       have?
        teaching in order to be interviewed, then cover         What are the general aims and purposes of the
        for their teaching may need to be found)                research?
                                                                 A FRAMEWORK FOR PLANNING RESEARCH                81

   What are the main priorities for and constraints          Where else will data be available (e.g.

                                                                                                                       Chapter 3
   on the research?                                          documentary sources)?
   Is access realistic?                                      How will the data be gathered (i.e.
   What are the time scales and time frames of               instrumentation)?
   the research?                                             Who will undertake the research?
   Who will own the research?
   At what point will the ownership of the
                                                          How to operationalize research questions
   research pass from the participants to the
   researcher and from the researcher to the              The process of operationalization is critical
   recipients of the research?                            for effective research. Operationalization means
   Who owns the data?                                     specifying a set of operations or behaviours that can
   What ethical issues are to be faced in                 be measured, addressed or manipulated. What is
   undertaking the research?                              required here is translating a very general research
   What resources (e.g. physical, material,               aim or purpose into specific, concrete questions to
   temporal, human, administrative) are required          which specific, concrete answers can be given.
   for the research?                                      The process moves from the general to the
                                                          particular, from the abstract to the concrete. Thus
It can be seen that decisions here establish some
                                                          the researcher breaks down each general research
key parameters of the research, including some
                                                          purpose or general aim into more specific research
political decisions (for example, on ownership and
                                                          purposes and constituent elements, continuing the
on the power of the recipients to take action on
                                                          process until specific, concrete questions have been
the basis of the research). At this stage the overall
                                                          reached to which specific answers can be provided.
feasibility of the research will be addressed.
                                                          Two examples of this are provided below.
                                                             Let us imagine that the overall research aim
Research design and methodology                           is to ascertain the continuity between primary
                                                          and secondary education (Morrison 1993: 31–3).
If the preceding orienting decisions are strategic
                                                          This is very general, and needs to be translated
then decisions in this field are tactical; they
                                                          into more specific terms. Hence the researcher
establish the practicalities of the research,
                                                          might deconstruct the term ‘continuity’ into
assuming that, generally, it is feasible (i.e. that the
                                                          several components, for example experiences,
orienting decisions have been taken). Decisions
                                                          syllabus content, teaching and learning styles,
here include addressing such questions as:
                                                          skills, concepts, organizational arrangements, aims
   What are the specific purposes of the research?         and objectives, ethos, assessment. Given the vast
   How are the general research purposes and              scope of this the decision is taken to focus on
   aims operationalized into specific research             continuity of pedagogy. This is then broken down
   questions?                                             into its component areas:
   What are the specific research questions?
                                                             the level of continuity of pedagogy
   What needs to be the focus of the research in
                                                             the nature of continuity of pedagogy
   order to answer the research questions?
                                                             the degree of success of continuity of pedagogy
   What is the main methodology of the research
                                                             the responsibility for continuity
   (e.g. a quantitative survey, qualitative research,
                                                             record keeping and documentation of continu-
   an ethnographic study, an experiment, a case
   study, a piece of action research etc.)?
                                                             resources available to support continuity.
   How will validity and reliability be addressed?
   What kinds of data are required?                       The researcher might take this further into
   From whom will data be acquired (i.e.                  investigating: the nature of the continuity (i.e. the
   sampling)?                                             provision of information about continuity); the

     degree of continuity (i.e. a measure against a given       It is now possible to identify not only the specific
     criterion); the level of success of the continuity (i.e.   questions to be posed, but also the instruments
     a judgement). An operationalized set of research           that might be needed to acquire data to an-
     questions, then, might be as follows:                      swer them (e.g. semi-structured interviews, rating
                                                                scales on questionnaires, or documentary analy-
        How much continuity of pedagogy is occurring
                                                                sis). By this process of operationalization we thus
        across the transition stages in each curriculum
                                                                make a general purpose amenable to investiga-
        area? What kind of evidence is required to
                                                                tion, e.g. by measurement (Rose and Sullivan
        answer this question? On what criteria will the
                                                                1993: 6) or some other means. The number of
        level of continuity be decided?
                                                                operationalized research questions is large here,
        What pedagogical styles operate in each
                                                                and may have to be reduced to maybe four or
        curriculum area? What are the most frequent
                                                                five at most, in order to render the research man-
        and most preferred? What is the balance of
                                                                ageable (see
        pedagogical styles? How is pedagogy influenced
                                                                9780415368780 – Chapter 3, file 3.3. ppt).
        by resources? To what extent is continuity
                                                                   An alternative way of operationalizing research
        planned and recorded? On what criteria will
                                                                questions takes the form of hypothesis raising and
        the nature of continuity be decided? What kind
                                                                hypothesis testing. A ‘good’ hypothesis has several
        of evidence is required to answer this question?
                                                                features. First, it is clear on whether it is direc-
        On what aspects of pedagogy does planning
                                                                tional or non-directional: a directional hypothesis
        take place? By what criteria will the level of
                                                                states the kind or direction of difference or rela-
        success of continuity be judged? Over how
                                                                tionship between two conditions or two groups of
        many students, teachers or curriculum areas
                                                                participants (e.g. students’ performance increases
        will the incidence of continuity have to occur
                                                                when they are intrinsically motivated). A non-
        for it to be judged successful? What kind of
                                                                directional hypothesis simply predicts that there
        evidence is required to answer this question?
                                                                will be a difference or relationship between two
        Is continuity occurring by accident or design?
                                                                conditions or two groups of participants (e.g. there
        How will the extent of planned and unplanned
                                                                is a difference in students’ performance according
        continuity be gauged? What kind of evidence
                                                                to their level of intrinsic motivation), without
        is required to answer this question?
                                                                stating whether the difference, for example, is an
        Who has responsibility for continuity at the
                                                                increase or a decrease). (For statistical purposes,
        transition points? What is being undertaken by
                                                                a directional hypothesis requires a one-tailed test
        these people?
                                                                whereas a non-directional hypothesis uses a two-
        How are records kept on continuity in the
                                                                tailed test, see Part Five.) Directional hypotheses
        schools? Who keeps these records? What is
                                                                are often used when past research, predictions, or
        recorded? How frequently are the records
                                                                theory suggest that the findings may go in a particu-
        updated and reviewed? What kind of evidence
                                                                lar direction, whereas non-directional hypotheses
        is required to answer this question?
                                                                are used when past research or theory is unclear or
        What resources are there to support continuity
                                                                contradictory or where prediction is not possible,
        at the point of transition? How adequate are
                                                                i.e. where the results are more open-ended.
        these resources? What kind of evidence is
                                                                   Second, a ‘good’ hypothesis is written in a
        required to answer this question?
                                                                testable form, in a way that makes it clear how
     It can be seen that these questions, several in            the researcher will design an experiment or survey
     number, have moved the research from simply                to test the hypothesis, for example, people perform
     an expression of interest (or a general aim) into          a mathematics task better when there is silence in
     a series of issues that lend themselves to being           the room than when there is not. The concept of
     investigated in concrete terms. This is precisely          interference by noise has been operationalized in
     what we mean by the process of operationalization.         order to produce a testable hypothesis.
                                                                  A FRAMEWORK FOR PLANNING RESEARCH               83

   Third, a ‘good’ hypothesis is written in a form       usually written thus:

                                                                                                                       Chapter 3
that can yield measurable results. For example,
                                                           H0 :    the null hypothesis
in the hypothesis people work better in quiet rather
than noisy conditions it is important to define the         H1 :    the alternative hypothesis.
operations for ‘work better’, ‘quiet’ and ‘noisy’.       We address the hypothesis testing approach fully
Here ‘perform better’ might mean ‘obtain a higher        in Part Five.
score on the mathematics test’, ‘quiet’ might
mean ‘silence’, and ‘noisy’ might mean ‘having
music playing’. Hence the fully operationalized
                                                         Distinguishing methods from methodologies
hypothesis might be people obtain a higher score on
a mathematics test when tested when there is silence     In planning research it is important to clarify
rather than when there is music playing. One can         a distinction that needs to be made between
see here that the score is measurable and that           methodology and methods, approaches and
there is zero noise, i.e. a measure of the noise         instruments, styles of research and ways of
level.                                                   collecting data. Several of the later chapters of
   In conducting research using hypotheses one           this book are devoted to specific instruments for
has to be prepared to use several hypotheses             collecting data; for example:
(Muijs 2004: 16) in order to catch the complexity
of the phenomenon being researched, and not
least because mediating variables have to be
included in the research. For example, the degree
of ‘willing cooperation’ (dependent variable)
in an organization’s staff is influenced by
                                                            biographies and case studies
professional leadership (independent variable)
and the personal leadership qualities of the
leader (mediating variable: Mastrangelo et al.
                                                            personal constructs.
2004) – which needs to be operationalized more
specifically, of course.                                  The decision on which instrument (method) to
   There is also the need to consider the                use frequently follows from an important earlier
null hypothesis and the alternative hypothesis           decision on which kind (methodology) of research
(discussed in Part Five) in research that is cast        to undertake, for example:
into a hypothesis testing model. The null hypothesis
                                                            a survey
states that, for example, there is no relationship
                                                            an experiment
between two variables, or that there has been
                                                            an in-depth ethnography
no difference in participants’ scores on a pretest
                                                            action research
and a post-test of history, or that there is no
                                                            case study research
difference between males and females in respect of
                                                            testing and assessment.
their science examination results. The alternative
hypothesis states, for example: there is a correlation      Subsequent chapters of this book set out each
between motivation and performance; there is a           of these research styles, their principles, rationales
difference between males’ and females’ scores on         and purposes, and the instrumentation and data
science; there is a difference between the pretest       types that seem suitable for them. For conceptual
and post-test scores on history. The alternative         clarity it is possible to set out some key features
hypothesis is often supported when the null              of these models (Box 3.2). It is intended that,
hypothesis is ‘not supported’, i.e. if the null          when decisions have been reached on the stage
hypothesis is not supported then the alternative         of research design and methodology, a clear
hypothesis is. The two kinds of hypothesis are           plan of action will have been prepared. To this

     Box 3.2
     Elements of research styles

       Model          Purposes                 Foci                    Key terms                  Characteristics
       Survey         Gathering large-scale    Opinions                Measuring                  Describes and explains
                      data in order to make    Scores                  Testing
                      generalizations          Outcomes                Representativeness         Represents wide
                                               Conditions              Generalizability           population
                      Generating statis-       Ratings                                            Gathers numerical data
                      tically manipulable
                      data                                                                        Much use of
                                                                                                  questionnaires and
                      Gathering                                                                   assessment/test data
                      context-free data
       Experiment     Comparing under          Initial states,         Pretest and post-test      Control and
                      controlled conditions    intervention and                                   experimental groups
                                               outcomes                Identification, isolation
                      Making                                           and control of key         Treats situations like a
                      generalizations about    Randomized controlled   variables                  laboratory
                      efficacy                  trials
                                                                       Generalizations            Causes due to
                      Objective                                                                   experimental
                      measurement of                                   Comparing                  intervention
                                                                       Causality                  Does not judge worth
                      Establishing causality
       Ethnography    Portrayal of events in   Perceptions and views   Subjectivity               Context specific
                      subjects’ terms          of participants
                                                                       Honesty, authenticity      Formative and
                      Subjective and           Issues as they emerge                              emergent
                      reporting of multiple    over time               Non-generalizable
                      perspectives                                                                Responsive to emerging
                                                                       Multiple perspectives      features
                      Description, unders-
                      tanding and                                      Exploration and rich       Allows room for
                      explanation of a                                 reporting of a specific     judgements and
                      specific situation                                context                    multiple perspectives

                                                                       Emergent issues            Wide database
                                                                                                  gathered over a long
                                                                                                  period of time

                                                                                                  Time-consuming to
                                                                                                  process data
                                                                   A FRAMEWORK FOR PLANNING RESEARCH                      85

Box 3.2

                                                                                                                               Chapter 3

  Model        Purposes                 Foci                    Key terms                   Characteristics
  Action       To plan, implement,      Everyday practices      Action                      Context-specific
  research     review and evaluate                              Improvement
               an intervention          Outcomes of             Reflection                   Participants as
               designed to improve      interventions           Monitoring                  researchers
               practice/solve local                             Evaluation
               problem                                          Intervention                Reflection on practice
               To empower                                       Empowering                  Interventionist – leading
               participants through                             Planning                    to solution of ‘real’
               research involvement     Participant             Reviewing                   problems and meeting
               and ideology critique    empowerment                                         ‘real’ needs

               To develop reflective                                                         Empowering for
               practice                                                                     participants

               To promote equality      Reflective practice                                  Collaborative
                                                                                            Promoting praxis and
               To link practice and     Social democracy and                                equality
               research                 equality
                                                                                            Stakeholder research
               To promote               Decision-making
               collaborative research
  Case study   To portray, analyse      Individuals and local   Individuality, uniqueness   In-depth, detailed data
               and interpret the        situations                                          from wide data source
               uniqueness of real                               In-depth analysis and
               individuals and          Unique instances        portrayal                   Participant and
               situations through                                                           non-participant
               accessible accounts      A single case           Interpretive and            observation
                                                                inferential analysis
               To catch the             Bounded phenomena                                   Non-interventionist
               complexity and           and systems:            Subjective
               situatedness of                                  Descriptive                 Empathic
               behaviour                                        Analytical
                                                                Understanding specific       Holistic treatment of
               To contribute to                                 situations                  phenomena
               action and
               intervention                                     Sincerity                   What can be learned
                                                                Complexity                  from the particular case
               To present and                                   Particularity
               represent reality – to
               give a sense of ‘being

     Box 3.2

       Model          Purposes                Foci                       Key terms               Characteristics
       Testing and    To measure              Academic and non-          Reliability             Materials designed to
       assessment     achievement and         academic, cognitive,       Validity                provide scores that can
                      potential               affective and              Criterion-referencing   be aggregated
                                              psychomotor                Norm-referencing
                      To diagnose strengths   domains – low-order to     Domain-referencing      Enables individuals and
                      and weaknesses          high-order                 Item-response           groups to be compared
                      To assess               Performance,               Summative               In-depth diagnosis
                      performance and         achievement, potential,    Diagnostic
                      abilities               abilities                  Standardization         Measures performance
                                              Personality                Moderation

     end, considering models of research might be                   the choice is not arbitrary (Siegel 1956; Cohen
     useful (Morrison 1993).                                        and Holliday 1996; Hopkins et al. 1996). For
                                                                    qualitative data analysis the researchers have at
     Data analysis                                                  their disposal a range of techniques, for example:
     The prepared researcher will need to consider                      coding and content analysis of field
     how the data will be analysed. This is very                        notes (Miles and Huberman 1984)
     important, as it has a specific bearing on the form                 cognitive mapping (Jones 1987; Morrison
     of the instrumentation. For example, a researcher                  1993)
     will need to plan the layout and structure of a                    seeking patterning of responses
     questionnaire survey very carefully in order to assist             looking for causal pathways and connec-
     data entry for computer reading and analysis; an                   tions (Miles and Huberman 1984)
     inappropriate layout may obstruct data entry and                   presenting cross-site analysis (Miles and
     subsequent analysis by computer. The planning of                   Huberman 1984)
     data analysis will need to consider:                               case studies
        What needs to be done with the data when                        personal constructs
        they have been collected? How will they be                      narrative accounts
        processed and analysed?                                         action research analysis
        How will the results of the analysis be verified,                analytic induction (Denzin 1970b)
        cross-checked and validated?                                    constant comparison and grounded theory
                                                                        (Glaser and Strauss 1967)
     Decisions will need to be taken with regard to                     discourse analysis (Stillar 1998)
     the statistical tests that will be used in data                    biographies and life histories (Atkinson 1998).
     analysis as this will affect the layout of research
     items (for example in a questionnaire), and the                   The criteria for deciding which forms of data
     computer packages that are available for processing            analysis to undertake are governed both by fitness
     quantitative and qualitative data, e.g. SPSS and               for purpose and legitimacy – the form of data anal-
     N-Vivo respectively. For statistical processing the            ysis must be appropriate for the kinds of data gath-
     researcher will need to ascertain the level of data            ered. For example, it would be inappropriate to use
     being processed – nominal, ordinal, interval or                certain statistics with certain kinds of numerical
     ratio (discussed in Chapter 24). Part Five addresses           data (e.g. using means on nominal data), or to use
     issues of data analysis and which statistics to use:           causal pathways on unrelated cross-site analysis.
                                                                    A PLANNING MATRIX FOR RESEARCH              87

Presenting and reporting the results                   explicit and maybe implicit purposes, whose

                                                                                                                     Chapter 3
                                                       purposes are being served by the research, and
As with the stage of planning data analysis, the
                                                       whose interests are being served by the research.
prepared researcher will need to consider the form
                                                       An example of these sub-issues and problems is
of the reporting of the research and its results,
                                                       contained in the second column.
giving due attention to the needs of different
                                                          At this point the planner is still at the divergent
audiences (for example, an academic audience may
                                                       phase of the research planning, dealing with
require different contents from a wider professional
                                                       planned possibilities (Morrison 1993: 19), opening
audience and, a fortiori, from a lay audience).
                                                       up the research to all facets and interpretations.
Decisions here will need to consider:
                                                       In the column headed ‘decisions’ the research
   how to write up and report the research             planner is moving towards a convergent phase,
   when to write up and report the research (e.g.      where planned possibilities become visible within
   ongoing or summative)                               the terms of constraints available to the researcher.
   how to present the results in tabular and/or        To do this the researcher has to move down
   written-out form                                    the column marked ‘decisions’ to see how well
   how to present the results in non-verbal forms      the decision which is taken in regard to one
   to whom to report (the necessary and possible       issue/question fits in with the decisions in regard
   audiences of the research)                          to other issues/questions. For one decision to fit
   how frequently to report.                           with another, four factors must be present:
For examples of setting out a research report,            All of the cells in the ‘decisions’ column must
see the accompanying web site (http://www.                be coherent – they must not contradict each –                   other.
Chapter 3, file 3.1.doc).                                  All of the cells in the ‘decisions’ column must
                                                          be mutually supporting.
                                                          All of the cells in the ‘decisions’ column must
A planning matrix for research                            be practicable when taken separately.
In planning a piece of research, the range of             All of the cells in the ‘decisions’ column must
questions to be addressed can be set into a matrix.       be practicable when taken together.
Box 3.3 provides such a matrix, in the left-hand       Not all of the planned possibilities might be
column of which are the questions which figure in       practicable when these four criteria are applied.
the four main areas set out so far:                    It would be of very little use if the methods of
   orienting decisions                                 data collection listed in the ‘decisions’ column of
   research design and methodology                     question 21 (‘How will the data be gathered?’)
   data analysis                                       offered little opportunity to fulfil the needs of
   presenting and reporting the results.               acquiring information to answer question 7 (‘What
                                                       must be the focus in order to answer the research
  Questions 1–10 are the orienting decisions,          questions?’), or if the methods of data collection
questions 11–22 concern the research design            were impracticable within the time scales available
and methodology, questions 23–4 cover data             in question 4.
analysis, and questions 25–30 deal with presenting        In the matrix of Box 3.3 the cells have been
and reporting the results. Within each of the          completed in a deliberately content-free way, i.e.
30 questions there are several sub-questions           the matrix as presented here does not deal with
which research planners may need to address.           the specific, actual points which might emerge in
For example, within question 5 (‘What are the          a particular research proposal. If the matrix were
purposes of the research?’) the researcher would       to be used for planning an actual piece of research,
have to differentiate major and minor purposes,        then, instead of couching the wording of each

     Box 3.3
     A matrix for planning research

       Orienting Decisions
       Question                     Sub-issues and problems                       Decisions
       1 Who wants the              Is the research going to be useful?           Find out the controls over the research
       research?                    Who might wish to use the research?           which can be exercised by respondents.
                                    Are the data going to be public?              What are the scope and audiences of the
                                    What if different people want different       research.
                                    things from the research?                     Determine the reporting mechanisms.
                                    Can people refuse to participate?
       2 Who will receive the       Will participants be able to veto the         Determine the proposed internal and
       research?                    release of parts of the research to           external audiences of the research.
                                    specified audiences?                           Determine the controls over the research
                                    Will participants be able to give the         which can be exercised by the
                                    research to whomsoever they wish?             participants.
                                    Will participants be told to whom the         Determine the rights of the participants
                                    research will go?                             and the researcher to control the release
                                                                                  of the research.
       3 What powers do the         What use will be made of the research?        Determine the rights of recipients to do
       recipients of the research   How might the research be used for or         what they wish with the research.
       have?                        against the participants?                     Determine the respondents’ rights to
                                    What might happen if the data fall into the   protection as a result of the research.
                                    ‘wrong’ hands?
                                    Will participants know in advance what
                                    use will and will not be made of the
       4 What are the time          Is there enough time to do all the            Determine the time scales and timing of
       scales of the research?      research?                                     the research.
                                    How to decide what to be done within
                                    the time scale?
       5 What are the purposes      What are the formal and hidden agendas        Determine all the possible uses of the
       of the research?             here?                                         research.
                                    Whose purposes are being served by the        Determine the powers of the respondents
                                    research?                                     to control the uses made of the research.
                                    Who decides the purposes of the               Decide on the form of reporting and the
                                    research?                                     intended and possible audiences of the
                                    How will different purposes be served in      research.
                                    the research?
       6 What are the research      Who decides what the questions will be?       Determine the participants’ rights and
       questions?                   Do participants have rights to refuse to      powers to participate in the planning,
                                    answer or take part?                          form and conduct of the research.
                                    Can participants add their own questions?     Decide the balance of all interests in the
       7 What must be the           Is sufficient time available to focus on all   Determine all the aspects of the research,
       focus in order to answer     the necessary aspects of the research?        prioritize them, and agree on the
       the research questions?      How will the priority foci be decided?        minimum necessary areas of the research.
                                    Who decides the foci?                         Determine decision-making powers on
                                                                                  the research.
                                                                               A PLANNING MATRIX FOR RESEARCH                  89

Box 3.3

                                                                                                                                    Chapter 3

  Question                    Sub-issues and problems                      Decisions
  8 What costs are            What support is available for the            Cost out the research.
  there – human, material,    researcher?
  physical, administrative,   What materials are necessary?
  9 Who owns the              Who controls the release of the report?      Determine who controls the release of
  research?                   What protections can be given to             the report.
                              participants?                                Decide the rights and powers of the
                              Will participants be identified and           researcher.
                              identifiable/traceable?                       Decide the rights of veto.
                              Who has the ultimate decision on what        Decide how to protect those who may be
                              data are included?                           identified/identifiable in the research.
  10 At what point does       Who decides the ownership of the             Determine the ownership of the research
  the ownership pass from     research?                                    at all stages of its progress.
  the respondent to the       Can participants refuse to answer certain    Decide the options available to the
  researcher and from the     parts if they wish, or, if they have the     participants.
  researcher to the           option not to take part, must they opt out   Decide the rights of different parties in
  recipients?                 of everything?                               the research, e.g. respondents,
                              Can the researcher edit out certain          researcher, recipients.
  Research design and methodology
  Question                     Sub-issues and problems                     Decisions
  11 What are the specific      How do these purposes derive from the       Decide the specific research purposes and
  purposes of the research?    overall aims of the research?               write them as concrete questions.
                               Will some areas of the broad aims be
                               covered, or will the specific research
                               purposes have to be selective?
                               What priorities are there?
  12 How are the general      Do the specific research questions            Ensure that each main research purpose is
  research purposes and       together cover all the research purposes?    translated into specific, concrete
  aims operationalized into   Are the research questions sufficiently       questions that, together, address the
  specific research            concrete as to suggest the kinds of          scope of the original research questions.
  questions?                  answers and data required and the            Ensure that the questions are sufficiently
                              appropriate instrumentation and sampling?    specific as to suggest the most
                              How to balance adequate coverage of          appropriate data types, kinds of answers
                              research purposes with the risk of           required, sampling, and instrumentation.
                              producing an unwieldy list of                Decide how to ensure that any selectivity
                              sub-questions?                               still represents the main fields of the
                                                                           research questions.
  13 What are the specific     Do the specific research questions            Ensure that the coverage and
  research questions?         demonstrate construct and content            operationalization of the specific
                              validity?                                    questions addresses content and
                                                                           construct validity respectively.
  14 What needs to be the     How many foci are necessary?                 Decide the number of foci of the research
  focus of the research in    Are the foci clearly identifiable and         questions.
  order to answer the         operationalizable?                           Ensure that the foci are clear and can be
  research questions?                                                      operationalized.

     Box 3.3

       Question                    Sub-issues and problems                         Decisions
       15 What is the main         How many methodologies are necessary?           Decide the number, type and purposes of
       methodology of the          Are several methodologies compatible            the methodologies to be used.
       research?                   with each other?                                Decide whether one or more
                                   Will a single focus/research question           methodologies is necessary to gain
                                   require more than one methodology (e.g.         answers to specific research questions.
                                   for triangulation and concurrent validity)?     Ensure that the most appropriate form of
                                                                                   methodology is employed.
       16 How will validity and    Will there be the opportunity for               Determine the process of respondent
       reliability be addressed?   cross-checking?                                 validation of the data.
                                   Will the depth and breadth required for         Decide a necessary minimum of topics to
                                   content validity be feasible within the         be covered.
                                   constraints of the research (e.g. time          Subject the plans to scrutiny by critical
                                   constraints, instrumentation)?                  friends (‘jury’ validity).
                                   In what senses are the research questions       Pilot the research.
                                   valid (e.g. construct validity)?                Build in cross-checks on data.
                                   Are the questions fair?                         Address the appropriate forms of
                                   How does the researcher know if people          reliability and validity.
                                   are telling the truth?                          Decide the questions to be asked and the
                                   What kinds of validity and reliability are to   methods used to ask them.
                                   be addressed?                                   Determine the balance of open and closed
                                   How will the researcher take back the           questions.
                                   research to respondents for them to
                                   check that the interpretations are fair and
                                   How will data be gathered consistently
                                   over time?
                                   How to ensure that each respondent is
                                   given the same opportunity to respond?
       17 How will reflexivity be   How will reflexivity be recognized?              Determine the need to address reflexivity
       addressed?                  Is reflexivity a problem?                        and to make this public.
                                   How can reflexivity be included in the           Determine how to address reflexivity in
                                   research?                                       the research.
       18 What kinds of data       Does the research need words, numbers           Determine the most appropriate types of
       are required?               or both?                                        data for the foci and research questions.
                                   Does the research need opinions, facts or       Balance objective and subjective data.
                                   both?                                           Determine the purposes of collecting
                                   Does the research seek to compare               different types of data and the ways in
                                   responses and results or simply to              which they can be processed.
                                   illuminate an issue?
       19 From whom will data      Will there be adequate time to go to all        Determine the minimum and maximum
       be acquired (i.e.           the relevant parties?                           sample.
       sampling)?                  What kind of sample is required (e.g.           Decide on the criteria for sampling.
                                   probability/non-probability/random/             Decide the kind of sample required.
                                   stratified etc.)?                                Decide the degree of representativeness
                                   How to achieve a representative sample          of the sample.
                                   (if required)?                                  Decide how to follow up and not to
                                                                                   follow up on the data gathered.
                                                                             A PLANNING MATRIX FOR RESEARCH                  91

Box 3.3

                                                                                                                                  Chapter 3

  Question                  Sub-issues and problems                      Decisions
  20 Where else will data   What documents and other written             Determine the necessary/desirable/
  be available?             sources of data can be used?                 possible documentary sources.
                            How to access and use confidential            Decide access and publication rights and
                            material?                                    protection of sensitive data.
                            What will be the positive or negative
                            effects on individuals of using certain
  21 How will the data be   What methods of data gathering are           Determine the most appropriate data
  gathered (i.e.            available and appropriate to yield data to   collection instruments to gather data to
  instrumentation)?         answer the research questions?               answer the research questions.
                            What methods of data gathering will be       Pilot the instruments and refine them
                            used?                                        subsequently.
                            How to construct interview sched-            Decide the strengths and weaknesses of
                            ules/questionnaires/tests/observation        different data collection instruments in the
                            schedules?                                   short and long term.
                            What will be the effects of observing        Decide which methods are most suitable
                            participants?                                for which issues.
                            How many methods should be used (e.g.        Decide which issues will require more
                            to ensure reliability and validity)?         than one data collection instrument.
                            Is it necessary or desirable to use more     Decide whether the same data collection
                            than one method of data collection on the    methods will be used with all the
                            same issue?                                  participants.
                            Will many methods yield more reliable
                            Will some methods be unsuitable for
                            some people or for some issues?
  22 Who will undertake     Can different people plan and carry out      Decide who will carry out the data
  the research?             different parts of the research?             collection, processing and reporting.
  Data analysis
  Question                  Sub-issues and problems                      Decisions
  23 How will the data be   Are the data to be processed numerically     Clarify the legitimate and illegitimate
  analysed?                 or verbally?                                 methods of data processing and analysis of
                            What computer packages are available to      quantitative and qualitative data.
                            assist data processing and analysis?         Decide which methods of data processing
                            What statistical tests will be needed?       and analysis are most appropriate for
                            How to perform a content analysis of         which types of data and for which
                            word data?                                   research questions.
                            How to summarize and present word            Check that the data processing and
                            data?                                        analysis will serve the research purposes.
                            How to process all the different             Determine the data protection issues if
                            responses to open-ended questions?           data are to be processed by ‘outsiders’ or
                            Will the data be presented person by         particular ‘insiders’.
                            person, issue by issue, aggregated to
                            groups, or a combination of these?
                            Does the research seek to make
                            Who will process the data?

     Box 3.3

       Question                      Sub-issues and problems                       Decisions
       24 How to verify and          What opportunities will there be for          Determine the process of respondent
       validate the data and their   respondents to check the researcher’s         validation during the research.
       interpretation?               interpretation?                               Decide the reporting of multiple
                                     At what stages of the research is             perspectives and interpretations.
                                     validation necessary?                         Decide respondents’ rights to have their
                                     What will happen if respondents disagree      views expressed or to veto reporting.
                                     with the researcher’s interpretation?
       Presenting and reporting the results
       Question                        Sub-issues and problems                     Decisions
       25 How to write up and          Who will write the report and for whom?     Ensure that the most appropriate form of
       report the research?            How detailed must the report be?            reporting is used for the audiences.
                                       What must the report contain?               Keep the report as short, clear and
                                       What channels of dissemination of the       complete as possible.
                                       research are to be used?                    Provide summaries if possible/fair.
                                                                                   Ensure that the report enables fair
                                                                                   critique and evaluation to be undertaken.
       26 When to write up and       How many times are appropriate for            Decide the most appropriate timing,
       report the research (e.g.     reporting?                                    purposes and audiences of the reporting.
       ongoing or summative)?        For whom are interim reports compiled?        Decide the status of the reporting (e.g.
                                     Which reports are public?                     formal, informal, public, private).
       27 How to present the         How to ensure that everyone will              Decide the most appropriate form of
       results in tabular and/or     understand the language or the statistics?    reporting.
       written-out form?             How to respect the confidentiality of the      Decide whether to provide a glossary of
                                     participants?                                 terms.
                                     How to report multiple perspectives?          Decide the format(s) of the reports.
                                                                                   Decide the number and timing of the
                                                                                   Decide the protection of the individual’s
                                                                                   rights, balancing this with the public’s
                                                                                   rights to know.
       28 How to present the         Will different parties require different      Decide the most appropriate form of
       results in non-verbal         reports?                                      reporting.
       forms?                        How to respect the confidentiality of the      Decide the number and timing of the
                                     participants?                                 reports.
                                     How to report multiple perspectives?          Ensure that a written record is kept of
                                                                                   oral reports.
                                                                                   Decide the protection of the individual’s
                                                                                   rights, balancing this with the public’s
                                                                                   rights to know.
       29 To whom to report          Do all participants receive a report?         Identify the stakeholders.
       (the necessary and            What will be the effects of not reporting     Determine the least and most material to
       possible audiences of the     to stakeholders?                              be made available to the stakeholders.
       30 How frequently to          Is it necessary to provide interim reports?   Decide on the timing and frequency of the
       report?                       If interim reports are provided, how might    reporting.
                                     this affect the future reports or the         Determine the formative and summative
                                     course of the research?                       nature of the reports.
                                                               MANAGING THE PLANNING OF RESEARCH               93

cell in generalized terms, it would be more useful      the research and with which sectors of the sample

                                                                                                                    Chapter 3
if specific, concrete responses were given which         population. Box 3.5 sets out a matrix of these
addressed particular issues and concerns in the         for planning (see also Morrison 1993: 109), for
research proposal in question.                          example, of a small-scale piece of research.
   Many of these questions concern rights,                 A matrix approach such as this enables research
responsibilities and the political uses (and abuses)    planners to see at a glance their coverage of the
of the research. This underlines the view that          sample and of the instruments used at particular
research is an inherently political and moral           points in time, making omissions clear, and
activity; it is not politically or morally neutral.     promoting such questions as the following:
The researcher has to be concerned with the uses
as well as the conduct of the research.                    Why are certain instruments used at certain
                                                           times and not at others?
                                                           Why are certain instruments used with certain
Managing the planning of research                          people and not with others?
The preceding discussion has revealed the                  Why do certain times in the research use more
complexity of planning a piece of research, yet            instruments than other times?
it should not be assumed that research will always         Why is there such a heavy concentration of
go according to plan! For example, the mortality of        instruments at the end of the study?
the sample might be a feature (participants leaving        Why are certain groups involved in more
during the research), or a poor response rate to           instruments than other groups?
questionnaires might be encountered, rendering             Why are some groups apparently neglected
subsequent analysis, reporting and generalization          (e.g. parents): is there a political dimension to
problematical; administrative support might not            the research?
be forthcoming, or there might be serious slippage         Why are questionnaires the main kinds of
in the timing. This is not to say that a plan              instrument to be used?
for the research should not be made; rather it             Why are some instruments (e.g. observation,
is to suggest that it is dangerous to put absolute         testing) not used at all?
faith in it! For an example of what to include             What makes the five stages separate?
in a research proposal see the accompanying                Are documents held only by certain parties
web site: (             (and, if so, might one suspect an ‘institutional
9780415368780 – Chapter 3, file 3.2.doc).                   line’ to be revealed in them)?
   To manage the complexity in planning outlined           Are some parties more difficult to contact than
above a simple four-stage model can be proposed:           others (e.g. university teacher educators)?
                                                           Are some parties more important to the
1   Identify the purposes of the research.                 research than others (e.g. the principals)?
2   Identify and give priority to the constraints          Why are some parties excluded from the sample
    under which the research will take place.              (e.g. school governors, policy-makers, teachers’
3   Plan the possibilities for the research within         associations and unions)?
    these constraints.                                     What is the difference between the three
4   Decide the research design.                            groups of teachers?
Each stage contains several operations (see http:          Matrix planning is useful for exposing key
//                          features of the planning of research. Further
9780415368780 – Chapter 3, file 3.4. ppt). Box 3.4       matrices might be constructed to indicate other
clarifies this four-stage model, drawing out the         features of the research, for example:
various operations contained in each stage.
   It may be useful for research planners to consider      the timing of the identification of the sample
which instruments will be used at which stage of           the timing of the release of interim reports

     Box 3.4
     A planning sequence for research

                      Stage 1
                                               What are the purposes of the research?
                      Identify the purposes
                      of the research

                                               Who wants the research?
                                               Who will receive the research?
                                               What powers do the recipients of the research have?
                      Stage 2                  What are the time scales of the research?
                                               What costs are there – human, physical, material,
                      Identify and give        administrative, temporal?
                      priority to the          Who owns the research?
                      constraints under        At what point does the ownership pass from the
                      which the research       respondent to the researcher and from the
                      will take place          researcher to the recipients?
                                               What are the powers of the researcher?
                                               What are the main foci of the research?
                                               What are the ethics of the research?

                                               What are the specific purposes of the research?
                                               What are the research questions?
                                               What needs to be the focus of the research in order
                                               to answer the research questions?
                                               What is the main methodology of the research?
                                               How will validity and reliability be addressed?
                                               How will reflexivity be addressed?
                      Stage 3                  What kinds of data are required?
                                               From whom will data be acquired (sampling)?
                      Plan the possibilities   Where else will data be available?
                      for the research         How will the data be gathered (instrumentation)?
                      within these             Who will undertake the research?
                      constraints              How will the data be processed and analysed?
                                               How to verify and validate the data and their
                                               How to write up and report the research?
                                               How to present the results in written and non-verbal
                                               To whom to report?
                                               When to report?
                      Stage 4

                      Decide the research      Achieving coherence and practicability in the design.
                                                                                                A WORKED EXAMPLE           95

Box 3.5

                                                                                                                                Chapter 3
A planning matrix for research

  Time sample       Stage 1 (start)   Stage 2 (3 months)   Stage 3 (6 months)   Stage 4 (9 months)   Stage 5 (12 months)
  Principal/        Documents         Interview            Documents            Interview            Documents
  headteacher       Interview                              Questionnaire 2                           Interview
                    Questionnaire 1                                                                  Questionnaire 3
  Teacher group 1   Questionnaire 1                        Questionnaire 2                           Questionnaire 3
  Teacher group 2   Questionnaire 1                        Questionnaire 2                           Questionnaire 3
  Teacher group 3   Questionnaire 1                        Questionnaire 2                           Questionnaire 3
  Students                                                 Questionnaire 2                           Interview
  Parents           Questionnaire 1                        Questionnaire 2                           Questionnaire 3
  University        Interview                                                                        Interview
  teacher           Documents                                                                        Documents

   the timing of the release of the final report                  To provide an indication of the strength of the
   the timing of pretests and post-tests (in an                  organizational culture(s).
   experimental style of research)                               To make suggestions and recommendations
   the timing of intensive necessary resource                    about the organizational culture of, and its
   support (e.g. reprographics)                                  development at, the school.
   the timing of meetings of interested parties.
These examples cover timings only; other matrices            Research questions
might be developed to cover other combinations,
for example: reporting by audiences; research                    What are the major and minor elements of
team meetings by reporting; instrumentation by                   organizational culture in the school?
participants etc. They are useful summary devices.               What are the organizational cultures and
                                                                 subcultures in the school?
A worked example                                                 Which (sub)cultures are the most and least
                                                                 prevalent in the school, and in which parts of
Let us say that a school is experiencing very                    the school are these most and least prevalent?
low morale and a researcher has been brought in                  How strong and intense are the (sub)cultures
to investigate the school’s organizational culture.              in the school?
The researcher has been given open access to                     What are the causes and effects of the
the school and has five months from the start                     (sub)cultures in the school?
of the project to producing the report (for                      How can the (sub)cultures be improved in the
a fuller version of this see the accompanying                    school?
web site,
9780415368780 – Chapter 3, file 3.3.doc). The
researcher plans the research as follows:
Purposes                                                     Three levels of organizational cultures will be
   To present an overall and in-depth picture of
   the organizational culture(s) and subcultures,                underlying values and assumptions
   including the prevailing cultures and subcul-                 espoused values and enacted behaviours
   tures, within the school.                                     artefacts.

     Organizational culture concerns values, assump-                superficial information about the school’s culture.
     tions, beliefs, espoused theories and mental mod-              In order to probe beneath the surface of the
     els, observed practices, areas of conflict and con-             school’s culture, to examine the less overt aspects
     sensus, the formal and hidden messages contained               of the school’s culture(s) and subcultures, it is
     in artefacts, messages, documents and language,                important to combine quantitative and qualitative
     the ‘way we do things’, the physical environ-                  methodologies for data collection. A mixed
     ment, relationships, power, control, communica-                methodology will be used for the empirical data
     tion, customs and rituals, stories, the reward system          collection, using numerical and verbal data, in
     and motivation, the micro-politics of the school,              order to gather rounded, reliable data. A survey
     involvement in decision-making, empowerment                    approach will be used to gain an overall picture,
     and exploitation/manipulation, leadership, com-                and a more fine-grained analysis will be achieved
     mitment, and so on.                                            through individual and group interviews and focus
                                                                    groups (Box 3.6).
     Organizational culture is intangible yet its impact
     on a school’s operations is very tangible. This                The data gathered will be largely perception based,
     suggests that, while quantitative measures may be              and will involve gathering employees’ views of the
     used, they are likely only to yield comparatively              (sub)cultures. As the concept of organizational

     Box 3.6
     Understanding the levels of organizational culture

              Levels of                                   Easy to                                    Non-participant
               culture          Instruments               uncover        Tangible      Superficial     observer

             Artefacts         Observational


                              Qualitative data

              Enacted             Survey
               values          questionnaires
            (behaviours)       and numerical
                              Qualitative data

             Underlying        Qualitative and
            assumptions         ethnographic

                             Interviews (group
                               and individual)

                                                          Hard to       Intangible       Deep          Participant
                                                          uncover                                       observer

                                                                                      A WORKED EXAMPLE          97

culture is derived, in part from ethnography and       aspects of the school’s culture, e.g. values, assump-

                                                                                                                     Chapter 3
anthropology, the research will use qualitative and    tions, beliefs, wishes, problems. Interviews will
ethnographic methods.                                  be semi-structured, i.e. with a given agenda and
   One of the difficulties anticipated is that the      open-ended questions. As face-to-face individual
less tangible aspects of the school might be the       interviews might be intimidating for some groups,
most difficult on which to collect data. Not            group interviews will be used. In all of the inter-
only will people find it harder to articulate           views the important part will be the supplementary
responses and constructs, but also they may be         question ‘why’.
reluctant to reveal these in public. The more
the project addresses intangible and unmeasurable
                                                       Observational data
elements, and the richer the data that are to be
collected, the more there is a need for increased      Observational data will comment on the physical
and sensitive interpersonal behaviour, face-to-face    environment, and will then be followed up
data collection methods and qualitative data.          with interview material to discover participants’
   There are several instruments for data collec-      responses to, perceptions of, messages contained
tion: questionnaires, semi-structured interviews       in and attitudes to the physical environment.
(individual and group), observational data and         Artefacts, clothing, shared and private spaces,
documentary data will constitute a necessary min-      furniture, notices, regulations etc. all give messages
imum, as follows:                                      to participants.

Questionnaires                                         Documentary data
Questionnaire surveys, use commercially available      Documentary analysis and additional stored data,
instruments, each of which measures different          reporting the formal matters in the school, will
aspects of school’s culture, in particular:            be examined for what they include and what they
   The Organizational Culture Questionnaire
   (Harrison and Stokes 1992) looks at overall
   cultures and provides a general picture in terms    Sampling
   of role, power, achievement and support cultures,
                                                       First, the questionnaire will be given to all
   and examines the differences between existing
                                                       employees who are willing to participate. Second,
   and preferred cultures.
                                                       the semi-structured interviews will be conducted
   The Organizational Culture Inventory (Cooke
                                                       on a ‘critical case’ basis, i.e. with participants who
   and Lafferty 1989) provides a comprehensive
                                                       are in key positions and who are ‘knowledgeable
   and reliable analysis of the presenting
                                                       people’ about the activities and operations of the
   organizational cultures.
Questionnaires, using rating scales, will catch           There will be stratified sampling for the
articulated, espoused, enacted, visible aspects        survey instruments, in order to examine how
of organizational culture, and will measure, for       perceptions of the school’s organizational culture
example, the extent of sharedness of culture,          vary according to the characteristics of the
congruence between existing and ideal, strength        subsamples. This will enable the levels of
and intensity of culture.                              congruence or disjunction between the responses
                                                       of the various subgroups to be charted. Nominal
                                                       characteristics of the sampling will be included,
Semi-structured interviews
                                                       for example, age, level in the school, departments,
Semi-structured qualitative interviews for individ-    sex, ethnicity, nationality, years of working in the
uals and groups gather data on the more intangible     school.

     Parameters                                             Stage 5: Reporting
     The data will be collected on a ‘one-shot’             A full report on the findings will include
     basis rather than longitudinally. A multi-method       conclusions, implications and recommendations.
     approach will be used for data collection.

                                                            Ethics and ownership
     Stages in the research
                                                            Participation in the project will be on the basis
     There are five stages in the research, as follows.      of informed consent, and on a voluntary basis,
                                                            with rights of withdrawal at any time. Given
     Stage 1: Development and operationalization            the size and scope of the cultural survey, it is
                                                            likely that key people in the school will be
     This stage includes:
                                                            able to be identified, even though the report
        a review of literature and commercially             is confidential. This will be made clear to the
        produced instruments                                potential participants. Copies of the report will be
        clarification of the research questions              available for all the employees. Data, once given to
        clarification of methodology and sampling.           the researcher, are his or hers, and the researcher
                                                            may not use them in any way which will publicly
     Stage 2: Instrumentation and the piloting of the       identify the school; the report is the property of
     instruments                                            the school.

     This stage includes:
                                                            Time frames
        questionnaire development and piloting
        semi-structured interview schedules and pilot-      The project will be completed in five months:
                                                               the first month for a review of the relevant
        gathering of observational data
        analysis of documentary data.
                                                               the second month to develop the instrumenta-
     Because of the limited number of senior staff, it         tion and research design
     will not be possible to conduct pilot interviews          the third month to gather the data
     with them, as this will preclude them from the            the fourth month to analyse the data
     final data collection.                                     the fifth month to complete the report.
                                                            The example indicates a systematic approach to
     Stage 3: Data collection                               the planning and conduct of the research, which
     This will proceed in the following sequence.           springs from a perceived need in the school. It
     First, administration of the questionnaire will        works within given constraints and makes clear
     be followed by analysis of questionnaire data to       what it will ‘deliver’. Though the research does
     provide material for the interviews. Interviews        not specify hypotheses to be tested, nevertheless
     will be conducted concurrently.                        it would not be difficult to convert the research
                                                            questions into hypotheses if this style of research
                                                            were preferred.
     Stage 4: Data analysis and interpretation
     Numerical data will be analysed using SPSS, which
     will also enable the responses from subgroups of
     the school to be separated for analysis. Qualitative   The notion of ‘fitness for purpose’ reins in
     data will be analysed using protocols of content       planning research; the research plan must suit
     analysis.                                              the purposes of the research. If the reader is
                                                                                       CONCLUSION      99

left feeling, at the end of this chapter, that       to be worthwhile and effective. For a checklist

                                                                                                            Chapter 3
the task of research is complex, then that is        for evaluating research see the accompanying
an important message, for rigour and thoughtful,     web site (
thorough planning are necessary if the research is   9780415368780 – Chapter 3, file 3.4.doc).
4        Sampling

Introduction                                           students (the total population being all the cases).
                                                       Therefore she has to be selective and to interview
The quality of a piece of research stands or falls
                                                       fewer than all 900 students. How will she decide
not only by the appropriateness of methodology
                                                       that selection; how will she select which students
and instrumentation but also by the suitability
                                                       to interview?
of the sampling strategy that has been adopted
                                                          If she were to interview 200 of the students,
(see also Morrison 1993: 112–17). Questions of
                                                       would that be too many? If she were to interview
sampling arise directly out of the issue of defining
                                                       just 20 of the students would that be too few? If she
the population on which the research will focus.
                                                       were to interview just the males or just the females,
Researchers must take sampling decisions early in
                                                       would that give her a fair picture? If she were to
the overall planning of a piece of research. Factors
                                                       interview just those students whom the science
such as expense, time, accessibility frequently
                                                       teachers had decided were ‘good at science’, would
prevent researchers from gaining information from
                                                       that yield a true picture of the total population of
the whole population. Therefore they often need
                                                       900 students? Perhaps it would be better for her to
to be able to obtain data from a smaller group or
                                                       interview those students who were experiencing
subset of the total population in such a way that
                                                       difficulty in science and who did not enjoy science,
the knowledge gained is representative of the total
                                                       as well as those who were ‘good at science’. Suppose
population (however defined) under study. This
                                                       that she turns up on the days of the interviews only
smaller group or subset is the sample. Experienced
                                                       to find that those students who do not enjoy sci-
researchers start with the total population and
                                                       ence have decided to absent themselves from the
work down to the sample. By contrast, less
                                                       science lesson. How can she reach those students?
experienced researchers often work from the
                                                          Decisions and problems such as these face
bottom up, that is, they determine the minimum
                                                       researchers in deciding the sampling strategy to
number of respondents needed to conduct the
                                                       be used. Judgements have to be made about four
research (Bailey 1978). However, unless they
                                                       key factors in sampling:
identify the total population in advance, it
is virtually impossible for them to assess how            the sample size
representative the sample is that they have drawn.        representativeness and parameters of the
   Suppose that a class teacher has been released         sample
from her teaching commitments for one month in            access to the sample
order to conduct some research into the abilities of      the sampling strategy to be used.
13-year-old students to undertake a set of science
experiments; that the research is to draw on              The decisions here will determine the sam-
three secondary schools which contain 300 such         pling strategy to be used (see http://www.routledge.
students each, a total of 900 students, and that       com/textbooks/9780415368780 – Chapter 4, file
the method that the teacher has been asked to use      4.1.ppt). This assumes that a sample is actu-
for data collection is a semi-structured interview.    ally required; there may be occasions on which
Because of the time available to the teacher it        the researcher can access the whole population
would be impossible for her to interview all 900       rather than a sample.
                                                                                               THE SAMPLE SIZE       101

The sample size                                              a commonly used test (discussed in Part Five)

                                                                                                                       Chapter 4
                                                             with cross-tabulated data, for example looking at
A question that often plagues novice researchers is
                                                             two subgroups of stakeholders in a primary school
just how large their samples for the research should
                                                             containing sixty 10-year-old pupils and twenty
be. There is no clear-cut answer, for the correct
                                                             teachers and their responses to a question on a
sample size depends on the purpose of the study
                                                             5-point scale (see diagram below).
and the nature of the population under scrutiny.
                                                                 Here one can notice that the sample size
However, it is possible to give some advice on this
                                                             is eighty cases, an apparently reasonably sized
matter. Generally speaking, the larger the sample
                                                             sample. However, six of the ten cells of responses
the better, as this not only gives greater reliability
                                                             (60 per cent) contain fewer than five cases.
but also enables more sophisticated statistics to be
                                                             The chi-square statistic requires there to be five
                                                             cases or more in 80 per cent of the cells (i.e.
   Thus, a sample size of thirty is held by many
                                                             eight out of the ten cells). In this example only
to be the minimum number of cases if researchers
                                                             40 per cent of the cells contained more than
plan to use some form of statistical analysis on
                                                             five cases, so even with a comparatively large
their data, though this is a very small number
                                                             sample, the statistical requirements for reliable
and we would advise very considerably more.
                                                             data with a straightforward statistic such as chi-
Researchers need to think out in advance of any
                                                             square have not been met. The message is clear,
data collection the sorts of relationships that they
                                                             one needs to anticipate, as far as one is able,
wish to explore within subgroups of their eventual
                                                             some possible distributions of the data and see if
sample. The number of variables researchers set
                                                             these will prevent appropriate statistical analysis;
out to control in their analysis and the types
                                                             if the distributions look unlikely to enable reliable
of statistical tests that they wish to make must
                                                             statistics to be calculated then one should increase
inform their decisions about sample size prior
                                                             the sample size, or exercise great caution in
to the actual research undertaking. Typically an
                                                             interpreting the data because of problems of
anticipated minimum of thirty cases per variable
                                                             reliability, or not use particular statistics, or,
should be used as a ‘rule of thumb’, i.e. one must
                                                             indeed, consider abandoning the exercise if the
be assured of having a minimum of thirty cases
                                                             increase in sample size cannot be achieved.
for each variable (of course, the thirty cases for
                                                                 The point here is that each variable may need
variable one could also be the same thirty as for
                                                             to be ensured of a reasonably large sample size (a
variable two), though this is a very low estimate
                                                             minimum of maybe six–ten cases). Indeed Gorard
indeed. This number rises rapidly if different
                                                             (2003: 63) suggests that one can start from the
subgroups of the population are included in the
                                                             minimum number of cases required in each cell,
sample (discussed below), which is frequently the
                                                             multiply this by the number of cells, and then
                                                             double the total. In the example above, with six
   Further, depending on the kind of analysis to
                                                             cases in each cell, the minimum sample would be
be performed, some statistical tests will require
                                                             120 (6 × 10 × 2), though, to be on the safe side,
larger samples. For example, less us imagine that
                                                             to try to ensure ten cases in each cell, a minimum
one wished to calculate the chi-square statistic,

 Variable: 10-year-old pupils should do one hour’s homework each weekday evening

                                   Strongly disagree     Disagree   Neither agree   Agree   Strongly agree
                                                                    nor disagree

10-year-old pupils in the school          25               20            3            8            4
Teachers in the school                     6                4            2            4            4

      sample of 200 might be better (10 × 10 × 2),           population involved, there is sampling error,
      though even this is no guarantee.                      discussed below.
         The issue arising out of the example here is           Sample size is also determined to some extent
      also that one can observe considerable variation       by the style of the research. For example, a survey
      in the responses from the participants in the          style usually requires a large sample, particularly if
      research. Gorard (2003: 62) suggests that if a         inferential statistics are to be calculated. In ethno-
      phenomenon contains a lot of potential variability     graphic or qualitative research it is more likely that
      then this will increase the sample size. Surveying     the sample size will be small. Sample size might
      a variable such as intelligence quotient (IQ) for      also be constrained by cost – in terms of time,
      example, with a potential range from 70 to around      money, stress, administrative support, the number
      150, may require a larger sample rather than a         of researchers, and resources. Borg and Gall (1979:
      smaller sample.                                        194–5) suggest that correlational research requires
         As well as the requirement of a minimum             a sample size of no fewer than thirty cases, that
      number of cases in order to examine relationships      causal-comparative and experimental methodolo-
      between subgroups, researchers must obtain the         gies require a sample size of no fewer than fifteen
      minimum sample size that will accurately represent     cases, and that survey research should have no
      the population being targeted. With respect to size,   fewer than 100 cases in each major subgroup and
      will a large sample guarantee representativeness?      twenty–fifty in each minor subgroup.
      Not necessarily! In our first example, the                 Borg and Gall (1979: 186) advise that sample
      researcher could have interviewed a total sample       size has to begin with an estimation of the smallest
      of 450 females and still not have represented          number of cases in the smallest subgroup of the
      the male population. Will a small size guarantee       sample, and ‘work up’ from that, rather than vice
      representativeness? Again, not necessarily! The        versa. So, for example, if 5 per cent of the sample
      latter falls into the trap of saying that 50 per       must be teenage boys, and this subsample must be
      cent of those who expressed an opinion said that       thirty cases (e.g. for correlational research), then
      they enjoyed science, when the 50 per cent was         the total sample will be 30 ÷ 0.05 = 600; if 15 per
      only one student, a researcher having interviewed      cent of the sample must be teenage girls and the
      only two students in all. Furthermore, too large       subsample must be forty-five cases, then the total
      a sample might become unwieldy and too small           sample must be 45 ÷ 0.15 = 300 cases.
      a sample might be unrepresentative (e.g. in the           The size of a probability (random) sample can be
      first example, the researcher might have wished to      determined in two ways, either by the researcher
      interview 450 students but this would have been        exercising prudence and ensuring that the sample
      unworkable in practice, or the researcher might        represents the wider features of the population with
      have interviewed only ten students, which, in all      the minimum number of cases or by using a table
      likelihood, would have been unrepresentative of        which, from a mathematical formula, indicates the
      the total population of 900 students).                 appropriate size of a random sample for a given
         Where simple random sampling is used, the           number of the wider population (Morrison 1993:
      sample size needed to reflect the population value      117). One such example is provided by Krejcie
      of a particular variable depends both on the size of   and Morgan (1970), whose work suggests that if
      the population and the amount of heterogeneity         the researcher were devising a sample from a wider
      in the population (Bailey 1978). Generally, for        population of thirty or fewer (e.g. a class of students
      populations of equal heterogeneity, the larger the     or a group of young children in a class) then she
      population, the larger the sample that must be         or he would be well advised to include the whole
      drawn. For populations of equal size, the greater      of the wider population as the sample.
      the heterogeneity on a particular variable, the           Krejcie and Morgan (1970) indicate that the
      larger the sample that is needed. To the extent        smaller the number of cases there are in the wider,
      that a sample fails to represent accurately the        whole population, the larger the proportion of
                                                                                          THE SAMPLE SIZE       103

that population must be which appears in the            less stringent confidence level (say 90 per cent of

                                                                                                                  Chapter 4
sample. The converse of this is true: the larger        the time), then the sample size will be smaller.
the number of cases there are in the wider,             Usually a compromise is reached, and researchers
whole population, the smaller the proportion            opt for a 95 per cent confidence level. Similarly,
of that population can be which appears in the          if we want a very small confidence interval (i.e. a
sample (see         limited range of variation, e.g. 3 per cent) then the
9780415368780 – Chapter 4, file 4.2.ppt). They           sample size will be high, and if we are comfortable
note that as the population increases the propor-       with a larger degree of variation (e.g. 5 per cent)
tion of the population required in the sample di-       then the sample size will be lower.
minishes and, indeed, remains constant at around           A full table of sample sizes for a probability
384 cases (Krejcie and Morgan 1970: 610). Hence,        sample is given in Box 4.1, with three confidence
for example, a piece of research involving all the      levels (90 per cent, 95 per cent and 99 per cent)
children in a small primary or elementary school        and three confidence intervals (5 per cent, 4 per
(up to 100 students in all) might require between       cent and 3 per cent).
80 per cent and 100 per cent of the school to be           We can see that the size of the sample reduces at
included in the sample, while a large secondary         an increasing rate as the population size increases;
school of 1,200 students might require a sample         generally (but, clearly, not always) the larger
of 25 per cent of the school in order to achieve        the population, the smaller the proportion of
randomness. As a rough guide in a random sample,        the probability sample can be. Also, the higher
the larger the sample, the greater is its chance of     the confidence level, the greater the sample, and
being representative.                                   the lower the confidence interval, the higher the
   In determining sample size for a probability         sample. A conventional sampling strategy will be
sample one has to consider not only the population      to use a 95 per cent confidence level and a 3 per
size but also the confidence level and confidence         cent confidence interval.
interval, two further pieces of terminology. The           There are several web sites that offer sample
confidence level, usually expressed as a percentage      size calculation services for random samples. One
(usually 95 per cent or 99 per cent), is an             free site at the time of writing is from Cre-
index of how sure we can be (95 per cent                ative Service Systems (http://www.surveysystem.
of the time or 99 per cent of the time)                 com/sscalc.htm), and another is from Pear-
that the responses lie within a given variation         son NCS (
range, a given confidence interval (e.g. ±3 per          sample-calc.htm), in which the researcher inputs
cent) (see          the desired confidence level, confidence interval
9780415368780 – Chapter 4, file 4.3.ppt). The            and the population size, and the sample size is
confidence interval is that degree of variation or       automatically calculated.
variation range (e.g. ±1 per cent, or ±2 per cent,         If different subgroups or strata (discussed below)
or ±3 per cent) that one wishes to ensure. For          are to be used then the requirements placed on
example, the confidence interval in many opinion         the total sample also apply to each subgroup. For
polls is ±3 per cent; this means that, if a voting      example, let us imagine that we are surveying
survey indicates that a political party has 52 per      a whole school of 1,000 students in a multi-
cent of the votes then it could be as low as 49 per     ethnic school. The formulae above suggest that
cent (52 − 3) or as high as 55 per cent (52 + 3).       we need 278 students in our random sample, to
A confidence level of 95 per cent here would             ensure representativeness. However, let us imagine
indicate that we could be sure of this result within    that we wished to stratify our groups into, for
this range (±3 per cent) for 95 per cent of the time.   example, Chinese (100 students), Spanish (50
   If we want to have a very high confidence level       students), English (800 students) and American
(say 99 per cent of the time) then the sample size      (50 students). From tables of random sample sizes
will be high. On the other hand, if we want a           we work out a random sample.

      Box 4.1
      Sample size, confidence levels and confidence intervals for random samples

        Population    Confidence level 90 per cent   Confidence level 95 per cent   Confidence level 99 per cent
                      Confi-    Confi-       Confi-    Confi-    Confi-       Confi-    Confi-    Confi-        Confi-
                      dence    dence       dence    dence    dence       dence    dence    dence        dence
                30     27         28         29       28        29          29      29         29          30
                50     42         45         47       44        46          48      46         48          49
                75     59         64         68       63        67          70      67         70          72
               100     73         81         88       79        86          91      87         91          95
               120     83         94        104       91       100         108     102        108         113
               150     97        111        125      108       120         132     122        131         139
               200    115        136        158      132       150         168     154        168         180
               250    130        157        188      151       176         203     182        201         220
               300    143        176        215      168       200         234     207        233         258
               350    153        192        239      183       221         264     229        262         294
               400    162        206        262      196       240         291     250        289         329
               450    170        219        282      207       257         317     268        314         362
               500    176        230        301      217       273         340     285        337         393
               600    187        249        335      234       300         384     315        380         453
               650    192        257        350      241       312         404     328        400         481
               700    196        265        364      248       323         423     341        418         507
               800    203        278        389      260       343         457     363        452         558
               900    209        289        411      269       360         468     382        482         605
             1,000    214        298        431      278       375         516     399        509         648
             1,100    218        307        448      285       388         542     414        534         689
             1,200    222        314        464      291       400         565     427        556         727
             1,300    225        321        478      297       411         586     439        577         762
             1,400    228        326        491      301       420         606     450        596         796
             1,500    230        331        503      306       429         624     460        613         827
             2,000    240        351        549      322       462         696     498        683         959
             2,500    246        364        581      333       484         749     524        733       1,061
             5,000    258        392        657      357       536         879     586        859       1,347
             7,500    263        403        687      365       556         934     610        911       1,480
            10,000    265        408        703      370       566         964     622        939       1,556
            20,000    269        417        729      377       583       1,013     642        986       1,688
            30,000    270        419        738      379       588       1,030     649     1,002        1,737
            40,000    270        421        742      381       591       1,039     653     1,011        1,762
            50,000    271        422        745      381       593       1,045     655     1,016        1,778
           100,000    272        424        751      383       597       1,056     659     1,026        1,810
           150,000    272        424        752      383       598       1,060     661     1,030        1,821
           200,000    272        424        753      383       598       1,061     661     1,031        1,826
           250,000    272        425        754      384       599       1,063     662     1,033        1,830
           500,000    272        425        755      384       600       1,065     663     1,035        1,837
         1,000,000    272        425        756      384       600       1,066     663     1,036        1,840

                     Population   Sample                             Our original sample size of 278 has now
       Chinese         100         80                             increased, very quickly, to 428. The message is very
       Spanish          50         44                             clear: the greater the number of strata (subgroups),
       English         800        260                             the larger the sample will be. Much educational
       American         50         44                             research concerns itself with strata rather than
       Total         1,000        428                             whole samples, so the issue is significant. One can
                                                                                         THE SAMPLE SIZE       105

rapidly generate the need for a very large sample.     an influence on the sample size. For nominal data

                                                                                                                 Chapter 4
If subgroups are required then the same rules for      the sample sizes may well have to be larger than
calculating overall sample size apply to each of the   for interval and ratio data (i.e. a variant of the
subgroups.                                             issue of the number of subgroups to be addressed,
   Further, determining the size of the sample         the greater the number of subgroups or possible
will also have to take account of non-response,        categories, the larger the sample will have to be).
attrition and respondent mortality, i.e. some             Borg and Gall (1979) set out a formula-
participants will fail to return questionnaires,       driven approach to determining sample size (see
leave the research, return incomplete or spoiled       also Moser and Kalton 1977; Ross and Rust 1997:
questionnaires (e.g. missing out items, putting        427–38), and they also suggest using correlational
two ticks in a row of choices instead of only          tables for correlational studies – available in most
one). Hence it is advisable to overestimate rather     texts on statistics – as it were ‘in reverse’ to
than to underestimate the size of the sample           determine sample size (Borg and Gall 1979:
required, to build in redundancy (Gorard 2003:         201), i.e. looking at the significance levels of
60). Unless one has guarantees of access, response     correlation coefficients and then reading off the
and, perhaps, the researcher’s own presence at         sample sizes usually required to demonstrate that
the time of conducting the research (e.g. presence     level of significance. For example, a correlational
when questionnaires are being completed), then         significance level of 0.01 would require a sample
it might be advisable to estimate up to double the     size of 10 if the estimated coefficient of correlation
size of required sample in order to allow for such     is 0.65, or a sample size of 20 if the estimated
loss of clean and complete copies of questionnaires    correlation coefficient is 0.45, and a sample size of
or responses.                                          100 if the estimated correlation coefficient is 0.20.
   In some circumstances, meeting the require-         Again, an inverse proportion can be seen – the
ments of sample size can be done on an evolu-          larger the sample population, the smaller the
tionary basis. For example, let us imagine that you    estimated correlation coefficient can be to be
wish to sample 300 teachers, randomly selected.        deemed significant.
You succeed in gaining positive responses from            With both qualitative and quantitative data,
250 teachers to, for example, a telephone survey       the essential requirement is that the sample
or a questionnaire survey, but you are 50 short of     is representative of the population from which
the required number. The matter can be resolved        it is drawn. In a dissertation concerned with
simply by adding another 50 to the random sam-         a life history (i.e. n = 1), the sample is the
ple, and, if not all of these are successful, then     population!
adding some more until the required number is
                                                       Qualitative data
   Borg and Gall (1979: 195) suggest that, as a
general rule, sample sizes should be large where       In a qualitative study of thirty highly able girls
                                                       of similar socio-economic background following
   there are many variables
                                                       an A level Biology course, a sample of five or
   only small differences or small relationships are
                                                       six may suffice the researcher who is prepared to
   expected or predicted
                                                       obtain additional corroborative data by way of
   the sample will be broken down into subgroups
   the sample is heterogeneous in terms of the
                                                          Where there is heterogeneity in the popula-
   variables under study
                                                       tion, then a larger sample must be selected on
   reliable measures of the dependent variable are
                                                       some basis that respects that heterogeneity. Thus,
                                                       from a staff of sixty secondary school teachers
  Oppenheim (1992: 44) adds to this the view           differentiated by gender, age, subject specialism,
that the nature of the scales to be used also exerts   management or classroom responsibility, etc., it

      would be insufficient to construct a sample con-                 It is clear that sample size is a matter of
      sisting of ten female classroom teachers of Arts             judgement as well as mathematical precision; even
      and Humanities subjects.                                     formula-driven approaches make it clear that there
                                                                   are elements of prediction, standard error and
      Quantitative data                                            human judgement involved in determining sample
      For quantitative data, a precise sample number
      can be calculated according to the level of accuracy
      and the level of probability that researchers require        Sampling error
      in their work. They can then report in their                 If many samples are taken from the same
      study the rationale and the basis of their research          population, it is unlikely that they will all have
      decisions (Blalock 1979).                                    characteristics identical with each other or with
         By way of example, suppose a teacher/researcher           the population; their means will be different. In
      wishes to sample opinions among 1,000 secondary              brief, there will be sampling error (see Cohen
      school students. She intends to use a 10-point               and Holliday 1979, 1996). Sampling error is often
      scale ranging from 1 = totally unsatisfactory to             taken to be the difference between the sample
      10 = absolutely fabulous. She already has data               mean and the population mean. Sampling error
      from her own class of thirty students and suspects           is not necessarily the result of mistakes made
      that the responses of other students will be                 in sampling procedures. Rather, variations may
      broadly similar. Her own students rated the                  occur due to the chance selection of different
      activity (an extracurricular event) as follows: mean         individuals. For example, if we take a large
      score = 7.27; standard deviation = 1.98. In other            number of samples from the population and
      words, her students were pretty much ‘bunched’               measure the mean value of each sample, then
      about a warm, positive appraisal on the 10-point             the sample means will not be identical. Some
      scale. How many of the 1,000 students does she               will be relatively high, some relatively low, and
      need to sample in order to gain an accurate (i.e.            many will cluster around an average or mean value
      reliable) assessment of what the whole school                of the samples. We show this diagrammatically in
      (n = 1, 000) thinks of the extracurricular event?            Box 4.2 (see
        It all depends on what degree of accuracy and what level   9780415368780 – Chapter 4, file 4.4.ppt).
        of probability she is willing to accept.                      Why should this occur? We can explain the
                                                                   phenomenon by reference to the Central Limit
        A simple calculation from a formula by Blalock             Theorem which is derived from the laws of
      (1979: 215–18) shows that:                                   probability. This states that if random large
         if she is happy to be within + or − 0.5 of a scale        samples of equal size are repeatedly drawn from
         point and accurate 19 times out of 20, then she           any population, then the mean of those samples
         requires a sample of 60 out of the 1,000;                 will be approximately normally distributed. The
         if she is happy to be within + or − 0.5 of a              distribution of sample means approaches the
         scale point and accurate 99 times out of 100,             normal distribution as the size of the sample
         then she requires a sample of 104 out of the              increases, regardless of the shape – normal or
         1,000                                                     otherwise – of the parent population (Hopkins
         if she is happy to be within + or − 0.5 of a scale        et al. 1996: 159, 388). Moreover, the average or
         point and accurate 999 times out of 1,000, then           mean of the sample means will be approximately
         she requires a sample of 170 out of the 1,000             the same as the population mean. Hopkins et al.
         if she is a perfectionist and wishes to be within         (1996: 159–62) demonstrate this by reporting
         + or − 0.25 of a scale point and accurate 999             the use of computer simulation to examine the
         times out of 1,000, then she requires a sample            sampling distribution of means when computed
         of 679 out of the 1,000.                                  10,000 times (a method that we discuss in
                                                                                               SAMPLING ERROR          107

Box 4.2                                                    However, as we are usually unable to ascertain the

                                                                                                                         Chapter 4
Distribution of sample means showing the spread            SD of the total population, the standard deviation
of a selection of sample means around the                  of the sample is used instead. The standard error
population mean                                            of the mean provides the best estimate of the
                                                           sampling error. Clearly, the sampling error depends
                                  Mpop   Population mean
                                                           on the variability (i.e. the heterogeneity) in the
                                  Ms     Sample means
                                                           population as measured by SDpop as well as the
                                                           sample size (N) (Rose and Sullivan 1993: 143).
                                                           The smaller the SDpop the smaller the sampling
                                                           error; the larger the N, the smaller the sampling
                                                           error. Where the SDpop is very large, then N
                                                           needs to be very large to counteract it. Where
                                                           SDpop is very small, then N, too, can be small
          Ms Ms Ms Ms      Mpop   Ms Ms Ms Ms              and still give a reasonably small sampling error.
                                                           As the sample size increases the sampling error
Source: Cohen and Holliday 1979                            decreases. Hopkins et al. (1996: 159) suggest that,
                                                           unless there are some very unusual distributions,
                                                           samples of twenty-five or greater usually yield a
                                                           normal sampling distribution of the mean. For
Chapter 10). Rose and Sullivan (1993: 144)                 further analysis of steps that can be taken to cope
remind us that 95 per cent of all sample means             with the estimation of sampling in surveys we refer
fall between plus or minus 1.96 standard errors            the reader to Ross and Wilson (1997).
of the sample and population means, i.e. that we
have a 95 per cent chance of having a single
sample mean within these limits, that the sample           The standard error of proportions
mean will fall within the limits of the population
mean.                                                      We said earlier that one answer to ‘How big a
   By drawing a large number of samples of equal           sample must I obtain?’ is ‘How accurate do I want
size from a population, we create a sampling               my results to be?’ This is well illustrated in the
distribution. We can calculate the error involved          following example:
in such sampling (see http://www.routledge.                  A school principal finds that the 25 students she talks
com/textbooks/9780415368780 – Chapter 4, file                 to at random are reasonably in favour of a proposed
4.5.ppt). The standard deviation of the theoretical          change in the lunch break hours, 66 per cent being in
distribution of sample means is a measure of                 favour and 34 per cent being against. How can she be
sampling error and is called the standard error              sure that these proportions are truly representative of
of the mean (SEM ). Thus,                                    the whole school of 1,000 students?
  SE = √                                                     A simple calculation of the standard error of
         N                                                 proportions provides the principal with her answer.
where SDS = the standard deviation of the sample
and N = the number in the sample.                            SE =
   Strictly speaking, the formula for the standard
error of the mean is:                                      where
       SDpop                                                  P = the percentage in favour
  SE = √
                                                             Q = 100 per cent − P
where SDpop = the standard deviation of the
population.                                                  N = the sample size

      The formula assumes that each sample is drawn           The representativeness of the sample
      on a simple random basis. A small correction fac-
      tor called the finite population correction (fpc) is     The researcher will need to consider the extent
      generally applied as follows:                           to which it is important that the sample in fact
                                                              represents the whole population in question (in
                               (1 − f)P × Q                   the example above, the 1,000 students), if it is
        SE of proportions =                 where f is the
                                     N                        to be a valid sample. The researcher will need
        proportion included in the sample.                    to be clear what it is that is being represented,
                                                              i.e. to set the parameter characteristics of the
      Where, for example, a sample is 100 out of 1,000,       wider population – the sampling frame – clearly
      f is 0.1.                                               and correctly. There is a popular example of
                                                              how poor sampling may be unrepresentative and
                              (1 − 0.1)(66 × 34)
        SE of proportions =                      = 4.49       unhelpful for a researcher. A national newspaper
                                                              reports that one person in every two suffers
      With a sample of twenty-five, the SE = 9.4. In           from backache; this headline stirs alarm in every
      other words, the favourable vote can vary between       doctor’s surgery throughout the land. However,
      56.6 per cent and 75.4 per cent; likewise, the un-      the newspaper fails to make clear the parameters
      favourable vote can vary between 43.4 per cent          of the study which gave rise to the headline.
      and 24.6 per cent. Clearly, a voting possibility        It turns out that the research took place in a
      ranging from 56.6 per cent in favour to 43.4 per        damp part of the country where the incidence
      cent against is less decisive than 66 per cent as op-   of backache might be expected to be higher
      posed to 34 per cent. Should the school principal       than elsewhere, in a part of the country which
      enlarge her sample to include 100 students, then        contained a disproportionate number of elderly
      the SE becomes 4.5 and the variation in the range       people, again who might be expected to have more
      is reduced to 61.5 per cent−70.5 per cent in favour     backaches than a younger population, in an area
      and 38.5 per cent−29.5 per cent against. Sampling       of heavy industry where the working population
      the whole school’s opinion (n = 1, 000) reduces         might be expected to have more backache than
      the SE to 1.5 and the ranges to 64.5 per cent−67.5      in an area of lighter industry or service industries,
      per cent in favour and 35.5 per cent−32.5 per cent      and used only two doctors’ records, overlooking
      against. It is easy to see why political opinion sur-   the fact that many backache sufferers went to
      veys are often based upon sample sizes of 1,000 to      those doctors’ surgeries because the two doctors
      1,500 (Gardner 1978).                                   concerned were known to be overly sympathetic
          What is being suggested here generally is that,     to backache sufferers rather than responsibly
      in order to overcome problems of sampling error,        suspicious.
      in order to ensure that one can separate random            These four variables – climate, age group,
      effects and variation from non-random effects,          occupation and reported incidence – were seen
      and in order for the power of a statistic to be         to exert a disproportionate effect on the study,
      felt, one should opt for as large a sample as           i.e. if the study were to have been carried
      possible. As Gorard (2003: 62) says, ‘power is an       out in an area where the climate, age group,
      estimate of the ability of the test you are using       occupation and reporting were to have been
      to separate the effect size from random variation’,     different, then the results might have been
      and a large sample helps the researcher to achieve      different. The newspaper report sensationally
      statistical power. Samples of fewer than thirty are     generalized beyond the parameters of the data,
      dangerously small, as they allow the possibility of     thereby overlooking the limited representativeness
      considerable standard error, and, for over around       of the study.
      eighty cases, any increases to the sample size have        It is important to consider adjusting the
      little effect on the standard error.                    weightings of subgroups in the sample once the
                                                                               THE ACCESS TO THE SAMPLE           109

data have been collected. For example, in a              picture. Weighting the results is an important

                                                                                                                    Chapter 4
secondary school where half of the students are          consideration.
male and half are female, consider pupils’ responses
to the question ‘How far does your liking of the
form teacher affect your attitude to work?’
                                                         The access to the sample
                                                         Access is a key issue and is an early factor that must
 Variable: How far does your liking of the form
                                                         be decided in research. Researchers will need to
 teacher affect your attitude to school work?
                                                         ensure that access is not only permitted but also, in
          Very       A      Some-   Quite   A very       fact, practicable. For example, if a researcher were
          little   little    what   a lot   great        to conduct research into truancy and unauthorized
                                             deal        absence from school, and decided to interview
Male       10      20        30      25       15         a sample of truants, the research might never
Female     50       80       30      25       15         commence as the truants, by definition, would not
Total      60      100       60      50       30         be present! Similarly access to sensitive areas might
                                                         be not only difficult but also problematical both
   Let us say that we are interested in the attitudes    legally and administratively, for example, access
according to the gender of the respondents, as well      to child abuse victims, child abusers, disaffected
as overall. In this example one could surmise that       students, drug addicts, school refusers, bullies and
generally the results indicate that the liking of the    victims of bullying. In some sensitive areas access
form teacher has only a small to moderate effect         to a sample might be denied by the potential
on the students’ attitude to work. However, we           sample participants themselves, for example AIDS
have to observe that twice as many girls as boys         counsellors might be so seriously distressed by their
are included in the sample, and this is an unfair        work that they simply cannot face discussing with
representation of the population of the school,          a researcher the subject matter of their traumatic
which comprises 50 per cent girls and 50 per cent        work; it is distressing enough to do the job without
boys, i.e. girls are over-represented and boys are       living through it again with a researcher.
under-represented. If one equalizes the two sets            Access might also be denied by the potential
of scores by gender to be closer to the school           sample participants themselves for very practical
population (either by doubling the number of boys        reasons, for example a doctor or a teacher
or halving the number of girls) then the results         simply might not have the time to spend with
look very different.                                     the researcher. Further, access might be denied
                                                         by people who have something to protect, for
 Variable: How far does your liking of the form
                                                         example a school which has recently received
 teacher affect your attitude to school work?
                                                         a very poor inspection result or poor results on
          Very       A      Some-   Quite   A very       external examinations, or people who have made
          little   little    what   a lot   great        an important discovery or a new invention and
                                             deal        who do not wish to disclose the secret of their
Male       20      40        60      50       30         success; the trade in intellectual property has
Female     50       80       30      25       15         rendered this a live issue for many researchers.
Total      70      120       90      75       45         There are very many reasons that might prevent
                                                         access to the sample, and researchers cannot afford
   In this latter case a much more positive picture is   to neglect this potential source of difficulty in
painted, indicating that the students regard their       planning research.
liking of the form teacher as a quite important             In many cases access is guarded by ‘gatekeep-
feature in their attitude to school work. Here           ers’ – people who can control researchers’ access to
equalizing the sample to represent more fairly           those whom they really want to target. For school
the population by weighting yields a different           staff this might be, for example, headteachers,

      school governors, school secretaries, form teach-        Probability samples
      ers; for pupils this might be friends, gang members,
                                                               A probability sample, because it draws randomly
      parents, social workers and so on. It is criti-
                                                               from the wider population, will be useful if the
      cal for researchers to consider not only whether
                                                               researcher wishes to be able to make generaliza-
      access is possible but also how access will be
                                                               tions, because it seeks representativeness of the
      undertaken – to whom does one have to go, both
                                                               wider population. It also permits two-tailed tests
      formally and informally, to gain access to the target
                                                               to be administered in statistical analysis of quan-
                                                               titative data. Probability sampling is popular in
         Not only might access be difficult but also
                                                               randomized controlled trials. On the other hand,
      its corollary – release of information – might be
                                                               a non-probability sample deliberately avoids rep-
      problematic. For example, a researcher might gain
                                                               resenting the wider population; it seeks only to
      access to a wealth of sensitive information and
                                                               represent a particular group, a particular named
      appropriate people, but there might be a restriction
                                                               section of the wider population, such as a class
      on the release of the data collection; in the field
                                                               of students, a group of students who are tak-
      of education in the UK reports have been known
                                                               ing a particular examination, a group of teach-
      to be suppressed, delayed or ‘doctored’. It is not
                                                               ers (see
      always enough to be able to ‘get to’ the sample, the
                                                               9780415368780 – Chapter 4, file 4.6.ppt).
      problem might be to ‘get the information out’ to
                                                                  A probability sample will have less risk of
      the wider public, particularly if it could be critical
                                                               bias than a non-probability sample, whereas,
      of powerful people.
                                                               by contrast, a non-probability sample, being
                                                               unrepresentative of the whole population, may
      The sampling strategy to be used                         demonstrate skewness or bias. (For this type of
                                                               sample a one-tailed test will be used in processing
      There are two main methods of sampling (Cohen
                                                               statistical data.) This is not to say that the former is
      and Holliday 1979; 1982; 1996; Schofield 1996).
                                                               bias free; there is still likely to be sampling error in a
      The researcher must decide whether to opt for
                                                               probability sample (discussed below), a feature that
      a probability (also known as a random sample)
                                                               has to be acknowledged, for example opinion polls
      or a non-probability sample (also known as a
                                                               usually declare their error factors, e.g. ±3 per cent.
      purposive sample). The difference between them
                                                                  There are several types of probability samples:
      is this: in a probability sample the chances of
                                                               simple random samples; systematic samples; strat-
      members of the wider population being selected
                                                               ified samples; cluster samples; stage samples, and
      for the sample are known, whereas in a non-
                                                               multi-phase samples. They all have a measure of
      probability sample the chances of members of the
                                                               randomness built into them and therefore have a
      wider population being selected for the sample
                                                               degree of generalizability.
      are unknown. In the former (probability sample)
      every member of the wider population has an
      equal chance of being included in the sample;
                                                               Simple random sampling
      inclusion or exclusion from the sample is a matter
      of chance and nothing else. In the latter (non-          In simple random sampling, each member of the
      probability sample) some members of the wider            population under study has an equal chance of
      population definitely will be excluded and others         being selected and the probability of a mem-
      definitely included (i.e. every member of the wider       ber of the population being selected is unaf-
      population does not have an equal chance of being        fected by the selection of other members of
      included in the sample). In this latter type the         the population, i.e. each selection is entirely
      researcher has deliberately – purposely – selected       independent of the next. The method involves
      a particular section of the wider population to          selecting at random from a list of the popula-
      include in or exclude from the sample.                   tion (a sampling frame) the required number of
                                                                                       PROBABILITY SAMPLES         111

subjects for the sample. This can be done by            the frequency interval (f) is:

                                                                                                                     Chapter 4
drawing names out of a container until the re-             1, 400
quired number is reached, or by using a table of                  = 4.635 (which rounds up to 5.0)
random numbers set out in matrix form (these            Hence the researcher would pick out every fifth
are reproduced in many books on quantitative            name on the list of cases.
research methods and statistics), and allocating            Such a process, of course, assumes that the
these random numbers to participants or cases           names on the list themselves have been listed in a
(e.g. Hopkins et al. 1996: 148–9). Because of           random order. A list of females and males might
probability and chance, the sample should con-          list all the females first, before listing all the males;
tain subjects with characteristics similar to the       if there were 200 females on the list, the researcher
population as a whole; some old, some young,            might have reached the desired sample size before
some tall, some short, some fit, some unfit,              reaching that stage of the list which contained
some rich, some poor etc. One problem as-               males, thereby distorting (skewing) the sample.
sociated with this particular sampling method           Another example might be where the researcher
is that a complete list of the population is            decides to select every thirtieth person identified
needed and this is not always readily avail-            from a list of school students, but it happens that:
able (see           (a) the school has just over thirty students in each
9780415368780 – Chapter 4, file 4.7.ppt).                class; (b) each class is listed from high ability to
                                                        low ability students; (c) the school listing identifies
                                                        the students by class.
Systematic sampling
                                                            In this case, although the sample is drawn
This method is a modified form of simple random          from each class, it is not fairly representing the
sampling. It involves selecting subjects from a         whole school population since it is drawing almost
population list in a systematic rather than a           exclusively on the lower ability students. This is
random fashion. For example, if from a population       the issue of periodicity (Calder 1979). Not only is
of, say, 2,000, a sample of 100 is required,            there the question of the order in which names
then every twentieth person can be selected.            are listed in systematic sampling, but also there
The starting point for the selection is chosen at       is the issue that this process may violate one of
random (see         the fundamental premises of probability sampling,
9780415368780 – Chapter 4, file 4.8.ppt).                namely that every person has an equal chance
   One can decide how frequently to make                of being included in the sample. In the example
systematic sampling by a simple statistic – the total   above where every fifth name is selected, this
number of the wider population being represented        guarantees that names 1–4, 6–9 etc. will be
divided by the sample size required:                    excluded, i.e. everybody does not have an equal
                                                        chance to be chosen. The ways to minimize this
  f=                                                    problem are to ensure that the initial listing is
                                                        selected randomly and that the starting point for
    f = frequency interval                              systematic sampling is similarly selected randomly.
  N = the total number of the wider population
  sn = the required number in the sample.               Stratified sampling
                                                        Stratified sampling involves dividing the pop-
Let us say that the researcher is working with a        ulation into homogenous groups, each group
school of 1,400 students; by looking at the table       containing subjects with similar characteristics.
of sample size (Box 4.1) required for a random          For example, group A might contain males and
sample of these 1,400 students we see that 302          group B, females. In order to obtain a sam-
students are required to be in the sample. Hence        ple representative of the whole population in

      terms of sex, a random selection of subjects             and spend an inordinate amount of time travelling
      from group A and group B must be taken. If               about in order to test them. By cluster sampling,
      needed, the exact proportion of males to fe-             the researcher can select a specific number of
      males in the whole population can be reflected            schools and test all the students in those selected
      in the sample. The researcher will have to iden-         schools, i.e. a geographically close cluster is sam-
      tify those characteristics of the wider population       pled (see
      which must be included in the sample, i.e. to            9780415368780 – Chapter 4, file 4.10.ppt).
      identify the parameters of the wider population.            One would have to be careful to ensure that
      This is the essence of establishing the sampling         cluster sampling does not build in bias. For
      frame (see           example, let us imagine that we take a cluster
      9780415368780 – Chapter 4, file 4.9.ppt).                 sample of a city in an area of heavy industry or
         To organize a stratified random sample is a            great poverty; this may not represent all kinds of
      simple two-stage process. First, identify those          cities or socio-economic groups, i.e. there may be
      characteristics that appear in the wider population      similarities within the sample that do not catch
      that must also appear in the sample, i.e. divide         the variability of the wider population. The issue
      the wider population into homogenous and, if             here is one of representativeness; hence it might be
      possible, discrete groups (strata), for example          safer to take several clusters and to sample lightly
      males and females. Second, randomly sample               within each cluster, rather to take fewer clusters
      within these groups, the size of each group              and sample heavily within each.
      being determined either by the judgement of                 Cluster samples are widely used in small-scale
      the researcher or by reference to Boxes 4.1              research. In a cluster sample the parameters of the
      or 4.2.                                                  wider population are often drawn very sharply; a
         The decision on which characteristics to include      researcher, therefore, would have to comment on
      should strive for simplicity as far as possible, as      the generalizability of the findings. The researcher
      the more factors there are, not only the more            may also need to stratify within this cluster sample
      complicated the sampling becomes, but often the          if useful data, i.e. those which are focused and
      larger the sample will have to be to include             which demonstrate discriminability, are to be
      representatives of all strata of the wider population.   acquired.
         A stratified random sample is, therefore, a
      useful blend of randomization and categorization,
                                                               Stage sampling
      thereby enabling both a quantitative and
      qualitative piece of research to be undertaken.          Stage sampling is an extension of cluster sampling.
      A quantitative piece of research will be able            It involves selecting the sample in stages, that
      to use analytical and inferential statistics, while      is, taking samples from samples. Using the large
      a qualitative piece of research will be able to          community example in cluster sampling, one type
      target those groups in institutions or clusters of       of stage sampling might be to select a number of
      participants who will be able to be approached to        schools at random, and from within each of these
      participate in the research.                             schools, select a number of classes at random,
                                                               and from within those classes select a number of
      Cluster sampling
                                                                  Morrison (1993: 121–2) provides an example
      When the population is large and widely dis-             of how to address stage sampling in practice. Let
      persed, gathering a simple random sample poses           us say that a researcher wants to administer a
      administrative problems. Suppose we want to sur-         questionnaire to all 16-year-old pupils in each
      vey students’ fitness levels in a particularly large      of eleven secondary schools in one region. By
      community or across a country. It would be com-          contacting the eleven schools she finds that there
      pletely impractical to select students randomly          are 2,000 16-year-olds on roll. Because of questions
                                                                               NON-PROBABILITY SAMPLES           113

of confidentiality she is unable to find out the         markedly different ways); phase three might be

                                                                                                                   Chapter 4
names of all the students so it is impossible to       based on a political criterion (e.g. schools whose
draw their names out of a container to achieve         students are drawn from areas with a tradition
randomness (and even if she had the names, it          of support for a particular political party), and
would be a mind-numbing activity to write out          so on. What is evident here is that the sample
2,000 names to draw out of a container!). From         population will change at each phase of the re-
looking at Box 4.1 she finds that, for a random         search (see
sample of the 2,000 students, the sample size is       9780415368780 – Chapter 4, file 4.11.ppt).
322 students. How can she proceed?
   The first stage is to list the eleven schools on
                                                       Non-probability samples
a piece of paper and then to write the names of
the eleven schools on to small cards and place         The selectivity which is built into a non-
each card in a container. She draws out the first       probability sample derives from the researcher
name of the school, puts a tally mark by the           targeting a particular group, in the full knowledge
appropriate school on her list and returns the         that it does not represent the wider population; it
card to the container. The process is repeated 321     simply represents itself. This is frequently the case
times, bringing the total to 322. The final totals      in small-scale research, for example, as with one
might appear thus:                                     or two schools, two or three groups of students, or
School       1 2 3 4 5 6 7 8 9 10 11 Total
                                                       a particular group of teachers, where no attempt
Required no.
                                                       to generalize is desired; this is frequently the case
of students 22 31 32 24 29 20 35 28 32 38 31 322
                                                       for some ethnographic research, action research
                                                       or case study research (see http://www.routledge.
   For the second stage the researcher then            com/textbooks/9780415368780 – Chapter 4, file
approaches the eleven schools and asks each of         4.12.ppt). Small-scale research often uses non-
them to select randomly the required number of         probability samples because, despite the disadvan-
students for each school. Randomness has been          tages that arise from their non-representativeness,
maintained in two stages and a large number            they are far less complicated to set up, are con-
(2,000) has been rendered manageable. The              siderably less expensive, and can prove perfectly
process at work here is to go from the general to      adequate where researchers do not intend to gener-
the specific, the wide to the focused, the large to     alize their findings beyond the sample in question,
the small. Caution has to be exercised here, as the    or where they are simply piloting a questionnaire
assumption is that the schools are of the same size    as a prelude to the main study.
and are large; that may not be the case in practice,      Just as there are several types of probability sam-
in which case this strategy may be inadvisable.        ple, so there are several types of non-probability
                                                       sample: convenience sampling, quota sampling,
                                                       dimensional sampling, purposive sampling and
Multi-phase sampling
                                                       snowball sampling. Each type of sample seeks only
In stage sampling there is a single unifying purpose   to represent itself or instances of itself in a similar
throughout the sampling. In the previous example       population, rather than attempting to represent
the purpose was to reach a particular group of         the whole, undifferentiated population.
students from a particular region. In a multi-phase
sample the purposes change at each phase, for
                                                       Convenience sampling
example, at phase one the selection of the sam-
ple might be based on the criterion of geography       Convenience sampling – or, as it is sometimes
(e.g. students living in a particular region); phase   called, accidental or opportunity sampling –
two might be based on an economic criterion            involves choosing the nearest individuals to serve
(e.g. schools whose budgets are administered in        as respondents and continuing that process until

      the required sample size has been obtained or              also appear in the sample, i.e. divide the wider
      those who happen to be available and accessible            population into homogenous and, if possible,
      at the time. Captive audiences such as students or         discrete groups (strata), for example, males
      student teachers often serve as respondents based          and females, Asian, Chinese and African
      on convenience sampling. Researchers simply                Caribbean.
      choose the sample from those to whom they              2   Identify the proportions in which the selected
      have easy access. As it does not represent any             characteristics appear in the wider population,
      group apart from itself, it does not seek to               expressed as a percentage.
      generalize about the wider population; for a           3   Ensure that the percentaged proportions of
      convenience sample that is an irrelevance. The             the characteristics selected from the wider
      researcher, of course, must take pains to report           population appear in the sample.
      this point – that the parameters of generalizability
                                                             Ensuring correct proportions in the sample may
      in this type of sample are negligible. A
                                                             be difficult to achieve if the proportions in the
      convenience sample may be the sampling strategy
                                                             wider community are unknown or if access to the
      selected for a case study or a series of case
                                                             sample is difficult; sometimes a pilot survey might
      studies (see
                                                             be necessary in order to establish those proportions
      9780415368780 – Chapter 4, file 4.13.ppt).
                                                             (and even then sampling error or a poor response
                                                             rate might render the pilot data problematical).
      Quota sampling                                            It is straightforward to determine the minimum
      Quota sampling has been described as the               number required in a quota sample. Let us say that
      non-probability equivalent of stratified sam-           the total number of students in a school is 1,700,
      pling (Bailey 1978). Like a stratified sample, a        made up thus:
      quota sample strives to represent significant char-      Performing arts                300 students
      acteristics (strata) of the wider population; unlike    Natural sciences               300 students
      stratified sampling it sets out to represent these       Humanities                     600 students
      in the proportions in which they can be found           Business and Social Sciences   500 students
      in the wider population. For example, suppose
                                                             The proportions being 3:3:6:5, a minimum of 17
      that the wider population (however defined) were
                                                             students might be required (3 + 3 + 6 + 5) for
      composed of 55 per cent females and 45 per cent
                                                             the sample. Of course this would be a minimum
      males, then the sample would have to contain 55
                                                             only, and it might be desirable to go higher than
      per cent females and 45 per cent males; if the
                                                             this. The price of having too many characteristics
      population of a school contained 80 per cent of
                                                             (strata) in quota sampling is that the minimum
      students up to and including the age of 16 and
                                                             number in the sample very rapidly could become
      20 per cent of students aged 17 and over, then
                                                             very large, hence in quota sampling it is advisable
      the sample would have to contain 80 per cent of
                                                             to keep the numbers of strata to a minimum. The
      students up to the age of 16 and 20 per cent of stu-
                                                             larger the number of strata, the larger the number
      dents aged 17 and above. A quota sample, then,
                                                             in the sample will become, usually at a geometric
      seeks to give proportional weighting to selected
                                                             rather than an arithmetic rate of progression.
      factors (strata) which reflects their weighting in
      which they can be found in the wider popu-
      lation (see        Purposive sampling
      9780415368780 – Chapter 4, file 4.14.ppt). The          In purposive sampling, often (but by no means
      researcher wishing to devise a quota sample can        exclusively) a feature of qualitative research,
      proceed in three stages:                               researchers handpick the cases to be included
      1   Identify those characteristics (factors) which     in the sample on the basis of their judgement
          appear in the wider population which must          of their typicality or possession of the particular
                                                                                 NON-PROBABILITY SAMPLES          115

characteristics being sought. In this way, they build     a primary school or nursery, or one might have a

                                                                                                                    Chapter 4
up a sample that is satisfactory to their specific         very small number of children from certain ethnic
needs. As its name suggests, the sample has been          minorities in a school, such that they may not
chosen for a specific purpose, for example: a group        feature in a sample. In this case the researcher will
of principals and senior managers of secondary            deliberately seek to include a sufficient number of
schools is chosen as the research is studying the         them to ensure appropriate statistical analysis or
incidence of stress among senior managers; a group        representation in the sample, adjusting any results
of disaffected students has been chosen because           from them, through weighting, to ensure that they
they might indicate most distinctly the factors           are not over-represented in the final results. This
which contribute to students’ disaffection (they          is an endeavour, perhaps, to reach and meet the
are critical cases, akin to ‘critical events’ discussed   demands of social inclusion.
in Chapter 18, or deviant cases – those cases which          A further variant of purposive sampling
go against the norm: (Anderson and Arsenault              is negative case sampling. Here the researcher
1998: 124); one class of students has been selected       deliberately seeks those people who might
to be tracked throughout a week in order to report        disconfirm the theories being advanced (the
on the curricular and pedagogic diet which is             Popperian equivalent of falsifiability), thereby
offered to them so that other teachers in the             strengthening the theory if it survives such
school might compare their own teaching to that           disconfirming cases. A softer version of negative
reported. While it may satisfy the researcher’s           case sampling is maximum variation sampling,
needs to take this type of sample, it does not            selecting cases from as diverse a population as
pretend to represent the wider population; it             possible (Anderson and Arsenault 1998: 124) in
is deliberately and unashamedly selective and             order to ensure strength and richness to the data,
biased (see           their applicability and their interpretation. In this
9780415368780 – Chapter 4, file 4.15.ppt).                 latter case, it is almost inevitable that the sample
   In many cases purposive sampling is used in            size will increase or be large.
order to access ‘knowledgeable people’, i.e. those
who have in-depth knowledge about particular
                                                          Dimensional sampling
issues, maybe by virtue of their professional
role, power, access to networks, expertise or             One way of reducing the problem of sample size in
experience (Ball 1990). There is little benefit            quota sampling is to opt for dimensional sampling.
in seeking a random sample when most of                   Dimensional sampling is a further refinement of
the random sample may be largely ignorant of              quota sampling. It involves identifying various
particular issues and unable to comment on                factors of interest in a population and obtaining
matters of interest to the researcher, in which           at least one respondent of every combination of
case a purposive sample is vital. Though they may         those factors. Thus, in a study of race relations,
not be representative and their comments may not          for example, researchers may wish to distinguish
be generalizable, this is not the primary concern         first, second and third generation immigrants.
in such sampling; rather the concern is to acquire        Their sampling plan might take the form of a
in-depth information from those who are in a              multidimensional table with ‘ethnic group’ across
position to give it.                                      the top and ‘generation’ down the side. A second
   Another variant of purposive sampling is the           example might be of a researcher who may be in-
boosted sample. Gorard (2003: 71) comments on             terested in studying disaffected students, girls and
the need to use a boosted sample in order to include      secondary-aged students and who may find a single
those who may otherwise be excluded from, or              disaffected secondary female student, i.e. a respon-
under-represented in, a sample because there are          dent who is the bearer of all of the sought charac-
so few of them. For example, one might have a very        teristics (see
small number of special needs teachers or pupils in       9780415368780 – Chapter 4, file 4.16.ppt).

      Snowball sampling                                        particular school or headteacher. Volunteers may
                                                               be well intentioned, but they do not necessarily
      In snowball sampling researchers identify a small
                                                               represent the wider population, and this would
      number of individuals who have the characteristics
                                                               have to be made clear.
      in which they are interested. These people are
      then used as informants to identify, or put the
      researchers in touch with, others who qualify
                                                               Theoretical sampling
      for inclusion and these, in turn, identify yet
      others – hence the term snowball sampling. This          This is a feature of grounded theory. In grounded
      method is useful for sampling a population where         theory the sample size is relatively immaterial, as
      access is difficult, maybe because it is a sensitive      one works with the data that one has. Indeed
      topic (e.g. teenage solvent abusers) or where            grounded theory would argue that the sample size
      communication networks are undeveloped (e.g.             could be infinitely large, or, as a fall-back position,
      where a researcher wishes to interview stand-in          large enough to saturate the categories and issues,
      ‘supply’ teachers – teachers who are brought in          such that new data will not cause the theory that
      on an ad-hoc basis to cover for absent regular           has been generated to be modified.
      members of a school’s teaching staff – but finds             Theoretical sampling requires the researcher
      it difficult to acquire a list of these stand-in          to have sufficient data to be able to generate
      teachers), or where an outside researcher has            and ‘ground’ the theory in the research context,
      difficulty in gaining access to schools (going            however defined, i.e. to create a theoretical
      through informal networks of friends/acquaintance        explanation of what is happening in the situation,
      and their friends and acquaintances and so on            without having any data that do not fit the theory.
      rather than through formal channels). The task for       Since the researcher will not know in advance
      the researcher is to establish who are the critical or   how much, or what range of data will be required,
      key informants with whom initial contact must be         it is difficult, to the point of either impossibility,
      made (see            exhaustion or time limitations, to know in advance
      9780415368780 – Chapter 4, file 4.17.ppt).                the sample size required. The researcher proceeds
                                                               in gathering more and more data until the theory
                                                               remains unchanged or until the boundaries of
      Volunteer sampling
                                                               the context of the study have been reached,
      In cases where access is difficult, the researcher may    until no modifications to the grounded theory are
      have to rely on volunteers, for example, personal        made in light of the constant comparison method.
      friends, or friends of friends, or participants who      Theoretical saturation (Glaser and Strauss 1967:
      reply to a newspaper advertisement, or those who         61) occurs when no additional data are found that
      happen to be interested from a particular school,        advance, modify, qualify, extend or add to the
      or those attending courses. Sometimes this is            theory developed.
      inevitable (Morrison 2006), as it is the only kind          Glaser and Strauss (1967) write that
      of sampling that is possible, and it may be better
                                                                 theoretical sampling is the process of data collection
      to have this kind of sampling than no research
                                                                 for generating theory whereby the analyst jointly
      at all.
                                                                 collects, codes, and analyzes his [sic.] data and decides
         In these cases one has to be very cautious
                                                                 what data to collect next and where to find them, in
      in making any claims for generalizability or
                                                                 order to develop his theory as it emerges.
      representativeness, as volunteers may have a range
                                                                                         (Glaser and Strauss 1967: 45)
      of different motives for volunteering, e.g. wanting
      to help a friend, interest in the research, wanting         The two key questions, for the grounded theorist
      to benefit society, an opportunity for revenge on a       using theoretical sampling are, first, to which
                                                                                              CONCLUSION        117

groups does one turn next for data? Second, for           4   Ensure that access to the sample is guaranteed.

                                                                                                                  Chapter 4
what theoretical purposes does one seek further               If not, be prepared to modify the sampling
data? In response to the first, Glaser and Strauss             strategy (step 2).
(1967: 49) suggest that the decision is based on          5   For probability sampling, identify the confi-
theoretical relevance, i.e. those groups that will            dence level and confidence intervals that you
assist in the generation of as many properties and            require.
categories as possible.                                       For non-probability sampling, identify the
   Hence the size of the data set may be fixed by the          people whom you require in the sample.
number of participants in the organization, or the        6   Calculate the numbers required in the sample,
number of people to whom one has access, but                  allowing for non-response, incomplete or
the researcher has to consider that the door may              spoiled responses, attrition and sample
have to be left open for him/her to seek further              mortality, i.e. build in redundancy.
data in order to ensure theoretical adequacy and to       7   Decide how to gain and manage access
check what has been found so far with further data            and contact (e.g. advertisement, letter,
(Flick et al. 2004: 170). In this case it is not always       telephone, email, personal visit, personal
possible to predict at the start of the research just         contacts/friends).
how many, and who, the research will need for the         8   Be prepared to weight (adjust) the data, once
sampling; it becomes an iterative process.                    collected.
   Non-probability samples also reflect the issue
that sampling can be of people but it can also
be of issues. Samples of people might be selected
because the researcher is concerned to address            The message from this chapter is the same as for
specific issues, for example, those students who           many of the others – that every element of the
misbehave, those who are reluctant to go to school,       research should not be arbitrary but planned and
those with a history of drug dealing, those who           deliberate, and that, as before, the criterion of
prefer extra-curricular to curricular activities. Here    planning must be fitness for purpose. The selection
it is the issue that drives the sampling, and so the      of a sampling strategy must be governed by the
question becomes not only ‘whom should I sample’          criterion of suitability. The choice of which
but also ‘what should I sample’ (Mason 2002:              strategy to adopt must be mindful of the purposes
127–32). In turn this suggests that it is not only        of the research, the time scales and constraints on
people who may be sampled, but texts, documents,          the research, the methods of data collection, and
records, settings, environments, events, objects,         the methodology of the research. The sampling
organizations, occurrences, activities and so on.         chosen must be appropriate for all of these factors
                                                          if validity is to be served.
                                                             To the question ‘how large should my sample
Planning a sampling strategy
                                                          be?’, the answer is complicated. This chapter has
There are several steps in planning the sampling          suggested that it all depends on:
                                                              population size
1   Decide whether you need a sample, or whether              confidence level and confidence interval
    it is possible to have the whole population.              required
2   Identify the population, its important features           accuracy required (the smallest sampling error
    (the sampling frame) and its size.                        sought)
3   Identify the kind of sampling strategy you                number of strata required
    require (e.g. which variant of probability and            number of variables included in the study
    non-probability sample you require).                      variability of the factor under study

        the kind of sample (different kinds of             need to keep proportionality in a proportionate
        sample within probability and non-probability      sample.
        representativeness of the sample                  That said, this chapter has urged researchers to
        allowances to be made for attrition and non-    use large rather than small samples, particularly in
        response                                        quantitative research.
5        Sensitive educational research

Much educational research can be sensitive, in          Contents, e.g. taboo or emotionally charged
several senses, and researchers have to be acutely      areas of study (Farberow 1963), e.g. criminal-
aware of a variety of delicate issues. This chapter     ity, deviance, sex, race, bereavement, viol-
sets out different ways in which educational            ence, politics, policing, human rights, drugs,
research might be sensitive. It then takes two          poverty, illness, religion and the sacred,
significant issues in the planning and conduct           lifestyle, family, finance, physical appearance,
of sensitive research – sampling and access – and       power and vested interests (Lee 1993; Arditti
indicates why these twin concerns might be              2002; Chambers 2003).
troublesome for researchers, and how they might         Situational and contextual circumstances (Lee
be addressed. Our outline includes a discussion         1993).
of gatekeepers and their roles. Sensitive research      Intrusion into private spheres and deep
raises a range of difficult, sometimes intractable,      personal experience (Lee and Renzetti 1993:
ethical issues, and we set out some of these in the     5), e.g. sexual behaviour, religious practices,
chapter. Investigations involving powerful people       death and bereavement, even income and age.
are taken as an instance of sensitive educational       Potential sanction, risk or threat of stigma-
research, and this is used as a vehicle for examining   tization, incrimination, costs or career loss
several key problematic matters in this area. The       to the researcher, participants or others, e.g.
chapter moves to a practical note, proffering advice    groups and communities (Lee and Renzetti
on how to ask questions in sensitive research.          1993; Renzetti and Lee 1993; De Laine 2000),
Finally, the chapter sets out a range of key issues     a particular issue for the researcher who studies
to be addressed in the planning, conduct and            human sexuality and who, consequently, suffers
reporting of sensitive research.                        from ‘stigma contagion’, i.e. sharing the same
                                                        stigma as those being studied (Lee 1993: 9).
                                                        Impingement on political alignments (Lee
What is sensitive research?                             1993).
                                                        Cultural and cross-cultural factors and inhibi-
Sensitive research is that ‘which potentially poses
                                                        tions (Sieber 1992: 129).
a substantial threat to those who are involved
                                                        Fear of scrutiny and exposure (Payne et al.
or have been involved in it’ (Lee 1993: 4), or
when those studied view the research as somehow
                                                        Threat to the researchers and to the
undesirable (Van Meter 2000). Sensitivity can
                                                        family members and associates of those
derive from many sources, including:
                                                        studied (Lee 1993); Lee (1993: 34) suggests
   Consequences for the participants (Sieber and        that ‘chilling’ may take place, i.e. where
   Stanley 1988: 49).                                   researchers are ‘deterred from producing or
   Consequences for other people, e.g. family           disseminating research’ because they anticipate
   members, associates, social groups and the           hostile reactions from colleagues, e.g. on
   wider community, research groups and                 race. ‘Guilty knowledge’ may bring personal
   institutions (Lee 1993: 5).                          and professional risk from colleagues; it

         is threatening both to researchers and               to investigate. Sensitive educational research can
         participants (De Laine 2000: 67, 84).                act as a voice for the weak, the oppressed, those
         Methodologies and conduct, e.g. when junior          without a voice or who are not listened to; equally
         researchers conduct research on powerful             it can focus on the powerful and those in high
         people, when men interview women, when               profile positions.
         senior politicians are involved, or where access         The three kinds of sensitivities indicated above
         and disclosure are difficult (Simons 1989; Ball       may appear separately or in combination. The
         1990; 1994a; Liebling and Shah 2001).                sensitivity concerns not only the topic itself, but
                                                              also, perhaps more importantly, ‘the relationship
         Sometimes all or nearly all of the issues listed     between that topic and the social context’
      above are present simultaneously. Indeed, in some       within which the research is conducted (Lee
      situations the very activity of actually undertaking    1993: 5). What appears innocent to the researcher
      educational research per se may be sensitive.           may be highly sensitive to the researched or
      This has long been the situation in totalitarian        to other parties. Threat is a major source of
      regimes, where permission has typically had to          sensitivity; indeed Lee (1993: 5) suggests that,
      be granted from senior government officers and           rather than generating a list of sensitive topics,
      departments in order to undertake educational           it is more fruitful to look at the conditions
      research. Closed societies may permit educational       under which ‘sensitivity’ arises within the research
      research only on approved, typically non-sensitive      process. Given this issue, the researcher will
      and comparatively apolitical topics. As Lee             need to consider how sensitive the educational
      (1993: 6) suggests: ‘research for some groups . . .     research will be, not only in terms of the
      is quite literally an anathema’. The very act           subject matter itself, but also in terms of the
      of doing the educational research, regardless of        several parties that have a stake in it, for
      its purpose, focus, methodology or outcome, is          example: headteachers and senior staff; parents;
      itself a sensitive matter (Morrison 2006). In this      students; schools; governors; local politicians and
      situation the conduct of educational research may       policy-makers; the researcher(s) and research
      hinge on interpersonal relations, local politics        community; government officers; the community;
      and micro-politics. What start as being simply          social workers and school counsellors; sponsors and
      methodological issues can turn out to be ethical        members of the public; members of the community
      and political/micro-political minefields.                being studied; and so on.
         Lee (1993: 4) suggests that sensitive research           Sensitivity inheres not only in the educational
      falls into three main areas: intrusive threat           topic under study, but also, much more
      (probing into areas which are ‘private, stressful       significantly, in the social context in which the
      or sacred’); studies of deviance and social control,    educational research takes place and on the likely
      i.e. which could reveal information that could          consequences of that research on all parties. Doing
      stigmatize or incriminate (threat of sanction);         research is not only a matter of designing a project
      and political alignments, revealing the vested          and collecting, analysing and reporting data – that
      interests of ‘powerful persons or institutions,         is the optimism of idealism or ignorance – but also
      or the exercise of coercion or domination’,             a matter of interpersonal relations, potentially
      or extremes of wealth and status (Lee 1993).            continual negotiation, delicate forging and
      As Beynon (1988: 23) says, ‘the rich and                sustaining of relationships, setback, modification
      powerful have encouraged hagiography, not               and compromise. In an ideal world educational
      critical investigation’. Indeed, Lee (1993: 8) argues   researchers would be able to plan and conduct
      that there has been a tendency to ‘study down’          their studies untrammelled; however, the ideal
      rather than ‘study up’, i.e. to direct attention        world, in the poet Yeats’s words, is ‘an image of
      to powerless rather than powerful groups, not           air’. Sensitive educational research exposes this
      least because these are easier and less sensitive       very clearly. While most educational research
                                                                                SAMPLING AND ACCESS         121

will incur sensitivities, the attraction of discussing   of confidentiality may prevent this from being

                                                                                                              Chapter 5
sensitive research per se is that it highlights what     employed).
these delicate issues might be and how they might        Screening: targeting a particular location and
be felt at their sharpest. We advise readers to          canvassing within it (which may require much
consider most educational research as sensitive, to      effort for little return).
anticipate what those sensitivities might be, and        Outcropping: this involves going to a particular
what trade-offs might be necessary.                      location where known members of the
                                                         target group congregate or can be found
Sampling and access                                      (e.g. Humphreys’ (1970) celebrated study of
                                                         homosexual ‘tearoom trade’); in education this
Walford (2001: 33) argues that gaining access and        may be a particular staffroom (for teachers),
becoming accepted is a slow process. Hammersley          or meeting place for students. Outcropping
and Atkinson (1983: 54) suggest that gaining             risks bias, as there is no simple check for
access not only is a practical matter but also           representativeness of the sample.
provides insights into the ‘social organisation of       Servicing: Lee (1993: 72) suggests that it may
the setting’.                                            be possible to reach research participants by
   Lee (1993: 60) suggests that there are potentially    offering them some sort of service in return
serious difficulties in sampling and access in            for their participation. Researchers must be
sensitive research, not least because of the problem     certain that they really are able to provide
of estimating the size of the population from            the services promised. As Walford (2001: 36)
which the sample is to be drawn, as members              writes: ‘people don’t buy products; they buy
of particular groups, e.g. deviant or clandestine        benefits’, and researchers need to be clear on
groups, will not want to disclose their associations.    the benefits offered.
Similarly, like-minded groups may not wish to            Professional informants: Lee (1993: 73) suggests
open themselves to public scrutiny. They may             these could be, for example, police, doctors,
have much to lose by revealing their membership          priests, or other professionals. In education
and, indeed, their activities may be illicit, critical   these may include social workers and
of others, unpopular, threatening to their own           counsellors. This may be unrealistic optimism,
professional security, deviant and less frequent         as these very people may be bound by terms
than activities in other groups, making access           of legal or ethical confidentiality or voluntary
to them a major obstacle. What if a researcher           self-censorship (e.g. an AIDS counsellor, after
is researching truancy, or teenage pregnancy, or         a harrowing day at work, may not wish
bullying, or solvent abuse among school students,        to continue talking to a stranger about
or alcohol and medication use among teachers, or         AIDS counselling, or a social worker or
family relationship problems brought about by the        counsellor may be constrained by professional
stresses of teaching?                                    confidentiality, or an exhausted teacher may
   Lee (1993: 61) suggests several strategies to         not wish to talk about teaching difficulties).
be used, either separately or in combination, for        Further, Lee suggests that, even if such people
sampling ‘special’ populations (e.g. rare or deviant     agree to participate, they may not know the
populations):                                            full story; Lee (1993: 73) gives the example
   List sampling: looking through public domain          of drug users whose contacts with the police
   lists of, for example, the recently divorced          may be very different from their contacts with
   (though such lists may be more helpful to social      doctors or social workers, or, the corollary of
   researchers than, specifically, educational            this, the police, doctors and social workers may
   researchers).                                         not see the same group of drug users.
   Multipurposing: using an existing survey to           Advertising: though this can potentially reach a
   reach populations of interest (though problems        wide population, it may be difficult to control

          the nature of those who respond, in terms of            pointed out. For example, a headteacher may
          representativeness or suitability.                      wish to confide in a researcher, teachers may
          Networking: this is akin to snowball sampling,          benefit from discussions with a researcher,
          wherein one set of contacts puts the researcher         students may benefit from being asked about
          in touch with more contacts, who puts the               their learning.
          researcher in touch with yet more contacts          4   Sale (where the participants agree to the
          and so on. This is a widely used technique,             research).
          though Lee (1993: 66) reports that it is not
          always easy for contacts to be passed on,              Whitty and Edwards (1994: 22) argue that in
          as initial informants may be unwilling to           order to overcome problems of access, ingenuity
          divulge members of a close-knit community.          and even the temptation to use subterfuge could
          On the other hand, Morrison (2006) reports          be considered: ‘denied co-operation initially by
          that networking is a popular technique where        an independent school, we occasionally contacted
          it is difficult to penetrate a formal organization   some parents through their child’s primary school
          such as a school, if the gatekeepers (those         and then told the independent schools we already
          who can grant or prevent access to others,          were getting some information about their pupils’.
          e.g. the headteacher or senior staff) refuse        They also add that it is sometimes necessary
          access. He reports the extensive use of informal    for researchers to indicate that they are ‘on the
          networks by researchers, in order to contact        same side’ as those being researched.1 Indeed they
          friends and professional associates, and, in        report that ‘we were questioned often about our
          turn, their friends and professional associates,    own views, and there were times when to be
          thereby sidestepping the formal lines of contact    viewed suspiciously from one side proved helpful in
          through schools.                                    gaining access to the other’ (Whitty and Edwards
                                                              1994: 22). This harks back to Becker’s (1968)
        Walford (2001: 36–47) sets out a four-step
                                                              advice to researchers to decide whose side they
      process of gaining access:
                                                              are on.
      1    Approach (gaining entry, perhaps through a            The use of snowball sampling builds in
           mutual friend or colleague – a link person). In    ‘security’ (Lee 1993), as the contacts are those
           this context Walford (2001) cautions that an       who are known and trusted by the members of
           initial letter should be used only to gain an      the ‘snowball’. That said, this itself can lead to
           initial interview or an appointment, or even       bias, as relationships between participants in the
           to arrange to telephone the headteacher in         sample may consist of ‘reciprocity and transitivity’
           order to arrange an interview, not to conduct      (Lee 1993: 67), i.e. participants may have close
           the research or to gain access.                    relationships with one another and may not wish
      2    Interest (using a telephone call to arrange        to break these. Thus homogeneity of the sample’s
           an initial interview). In this respect Walford     attributes may result.
           (2001: 43) notes that headteachers like to            Such snowball sampling may alter the
           talk, and so it is important to let them talk,     research, for example changing random, stratified
           even on the telephone when arranging an            or proportionate sampling into convenience
           interview to discuss the research.                 sampling, thereby compromising generalizability
      3    Desire (overcoming objections and stressing        or generating the need to gain generalizability
           the benefits of the research). As Walford           by synthesizing many case studies. Nevertheless,
           (2001: 44) wisely comments: ‘after all, schools    it often comes to a choice between accepting
           have purposes other than to act as research        non-probability strategies or doing nothing.
           sites’. He makes the telling point that the           The issues of access to people in order to
           research may actually benefit the school, but       conduct sensitive research may require researchers
           that the school may not realize this until it is   to demonstrate a great deal of ingenuity and
                                                                                  SAMPLING AND ACCESS          123

forethought in their planning. Investigators have         schools/institutions not wishing to be identifi-

                                                                                                                 Chapter 5
to be adroit in anticipating problems of access, and      able, even with protections guaranteed
set up their studies in ways that circumvent such         local political factors that impinge on the
problems, preventing them from arising in the             school/educational institution
first place, e.g. by exploring their own institutions      teachers’/participants’ fear of being identi-
or personal situations, even if this compromises          fied/traceable, even with protections guaran-
generalizability. Such anticipatory behaviour can         teed
lead to a glut of case studies, action research and       fear of participation by teachers (e.g. if they
accounts of their own institutions, as these are the      say critical matters about the school or others
only kinds of research possible, given the problem        they could lose their contracts)
of access.                                                unwillingness of teachers to be involved
                                                          because of their workload
                                                          the principal deciding on whether to involve
Gatekeepers                                               the staff, without consultation with the staff
                                                          schools’ fear of criticism/loss of face or
Access might be gained through gatekeepers, that
is, those who control access. Lee (1993: 123)
                                                          the sensitivity of the research – the issues being
suggests that ‘social access crucially depends on
establishing interpersonal trust. Gatekeepers play a
                                                          the power/position of the researcher (e.g. if the
significant role in research, particularly in ethno-
                                                          researcher is a junior or senior member of staff
graphic research (Miller and Bell 2002: 53). They
                                                          or an influential person in education).
control access and re-access (Miller and Bell
2002: 55). They may provide or block access;
they may steer the course of a piece of research,         Risk reduction may result in participants
‘shepherding the fieldworker in one direction or        imposing conditions on research (e.g. on what
another’ (Hammersley and Atkinson 1983: 65), or        information investigators may or may not use; to
exercise surveillance over the research.               whom the data can be shown; what is ‘public’;
   Gatekeepers may wish to avoid, contain, spread      what is ‘off the record’ (and what should be
or control risk and therefore may bar access           done with off-the-record remarks). It may also
or make access conditional. Making research            lead to surveillance/‘chaperoning’ of the researcher
conditional may require researchers to change          while the study is being conducted on site (Lee
the nature of their original plans in terms of         1993: 125).
methodology, sampling, focus, dissemination,              Gatekeepers may want to ‘inspect, modify or
reliability and validity, reporting and control of     suppress the published products of the research’
data (Morrison 2006).                                  (Lee 1993: 128). They may also wish to use the
   Morrison (2006) found that in conducting            research for their own ends, i.e. their involvement
sensitive educational research there were problems     may not be selfless or disinterested, or they may
of                                                     wish for something in return, e.g. for the researcher
                                                       to include in the study an area of interest to the
   gaining access to schools and teachers              gatekeeper, or to report directly – and maybe ex-
   gaining permission to conduct the research          clusively – to the gatekeeper. The researcher has to
   (e.g. from school principals)                       negotiate a potential minefield here, for example,
   resentment by principals                            not to be seen as an informer for the headteacher.
   people vetting which data could be used             As Walford (2001: 45) writes: ‘headteachers [may]
   finding enough willing participants for the          suggest that researchers observe certain teachers
   sample                                              whom they want information about’. Researchers
   schools/institutions/people not wishing to          may need to reassure participants that their data
   divulge information about themselves                will not be given to the headteacher.

         On the other hand, Lee (1993: 127) suggests                Box 5.1
      that the researcher may have to make a few                    Issues of sampling and access in sensitive research
      concessions in order to be able to undertake the
      investigation, i.e. that it is better to do a little of             How to calculate the population and sample.
      the gatekeeper’s bidding rather than not to be able                 How representative of the population the sample
      to do the research at all.                                          may or may not be.
         In addition to gatekeepers the researcher may                    What kind of sample is desirable (e.g. random), but
      find a ‘sponsor’ in the group being studied. A                       what kind may be the only sort that is practicable
                                                                          (e.g. snowball).
      sponsor may provide access, information and
                                                                          How to use networks for reaching the sample, and
      support. A celebrated example of this is in the                     what kinds of networks to utilize.
      figure of ‘Doc’ in Whyte’s classic study of Street                   How to research in a situation of threat to the
      Corner Society (1993: the original study published                  participants (including the researcher).
      in 1943). Here Doc, a leading gang figure in the                     How to protect identities and threatened groups.
                                                                          How to contact the hard-to-reach.
      Chicago street corner society, is quoted as saying
                                                                          How to secure and sustain access.
      (p. 292):                                                           How to find and involve gatekeepers and sponsors.
                                                                          What to offer gatekeepers and sponsors.
        You tell me what you want me to see, and we’ll
                                                                          On what matters compromise may need to be
        arrange it. When you want some information, I’ll ask              negotiated.
        for it, and you listen. When you want to find out their            On what matters can there be no compromise.
        philosophy of life, I’ll start an argument and get it for         How to negotiate entry and sustained field relations.
        you . . .. You won’t have any trouble. You come in as             What services the researcher may provide.
                                                                          How to manage initial contacts with potential
        a friend.
                                                                          groups for study.
                                             (Whyte 1993: 292)

      As Whyte writes:
        My relationship with Doc changed rapidly . . .. At
        first he was simply a key informant – and also my
                                                                    1993 edition, Whyte reflects on the study with the
        sponsor. As we spent more time together, I ceased to
                                                                    question as to whether he exploited Doc (p. 362);
        treat him as a passive informant. I discussed with him
                                                                    it is a salutary reminder of the essential reciprocity
        quite frankly what I was trying to do, what problems
                                                                    that might be involved in conducting sensitive
        were puzzling me, and so on . . . so that Doc became,
        in a real sense, a collaborator in the research.
                                                                        In addressing issues of sampling and access, there
                                            (Whyte 1993: 301)
                                                                    are several points that arise from the discussion
      Whyte comments on how Doc was able to give                    (Box 5.1).
      him advice on how best to behave when meeting                     Much research stands or falls on the sampling.
      people as part of the research:                               These points reinforce our view that, rather than
                                                                    barring the research altogether, compromises may
        Go easy on that ‘who’, ‘what’, ‘why’, ‘when’, ‘where’
                                                                    have to be reached in sampling and access. It may
        stuff, Bill. You ask those questions and people will
                                                                    be better to compromise rather than to abandon
        clam up on you. If people accept you, you can just
                                                                    the research altogether.
        hang around, and you’ll learn the answers in the long
        run without even having to ask the questions’
                                          (Whyte 1993: 303)         Ethical issues in sensitive research
      Indeed Doc played a role in the writing of the                A difficulty arises in sensitive research in that
      research: ‘As I wrote, I showed the various parts to          researchers can be party to ‘guilty knowledge’ (De
      Doc and went over them in detail. His criticisms              Laine 2000) and have ‘dirty hands’ (Klockars
      were invaluable in my revision’ (p. 341). In his              1979) about deviant groups or members of a school
                                                                    ETHICAL ISSUES IN SENSITIVE RESEARCH           125

who may be harbouring counter-attitudes to those          deny access. The ethical issue of informed consent,

                                                                                                                     Chapter 5
prevailing in the school’s declared mission. Pushed       in this case, is violated in the interests of exposing
further, this means that researchers will need to         matters that are in the public interest.
decide the limits of tolerance, beyond which they            To the charge that this is akin to spy-
will not venture. For example, in Patrick’s (1973)        ing, Mitchell (1993: 46) makes it clear that there
study of a Glasgow gang, the researcher is witness        is a vast difference between covert research and
to a murder. Should he report the matter to the           spying:
police and, thereby, ‘blow his cover’, or remain
silent in order to keep contact with the gang,               ‘Spying is ideologically proactive, whereas
thereby breaking the law, which requires a murder                                         ı
                                                             research is ideologically na¨ve’ (Mitchell 1993:
to be reported?                                              46). Spies, he argues, seek to further a particular
   In interviewing students they may reveal                  value system or ideology; research seeks to
sensitive matters about themselves, their family,            understand rather than to persuade.
their teachers, and the researcher will need to              Spies have a sense of mission and try to achieve
decide whether and how to act on this kind of                certain instrumental ends, whereas research
information. What should the researcher do, for              has no such specific mission.
example, if, during the course of an interview with          Spies believe that they are morally superior
a teacher about the leadership of the headteacher,           to their subjects, whereas researchers have no
the interviewee indicates that the headteacher               such feelings; indeed, with reflexivity being
has had sexual relations with a parent, or has               so important, they are sensitive to how their
an alcohol problem? Does the researcher, in                  own role in the investigation may distort the
such cases, do nothing in order to gain research             research.
knowledge, or does the researcher act? What                  Spies are supported by institutions which train
is in the public interest – the protection of an             them to behave in certain ways of subterfuge,
individual participant’s private life, or the interests      whereas researchers have no such training.
of the researcher? Indeed Lee (1993: 139) suggests           Spies are paid to do the work, whereas
that some participants may even deliberately                 researchers often operate on a not-for-profit
engineer situations whereby the researcher gains             or individualistic basis.
‘guilty knowledge’ in order to test the researcher’s
affinities: ‘trust tests’.                                 On the other hand, not to gain informed consent
   Ethical issues are thrown into sharp relief in         could lead to participants feeling duped, very
sensitive educational research. The question of           angry, used and exploited, when the results of the
covert research rises to the fore, as the study           research are eventually published and they realize
of deviant or sensitive situations may require            that they have been studied without their approval
the researcher to go under cover in order to              consent.2 The researcher is seen as a predator (Lee
obtain data. Covert research may overcome                 1993: 157), using the research ‘as a vehicle for
‘problems of reactivity’ (Lee 1993: 143) wherein          status, income or professional advancement which
the research influences the behaviour of the               is denied to those studied’. As Lee (1993: 157)
participants (Hammersley and Atkinson 1983:               remarks, ‘it is not unknown for residents in some
71). It may also enable the researcher to obtain          ghetto areas of the United States to complain
insiders’ true views, for, without the cover of those     wryly that they have put dozens of students through
being researched not knowing that they are being          graduate school’. Further, the researched may have
researched, entry could easily be denied, and access      no easy right of reply; feel misrepresented by the
to important areas of understanding could be lost.        research; feel that they have been denied a voice;
This is particularly so in the case of researching        have wished not to be identified and their situation
powerful people who may not wish to disclose              put into the public arena; feel that they have been
information and who, therefore, may prevent or            exploited.

         The cloak of anonymity is often vital in sensitive   interest (e.g. a school’s failure to keep to proper
      research, such that respondents are entirely            accounting procedures) is brought to light? Should
      untraceable. This raises the issue of ‘deductive        the researcher follow up the matter privately,
      disclosure’ (Boruch and Cecil 1979), wherein it is      publicly, or not at all? If it is followed up
      possible to identify individuals (people, schools,      then certainly harm may come to the school’s
      departments etc.) in question by reconstructing         officers.
      and combining data. Researchers should guard               Ethical issues in the conduct of research are
      against this possibility. Where the details that are    thrown into sharp relief against a backdrop of
      presented could enable identification of a person        personal, institutional and societal politics, and
      (e.g. in a study of a school there may be only          the boundaries between public and private spheres
      one male teacher aged 50 who teaches biology,           are not only relative but also highly ambiguous.
      such that putting a name is unnecessary, as he          The ethical debate is heightened, for example
      will be identifiable), it may be incumbent on the        concerning the potential tension between the
      researcher not to disclose such details, so that        individual’s right to privacy versus the public’s
      readers, even if they wished to reassemble the          right to know and the concern not to damage
      details in order to identify the respondent, are        or harm individuals versus the need to serve the
      unable to do so.                                        public good. Because public and private spheres
         The researcher may wish to preserve confiden-         may merge, it is difficult, if not impossible, to
      tiality, but may also wish to be able to gather data    resolve such tensions straightforwardly (cf. Day
      from individuals on more than one occasion. In          1985; Lee 1993). As Walford (2001: 30) writes:
      this case a ‘linked file’ system (Lee 1993: 173) can     ‘the potential gain to public interest . . . was great.
      be employed. Here three files are kept; in the first      There would be some intrusion into the private
      file the data are held and arbitrary numbers are as-     lives of those involved, but this could be justified
      signed to each participant; the second file contains     in research on . . . an important policy issue’. The
      the list of respondents; the third file contains the     end justified the means.
      list of information necessary to be able to link the       These issues are felt most sharply if the
      arbitrarily assigned numbers from the first file to       research risks revealing negative findings. To
      the names of the respondents in the second, and         expose practices to research scrutiny may be like
      this third file is kept by a neutral ‘broker’, not the   taking the plaster off an open wound. What
      researcher. This procedure is akin to double-blind      responsibility to the research community does the
      clinical experiments, in which the researcher does      researcher have? If a negative research report is
      not know the names of those who are or are not          released, will schools retrench, preventing future
      receiving experimental medication or a placebo.         research in schools from being undertaken (a
      That this may be easier in respect of quantita-         particular problem if the researcher wishes to
      tive rather than qualitative data is acknowledged       return or wishes not to prevent further researchers
      by Lee (1993: 179).                                     from gaining access)? Whom is the researcher
         Clearly, in some cases, it is impossible for         serving – the public, the schools, the research
      individual people, schools and departments not          community? The sympathies of the researcher
      to be identified, for example schools may be highly      may be called into question here; politics and
      distinctive and, therefore, identifiable (Whitty         ethics may be uncomfortable bedfellows in such
      and Edwards 1994: 22). In such cases clearance          circumstances. Negative research data, such as
      may need to be obtained for the disclosure of           the negative hidden curriculum of training for
      information. This is not as straightforward as          conformity in schools (Morrison 2005a) may not
      it may seem. For example, a general principle           endear researchers to schools. This can risk stifling
      of educational research is that no individuals          educational research – it is simply not worth the
      should be harmed (non-maleficence), but what             personal or public cost. As Simons (2000: 45)
      if a matter that is in the legitimate public            writes: ‘the price is too high’.
                                                                                 RESEARCHING POWERFUL PEOPLE             127

   Further, Mitchell (1993: 54) writes that ‘timo-              may say nothing rather than criticize (Burgess

                                                                                                                           Chapter 5
rous social scientists may excuse themselves                    1993; Morrison 2001a; 2002b).
from the risk of confronting powerful, privileged,                 The field of ethics in sensitive research is
and cohesive groups that wish to obscure their                  different from ethics in everyday research in
actions and interests from public scrutiny’ (see                significance rather than range of focus. The same
also Lee 1993: 8). Researchers may not wish to                  issues as must be faced in all educational research
take the risk of offending the powerful or of                   are addressed here, and we advise readers to review
placing themselves in uncomfortable situations.                 Chapter 2 on ethics. However, sensitive research
As Simons and Usher (2000: 5) remark: ‘politics                 highlights particular ethical issues very sharply;
and ethics are inextricably entwined’.                          these are presented in Box 5.2.
   In private, students and teachers may criticize                 These are only introductory issues. We refer
their own schools, for example, in terms of                     the reader to Chapter 2 for further discussion of
management, leadership, work overload and stress,               these and other ethical issues. The difficulty with
but they may be reluctant to do so in public and,               ethical issues is that they are ‘situated’ (Simons
indeed, teachers who are on renewable contracts                 and Usher 2000), i.e. contingent on specific local
will not bite the hand that feeds them; they                    circumstances and situations. They have to be
                                                                negotiated and worked out in relation to the
                                                                specifics of the situation; universal guidelines may
                                                                help but they don’t usually solve the practical
Box 5.2                                                         problems, they have to be interpreted locally.
Ethical issues in sensitive research

      How does the researcher handle ‘guilty knowledge’         Researching powerful people
      and ‘dirty hands’?
                                                                A branch of sensitive research concerns that which
      Whose side is the researcher on? Does this need to
      be disclosed? What if the researcher is not on the        is conducted on, or with, powerful people, those
      side of the researched?                                   in key positions, or elite institutions. In education,
      When is covert research justified?                         for example, this would include headteachers and
      When is the lack of informed consent justified?            senior teachers, politicians, senior civil servants,
      Is covert research spying?
                                                                decision-makers, local authority officers and school
      How should the researcher overcome the charge of
      exploiting the participants (i.e. treating them as        governors. This is particularly the case in respect of
      objects instead of as subjects of research)?              research on policy and leadership issues (Walford
      How should the researcher address confidentiality          1994a: 3). Researching the powerful is an
      and anonymity?                                            example of ‘researching up’ rather than the more
      How should the balance be struck between the
                                                                conventional ‘researching down’ (e.g. researching
      individual’s right to privacy and the public’s right to
      know?                                                     children, teachers and student teachers).
      What is really in the public interest?                       What makes the research sensitive is that it is
      How to handle the situation where it is unavoidable       often dealing with key issues of policy generation
      to identify participants?                                 and decision-making, or issues about which there is
      What responsibility does the researcher have to the
                                                                high-profile debate and contestation, as issues of a
      research community, some of whom may wish to
      conduct further research in the field?                     politically sensitive nature. Policy-related research
      How does the researcher handle frightened or              is sensitive. This can be also one of the reasons
      threatened groups who may reveal little?                  why access is frequently refused. The powerful are
      What protections are in the research, for whom,           those who exert control to secure what they want
      and from what?
                                                                or can achieve, those with great responsibility and
      What obligations does the researcher have?
                                                                whose decisions have significant effects on large
                                                                numbers of people.

         Academic educational research on the powerful       powerful can produce difficulties which include
      may be unlike other forms of educational research      ‘misrepresentation of the research intention, loss
      in that confidentiality may not be able to be           of researcher control, mediation of the research
      assured. The participants are identifiable and          process, compromise and researcher dependence’.
      public figures. This may produce ‘problems of              Research with powerful people usually takes
      censorship and self-censorship’ (Walford 1994c:        place on their territory, under their conditions
      229). It also means that information given in          and agendas (a ‘distinctive civil service voice’: Fitz
      confidence and ‘off the record’ unfortunately           and Halpin 1994: 42), working within discourses
      may have to remain so. The issue raised in             set by the powerful (and, in part, reproduced by
      researching the powerful is the disclosure of          the researchers), and with protocols concerning
      identities, particularly if it is unclear what has     what may or may not be disclosed (e.g.
      been said ‘on the record’ and ‘off the record’ (Fitz   under a government’s Official Secrets Act or
      and Halpin 1994: 35–6).                                privileged information), within a world which
         Fitz and Halpin (1994) indicate that the            may be unfamiliar and, thereby, disconcerting
      government minister whom they interviewed              for researchers and with participants who may
      stated, at the start of the interview, what was        be overly assertive, and sometimes rendering the
      to be attributable. They also report that they         researcher as having to pretend to know less than
      used semi-structured interviews in their research      he or she actually knows. As Fitz and Halpin
      of powerful people, valuing both the structure and     (1994: 40) commented: ‘we glimpsed an unfamiliar
      the flexibility of this type of interview, and that     world that was only ever partially revealed’, and
      they gained permission to record the interviews        one in which they did not always feel comfortable.
      for later transcription, for the sake of a research    Similarly, Ball (1994b: 113) suggests that ‘we need
      record. They also used two interviewers for each       to recognize . . . the interview as an extension of
      session, one to conduct the main part of the           the ‘‘play of power’’ rather than separate from it,
      interview and the other to take notes and ask          merely a commentary upon it’, and that, when
      supplementary questions; having two interviewers       interviewing powerful people ‘the interview is
      present also enabled a post-interview cross-check      both an ethnographic . . . and a political event’.
      to be undertaken. Indeed having two questioners        As Walford (1994c) remarks:
      helped to negotiate the way through the interview
                                                               Those in power are well used to their ideas being
      in which advisers to the interviewee were also
                                                               taken notice of. They are well able to deal with
      present, to monitor the proceedings and interject
                                                               interviewers, to answer and avoid particular questions
      where deemed fitting, and to take notes (Fitz and
                                                               to suit their own ends, and to present their own role
      Halpin 1994: 38, 44, 47).
                                                               in events in a favourable light. They are aware of
         Fitz and Halpin (1994: 40) comment on
                                                               what academic research involves, and are familiar
      the considerable amount of gatekeeping that
                                                               with being interviewed and having their words tape-
      was present in researching the powerful, in
                                                               recorded. In sum, their power in the educational
      terms of access to people (with officers
                                                               world is echoed in the interview situation, and
      guarding entrances and administrators deciding
                                                               interviews pose little threat to their own positions.
      whether interviews will take place), places (‘elite
                                                                                                (Walford 1994c: 225)
      settings’), timing (and scarcity of time with
      busy respondents), ‘conventions that screen off           McHugh (1994: 55) comments that access to
      the routines of policy-making from the public          powerful people may take place not only through
      and the academic gaze’, conditional access and         formal channels but also through intermediaries
      conduct of the research (‘boundary maintenance’)       who introduce researchers to them. Here his own
      monitoring and availability (Fitz and Halpin           vocation as a priest helped him to gain access
      1994: 48–9). Gewirtz and Ozga (1994: 192–3)            to powerful Christian policy-makers and, as he
      suggest that gatekeeping in researching the            was advised, ‘if you say whom you have met,
                                                                          RESEARCHING POWERFUL PEOPLE                  129

they’ll know you are not a way-out person who           of libel actions if public figures are named. He

                                                                                                                         Chapter 5
will distort what they say’ (McHugh 1994: 56).          asks (1994b: 84) ‘to what extent is it right
Access is a significant concern in researching the       to allow others to believe that you agree with
powerful, particularly if the issues being researched   them?’, even if you do not? Should the researcher’s
are controversial or contested. Walford (1994c:         own political, ideological or religious views be
222, 223) suggests that it can be eased through         declared? As Mickelson (1994: 147) states: ‘I
informal and personal ‘behind the scenes’ contacts:     was not completely candid when I interviewed
‘the more sponsorship that can be obtained, the         these powerful people. I am far more genuine
better’, be it institutional or personal. Access can    and candid when I am interviewing non-powerful
be eased if the research is seen to be ‘harmless’       people’. Deem (1994: 156) reports that she and her
(Walford 1994c: 223); in this respect Walford           co-researcher encountered ‘resistance and access
reports that female researchers may be at an            problems in relation to our assumed ideological
advantage in that they are viewed as more harmless      opposition to Conservative government education
and non-threatening. Walford also makes the             reforms’, where access might be blocked ‘on the
point that ‘persistence pays’ (p. 224); as he writes    grounds that ours was not a neutral study’.
elsewhere (Walford 2001: 31), ‘access is a process         Mickelson (1994: 147) takes this further in
and not a once-only decision’.                          identifying an ethical dilemma when ‘at times, the
   McHugh (1994) also reports the need for              powerful have uttered abhorrent comments in the
meticulous preparation for an interview with the        course of the interview’. Should the researcher say
powerful person, to understand the full picture         nothing, thereby tacitly condoning the speaker’s
and to be as fully informed as the interviewee,         comments, or speak out, thereby risking closing
in terms of facts, information and terminology,         the interview? She contends that, in retrospect,
so that it is an exchange between the informed          she wished that she had challenged these views,
rather than an airing of ignorance, i.e. to do          and had been more assertive (Mickelson 1994:
one’s homework. He also states the need for the         148). Walford (2001) reports the example of an
interview questions to be thoroughly planned and        interview with a church minister whose views
prepared, with very careful framing of questions.       included ones with which he disagreed:
McHugh (1994: 60, 62) suggests that during the
                                                          AIDS is basically a homosexual disease . . . and is
interview it is important for the interviewer not
                                                          doing a very effective job of ridding the population
only to be as flexible as possible, to follow the
                                                          of undesirables. In Africa it’s basically a non-existent
train of thought of the respondent, but also to be
                                                          disease in many places . . . . If you’re a woolly woofter,
persistent if the interviewee does not address the
                                                          you get what you deserve . . . . I would never employ
issue. However, he reminds us that ‘an interview
                                                          a homosexual to teach at my school.
is of course not a courtroom’ (p. 62) and so
                                                                                              (Walford 2001: 137)
tact, diplomacy and – importantly – empathy are
essential. Diplomacy in great measure is necessary         In researching powerful people Mickelson
when tackling powerful people about issues that         (1994: 132) observes that they are rarely women,
might reveal their failure or incompetence, and         yet researchers are often women. This gender
powerful people may wish to exercise control over       divide might prove problematic. Deem (1994:
which questions they answer. Preparation for the        157) reports that, as a woman, she encountered
conduct as well as the content of the interview         greater difficulty in conducting research than did
is vital.                                               her male colleague, even though, in fact, she
   There are difficulties in reporting sensitive         held a more senior position than him. On the
research with the powerful, as charges of bias may      other hand, she reports that males tended to be
be difficult to avoid, not least because research        more open with female than male researchers,
reports and publications are placed in the public       as females researchers were regarded as less
domain. Walford (2001: 141) indicates the risk          important. Gewirtz and Ozga (1994) report:

      Box 5.3                                                     suggest that while short questions may be useful
      Researching powerful people                                 for gathering information about attitudes, longer
                                                                  questions are more suitable for asking questions
           What renders the research sensitive.
                                                                  about behaviour, and can include examples to
           How to gain and sustain access to powerful people.     which respondents may wish to respond. Longer
           How much the participants are likely to disclose or    questions may reduce the under-reporting of the
           withhold.                                              frequency of behaviour addressed in sensitive
           What is on and off the record.                         topics (for example, the use of alcohol or
           How to prepare for interviews with powerful
                                                                  medication by stressed teachers). On the other
           How to probe and challenge powerful people.            hand, the researcher has to be cautious to avoid
           How to conduct interviews that balance the             tiring, emotionally exhausting or stressing the
           interviewer’s agenda and the interviewee’s agenda      participant by a long question or interview.
           and frame of reference.                                   Lee (1993: 78) advocates using familiar words
           How to reveal the researcher’s own knowledge,
                                                                  in questions as these can reduce a sense
           preparation and understanding of the key issues.
           The status of the researcher vis-` -vis the
                                            a                     of threat in addressing sensitive matters and
           participants.                                          help the respondent to feel more relaxed.
           Who should conduct interviews with powerful            He also suggests the use of ‘vignettes’: ‘short
           people.                                                descriptions of a person or a social situation
           How neutral and accepting the researcher should be
                                                                  which contain precise references to what are
           with the participant.
           Whether to identify the participants in the            thought to be the most important factors in the
           reporting.                                             decision-making or judgement-making processes
           How to balance the public’s right to know and the      of respondents’ (Lee 1993: 79). These can not
           individual’s right to privacy.                         only encapsulate concretely the issues under study,
           What is in the public interest.
                                                                  but also deflect attention away from personal
                                                                  sensitivities by projecting them onto another
                                                                  external object – the case or vignette – and the
                                                                  respondent can be asked to react to them
                                                                  personally, e.g. ‘What would you do in this
        we felt [as researchers] that we were viewed as women
        in very stereotypical ways, which included being seen
                                                                     Researchers investigating sensitive topics have
        as receptive and supportive, and that we were obliged
                                                                  to be acutely percipient of the situation
        to collude, to a degree, with that version of ourselves
                                                                  themselves. For example, their non-verbal
        because it was productive of the project.
                                                                  communication may be critical in interviews.
                                 (Gewirtz and Ozga 1994: 196)
                                                                  They must, therefore, give no hint of judgement,
      In approaching researching powerful people, then,           support or condemnation. They must avoid
      it is wise to consider several issues. These are set        counter-transference (projecting the researchers’
      out in Box 5.3.                                             own views, values, attitudes biases, background
                                                                  onto the situation). Interviewer effects are
                                                                  discussed in Chapter 16 in connection with
      Asking questions                                            sensitive research; these effects concern the
      In asking questions in research, Sudman and                 characteristics of the researcher (e.g. sex,
      Bradburn (1982: 50–1) suggest that open                     race, age, status, clothing, appearance, rapport,
      questions may be preferable to closed questions             background, expertise, institutional affiliation,
      and long questions may be preferable to short               political affiliation, type of employment or
      questions. Both of these enable respondents to              vocation, e.g. a priest). Females may feel more
      answer in their own words, which might be                   comfortable being interviewed by a female; males
      more suitable for sensitive topics. Indeed they             may feel uncomfortable being interviewed by a
                                                                                           CONCLUSION        131

female; powerful people may feel insulted by              the interviewer power over the respondent and

                                                                                                               Chapter 5
being interviewed by a lowly, novice research             make the respondent feel vulnerable; (e) what
assistant. Interviewer effects also concern the           the interviewer should do with information
expectations that the interviewers may have               that may act against the interests of the
of the interview (Lee 1993: 99). For example,             people who gave it (e.g. if some groups in
a researcher may feel apprehensive about, or              society say that they are not clever enough
uncomfortable with, an interview about a sensitive        to handle higher or further education); and
matter. Bradburn and Sudman (1979, in Lee                 (f) how to handle the conduct of the interview
1993: 101) report that interviewers who did not           (e.g. conversational, formal, highly structured,
anticipate difficulties in the interview achieved a        highly directed).
5–30 per cent higher level of reporting on sensitive      Handling the conditions under which the
topics than those who anticipated difficulties. This       exchange takes place Lee (1993: 112) suggests
suggests the need for interviewer training.               that interviews on sensitive matters should
   Lee (1993: 102–14) suggests several issues to be       ‘have a one-off character’, i.e. the respondent
addressed in conducting sensitive interviews:             should feel that the interviewer and the
                                                          interviewee may never meet again. This can
   How to approach the topic (in order to                 secure trust, and can lead to greater disclosure
   prevent participants’ inhibitions and to help          than in a situation where a closer relationship
   them address the issue in their preferred way).        between interviewer and interviewee exists.
   Here the advice is to let the topic ‘emerge            On the other hand, this does not support
   gradually over the course of the interview’ (Lee       the development of a collaborative research
   1993: 103) and to establish trust and informed         relationship (Lee 1993: 113).
                                                       Much educational research is more or less
   How to deal with contradictions, complexities
                                                       sensitive; it is for the researcher to decide how
   and emotions (which may require training and
                                                       to approach the issue of sensitivities and how
   supervision of interviewers); how to adopt an
                                                       to address their many forms, allegiances, ethics,
   accepting and non-judgemental stance, how to
                                                       access, politics and consequences.
   handle respondents who may not be people
   whom interviewers particularly like or with
   whom they agree).
   How to handle the operation of power and
   control in the interview: (a) where differences     In approaching educational research, our advice
   of power and status operate, where the              is to consider it to be far from a neat, clean,
   interviewer has greater or lesser status than       tidy, unproblematic and neutral process, but
   the respondent and where there is equal status      to regard it as shot through with actual and
   between the interviewer and the respondent;         potential sensitivities. With this in mind we
   (b) how to handle the situation where the           have resisted the temptation to provide a list
   interviewer wants information but is in no          of sensitive topics, as this could be simplistic
   position to command that this be given              and overlook the fundamental issue which is
   and where the respondent may or may                 that it is the social context of the research that
   not wish to disclose information; (c) how           makes the research sensitive. What may appear
   to handle the situation wherein powerful            to the researcher to be a bland and neutral study
   people use the interview as an opportunity for      can raise deep sensitivities in the minds of the
   lengthy and perhaps irrelevant self-indulgence;     participants. We have argued that it is these that
   (d) how to handle the situation in which the        often render the research sensitive rather than
   interviewer, by the end of the session, has         the selection of topics of focus. Researchers have
   information that is sensitive and could give        to consider the likely or possible effects of the

      research project, conduct, outcomes, reporting                     2000), i.e. contingent on particular situations
      and dissemination not only on themselves but                       rather than largely on ethical codes and guidelines.
      also on the participants, on those connected to                    In this respect sensitive educational research
      the participants and on those affected by, or with                 is like any other research, but sharper in the
      a stakeholder interest in, the research (i.e. to                   criticality of ethical issues. Also, behind many of
      consider ‘consequential validity’: the effects of the              these questions of sensitivity lurks the nagging
      research). This suggests that it is wise to be cautious            issue of power: who has it, who does not,
      and to regard all educational research as potentially              how it circulates around research situations (and
      sensitive. There are several questions that can be                 with what consequences), and how it should be
      asked by researchers, in their planning, conduct,                  addressed. Sensitive educational research is often
      reporting and dissemination of their studies, and                  as much a power play as it is substantive. We advise
      we present these in Box 5.4.                                       researchers to regard most educational research as
         These questions reinforce the importance of                     involving sensitivities; these need to be identified
      regarding ethics as ‘situated’ (Simons and Usher                   and addressed.

      Box 5.4
      Key questions in considering sensitive educational research

           What renders the research sensitive?
           What are the obligations of the researcher, to whom, and how will these be addressed? How do these obligations
           manifest themselves?
           What is the likely effect of this research (at all stages) to be on participants (individuals and groups), stakeholders, the
           researcher, the community? Who will be affected by the research, and how?
           Who is being discussed and addressed in the research?
           What rights of reply and control do participants have in the research?
           What are the ethical issues that are rendered more acute in the research?
           Over what matters in the planning, focus, conduct, sampling, instrumentation, methodology, reliability, analysis,
           reporting and dissemination might the researcher have to compromise in order to effect the research? On what can
           there be compromise? On what can there be no compromise?
           What securities, protections (and from what), liabilities and indemnifications are there in the research, and for whom?
           How can these be addressed?
           Who is the research for? Who are the beneficiaries of the research? Who are the winners and losers in the research
           (and about what issues)?
           What are the risks and benefits of the research, and for whom? What will the research ‘deliver’ and do?
           Should researchers declare their own values, and challenge those with which they disagree or consider to be abhorrent?
           What might be the consequences, repercussions and backlash from the research, and for whom?
           What sanctions might there be in connection with the research?
           What has to be secured in a contractual agreement, and what is deliberately left out?
           What guarantees must and should the researcher give to the participants?
           What procedures for monitoring and accountability must there be in the research?
           What must and must not, should and should not, may or may not, could or could not be disclosed in the research?
           Should the research be covert, overt, partially overt, partially covert, honest in its disclosure of intentions?
           Should participants be identifiable and identified? What if identification is unavoidable?
           How will access and sampling be secured and secure respectively?
           How will access be sustained over time?
           Who are the gatekeepers and how reliable are they?
6        Validity and reliability

There are many different types of validity and reli-     participants approached, the extent of triangula-
ability. Threats to validity and reliability can never   tion and the disinterestedness or objectivity of the
be erased completely; rather the effects of these        researcher (Winter 2000). In quantitative data va-
threats can be attenuated by attention to validity       lidity might be improved through careful sampling,
and reliability throughout a piece of research.          appropriate instrumentation and appropriate sta-
   This chapter discusses validity and reliability in    tistical treatments of the data. It is impossible
quantitative and qualitative, naturalistic research.     for research to be 100 per cent valid; that is the
It suggests that both of these terms can be              optimism of perfection. Quantitative research pos-
applied to these two types of research, though how       sesses a measure of standard error which is inbuilt
validity and reliability are addressed in these two      and which has to be acknowledged. In qualita-
approaches varies. Finally validity and reliability      tive data the subjectivity of respondents, their
are addressed, using different instruments for data      opinions, attitudes and perspectives together con-
collection. It is suggested that reliability is a        tribute to a degree of bias. Validity, then, should be
necessary but insufficient condition for validity         seen as a matter of degree rather than as an absolute
in research; reliability is a necessary precondition     state (Gronlund 1981). Hence at best we strive to
of validity, and validity may be a sufficient but         minimize invalidity and maximize validity.
not necessary condition for reliability. Brock-             There are several different kinds of va-
Utne (1996: 612) contends that the widely                lidity (see
held view that reliability is the sole preserve          9780415368780 – Chapter 6, file 6.2. ppt):
of quantitative research has to be exploded,
and this chapter demonstrates the significance               content validity
of her view.                                                criterion-related validity
                                                            construct validity
                                                            internal validity
Defining validity
                                                            external validity
Validity is an important key to effective re-               concurrent validity
search. If a piece of research is invalid then it           face validity
is worthless. Validity is thus a requirement for            jury validity
both quantitative and qualitative/naturalistic re-          predictive validity
search (see             consequential validity
9780415368780 – Chapter 6, file 6.1. ppt).                   systemic validity
   While earlier versions of validity were based on         catalytic validity
the view that it was essentially a demonstration            ecological validity
that a particular instrument in fact measures what          cultural validity
it purports to measure, more recently validity has          descriptive validity
taken many forms. For example, in qualitative data          interpretive validity
validity might be addressed through the honesty,            theoretical validity
depth, richness and scope of the data achieved, the         evaluative validity.

         It is not our intention in this chapter to              sufficiently complex instrument to understand
      discuss all of these terms in depth. Rather the            human life is another human (Lave and Kvale
      main types of validity will be addressed. The              1995: 220), but that this risks human error in
      argument will be made that, while some of                  all its forms.
      these terms are more comfortably the preserve of           There should be holism in the research.
      quantitative methodologies, this is not exclusively        The researcher – rather than a research
      the case. Indeed, validity is the touchstone of            tool – is the key instrument of research.
      all types of educational research. That said, it           The data are descriptive.
      is important that validity in different research           There is a concern for processes rather than
      traditions is faithful to those traditions; it would       simply with outcomes.
      be absurd to declare a piece of research invalid           Data are analysed inductively rather than using
      if it were not striving to meet certain kinds              a priori categories.
      of validity, e.g. generalizability, replicability and      Data are presented in terms of the respondents
      controllability. Hence the researcher will need            rather than researchers.
      to locate discussions of validity within the               Seeing and reporting the situation should be
      research paradigm that is being used. This is              through the eyes of participants – from the
      not to suggest, however, that research should be           native’s point of view (Geertz 1974).
      paradigm-bound, that is a recipe for stagnation            Respondent validation is important.
      and conservatism. Nevertheless, validity must be           Catching meaning and intention are essential.
      faithful to its premises and positivist research
      has to be faithful to positivist principles, for           Indeed Maxwell (1992) argues that qualitative
      example:                                                researchers need to be cautious not to be
                                                              working within the agenda of the positivists in
         controllability                                      arguing for the need for research to demonstrate
         replicability                                        concurrent, predictive, convergent, criterion-
         predictability                                       related, internal and external validity. The
         the derivation of laws and universal statements      discussion below indicates that this need not be
         of behaviour                                         so. He argues, with Guba and Lincoln (1989),
         context-freedom                                      for the need to replace positivist notions of
         fragmentation and atomization of research            validity in qualitative research with the notion
         randomization of samples                             of authenticity. Maxwell (1992), echoing Mishler
         observability.                                       (1990), suggests that ‘understanding’ is a more
        By way of contrast, naturalistic research has         suitable term than ‘validity’ in qualitative research.
      several principles (Lincoln and Guba 1985;              We, as researchers, are part of the world
      Bogdan and Biklen, 1992):                               that we are researching, and we cannot be
                                                              completely objective about that, hence other
         The natural setting is the principal source of       people’s perspectives are equally as valid as our
         data.                                                own, and the task of research is to uncover
         Context-boundedness and ‘thick description’          these. Validity, then, attaches to accounts, not
         are important.                                       to data or methods (Hammersley and Atkinson
         Data are socially situated, and socially and         1983); it is the meaning that subjects give to
         culturally saturated.                                data and inferences drawn from the data that
         The researcher is part of the researched world.      are important. ‘Fidelity’ (Blumenfeld-Jones 1995)
         As we live in an already interpreted world, a        requires the researcher to be as honest as possible
         doubly hermeneutic exercise (Giddens 1979) is        to the self-reporting of the researched.
         necessary to understand others’ understandings          The claim is made (Agar 1993) that, in
         of the world; the paradox here is that the most      qualitative data collection, the intensive personal
                                                                                        DEFINING VALIDITY       135

involvement and in-depth responses of individuals          Theoretical validity (the theoretical construc-

                                                                                                                  Chapter 6
secure a sufficient level of validity and reliability.      tions that the researcher brings to the research,
This claim is contested by Hammersley (1992:               including those of the researched): theory here
144) and Silverman (1993: 153), who argue that             is regarded as explanation. Theoretical validity
these are insufficient grounds for validity and             is the extent to which the research explains
reliability, and that the individuals concerned            phenomena; in this respect is it akin to con-
have no privileged position on interpretation. (Of         struct validity (discussed below); in theoretical
course, neither are actors ‘cultural dopes’ who            validity the constructs are those of all the
need a sociologist or researcher to tell them              participants.
what is ‘really’ happening!) Silverman (1993)              Generalizability (the view that the theory
argues that, while immediacy and authenticity              generated may be useful in understanding
make for interesting journalism, ethnography               other similar situations): generalizing here
must have more rigorous notions of validity                refers to generalizing within specific groups
and reliability. This involves moving beyond               or communities, situations or circumstances
selecting data simply to fit a preconceived or ideal        validly and, beyond, to specific outsider
conception of the phenomenon or because they               communities, situations or circumstances
are spectacularly interesting (Fielding and Fielding       (external validity); internal validity has greater
1986). Data selected must be representative of the         significance here than external validity.
sample, the whole data set, the field, i.e. they            Evaluative validity (the application of an eval-
must address content, construct and concurrent             uative, judgemental of that which is being
validity.                                                  researched, rather than a descriptive, explana-
   Hammersley (1992: 50–1) suggests that validity          tory or interpretive framework). Clearly this
in qualitative research replaces certainty with            resonates with critical-theoretical perspectives,
confidence in our results, and that, as reality is in-      in that the researcher’s own evaluative agenda
dependent of the claims made for it by researchers,        might intrude.
our accounts will be only representations of that
reality rather than reproductions of it.                  Both qualitative and quantitative methods can
   Maxwell (1992) argues for five kinds of validity      address internal and external validity.
in qualitative methods that explore his notion of
                                                        Internal validity
   Descriptive validity (the factual accuracy of the    Internal validity seeks to demonstrate that the
   account, that it is not made up, selective or        explanation of a particular event, issue or set
   distorted): in this respect validity subsumes        of data which a piece of research provides can
   reliability; it is akin to Blumenfeld-Jones’s        actually be sustained by the data. In some degree
   (1995) notion of ‘truth’ in research – what          this concerns accuracy, which can be applied to
   actually happened (objectively factual).             quantitative and qualitative research. The findings
   Interpretive validity (the ability of the research   must describe accurately the phenomena being
   to catch the meaning, interpretations, terms,        researched.
   intentions that situations and events, i.e. data,       In ethnographic research internal validity can
   have for the participants/subjects themselves,       be addressed in several ways (LeCompte and
   in their terms): it is akin to Blumenfeld-Jones’s    Preissle 1993: 338):
   (1995) notion of ‘fidelity’ – what it means to
   the researched person or group (subjectively            using low-inference descriptors
   meaningful); interpretive validity has no               using multiple researchers
   clear counterpart in experimental/positivist            using participant researchers
   methodologies.                                          using peer examination of data

         using mechanical means to record, store and             clarity on the kinds of claim made from
         retrieve data.                                          the research (e.g. definitional, descriptive,
                                                                 explanatory, theory generative).
      In ethnographic, qualitative research there are
      several overriding kinds of internal validity              Lincoln and Guba (1985: 219, 301) suggest that
      (LeCompte and Preissle 1993: 323–4):                    credibility in naturalistic inquiry can be addressed
         confidence in the data
         the authenticity of the data (the ability of the        Prolonged engagement in the field.
         research to report a situation through the eyes         Persistent observation: in order to establish the
         of the participants)                                    relevance of the characteristics for the focus.
         the cogency of the data                                 Triangulation: of methods, sources, investiga-
         the soundness of the research design                    tors and theories.
         the credibility of the data                             Peer debriefing: exposing oneself to a dis-
         the auditability of the data                            interested peer in a manner akin to
         the dependability of the data                           cross-examination, in order to test honesty,
         the confirmability of the data.                          working hypotheses and to identify the next
      LeCompte and Preissle (1993) provide greater               steps in the research.
      detail on the issue of authenticity, arguing for the       Negative case analysis: in order to establish a
      following:                                                 theory that fits every case, revising hypotheses
         Fairness: there should be a complete and                Member checking: respondent validation, to
         balanced representation of the multiple                 assess intentionality, to correct factual errors,
         realities in, and constructions of, a situation.        to offer respondents the opportunity to add
         Ontological authenticity: the research should           further information or to put information on
         provide a fresh and more sophisticated                  record; to provide summaries and to check the
         understanding of a situation, e.g. making               adequacy of the analysis.
         the familiar strange, a significant feature in
         reducing ‘cultural blindness’ in a researcher,       Whereas in positivist research history and
         a problem which might be encountered in              maturation are viewed as threats to the validity
         moving from being a participant to being an          of the research, ethnographic research simply
         observer (Brock-Utne 1996: 610).                     assumes that this will happen; ethnographic
         Educative authenticity: the research should          research allows for change over time – it builds
         generate a new appreciation of these                 it in. Internal validity in ethnographic research
         understandings.                                      is also addressed by the reduction of observer
         Catalytic authenticity: the research gives rise to   effects by having the observers sample both widely
         specific courses of action.                           and staying in the situation for such a long
         Tactical authenticity: the research should bring     time that their presence is taken for granted.
         benefit to all involved – the ethical issue of        Further, by tracking and storing information
         ‘beneficence’.                                        clearly, it is possible for the ethnographer
                                                              to eliminate rival explanations of events and
        Hammersley (1992: 71) suggests that internal          situations.
      validity for qualitative data requires attention to
         plausibility and credibility
                                                              External validity
         the kinds and amounts of evidence required
         (such that the greater the claim that is being       External validity refers to the degree to which
         made, the more convincing the evidence has           the results can be generalized to the wider
         to be for that claim)                                population, cases or situations. The issue of
                                                                                         DEFINING VALIDITY       137

generalization is problematical. For positivist          in qualitative research. Positivist researchers, they

                                                                                                                   Chapter 6
researchers generalizability is a sine qua non, while    argue, are more concerned to derive universal
this is attenuated in naturalistic research. For         statements of general social processes rather than
one school of thought, generalizability through          to provide accounts of the degree of commonality
stripping out contextual variables is fundamental,       between various social settings (e.g. schools and
while, for another, generalizations that say             classrooms). Bogdan and Biklen (1992) are more
little about the context have little that is             interested not with the issue of whether their
useful to say about human behaviour (Schofield            findings are generalizable in the widest sense but
1990). For positivists variables have to be              with the question of the settings, people and
isolated and controlled, and samples randomized,         situations to which they might be generalizable.
while for ethnographers human behaviour is                  In naturalistic research threats to external
infinitely complex, irreducible, socially situated        validity include (Lincoln and Guba 1985:
and unique.                                              189, 300):
   Generalizability in naturalistic research is
                                                            selection effects: where constructs selected in
interpreted as comparability and transferability
                                                            fact are only relevant to a certain group
(Lincoln and Guba 1985; Eisenhart and Howe
                                                            setting effects: where the results are largely a
1992: 647). These writers suggest that it is
                                                            function of their context
possible to assess the typicality of a situation – the
                                                            history effects: where the situations have
participants and settings, to identify possible
                                                            been arrived at by unique circumstances and,
comparison groups, and to indicate how data might
                                                            therefore, are not comparable
translate into different settings and cultures (see
                                                            construct effects: where the constructs being
also LeCompte and Preissle 1993: 348). Schofield
                                                            used are peculiar to a certain group.
(1990: 200) suggests that it is important in
qualitative research to provide a clear, detailed
                                                         Content validity
and in-depth description so that others can decide
the extent to which findings from one piece of            To demonstrate this form of validity the
research are generalizable to another situation,         instrument must show that it fairly and
i.e. to address the twin issues of comparability         comprehensively covers the domain or items
and translatability. Indeed, qualitative research        that it purports to cover. It is unlikely that
can be generalizable (Schofield 1990: 209), by            each issue will be able to be addressed in its
studying the typical (for its applicability to other     entirety simply because of the time available or
situations – the issue of transferability: LeCompte      respondents’ motivation to complete, for example,
and Preissle 1993: 324) and by performing                a long questionnaire. If this is the case, then
multi-site studies (e.g. Miles and Huberman              the researcher must ensure that the elements of
1984), though it could be argued that this is            the main issue to be covered in the research
injecting a degree of positivism into non-positivist     are both a fair representation of the wider issue
research. Lincoln and Guba (1985: 316) caution           under investigation (and its weighting) and that
the naturalistic researcher against this; they argue     the elements chosen for the research sample
that it is not the researcher’s task to provide          are themselves addressed in depth and breadth.
an index of transferability; rather, they suggest,       Careful sampling of items is required to ensure their
researchers should provide sufficiently rich data         representativeness. For example, if the researcher
for the readers and users of research to determine       wished to see how well a group of students could
whether transferability is possible. In this respect     spell 1,000 words in French but decided to have a
transferability requires thick description.              sample of only 50 words for the spelling test, then
   Bogdan and Biklen (1992: 45) argue that               that test would have to ensure that it represented
generalizability, construed differently from its         the range of spellings in the 1,000 words – maybe
usage in positivist methodologies, can be addressed      by ensuring that the spelling rules had all been

      included or that possible spelling errors had been     then stipulate the interpretation that will be
      covered in the test in the proportions in which        used.
      they occurred in the 1,000 words.                         In qualitative/ethnographic research construct
                                                             validity must demonstrate that the categories that
                                                             the researchers are using are meaningful to the
                                                             participants themselves (Eisenhart and Howe 1992:
      Construct validity
                                                             648), i.e. that they reflect the way in which
      A construct is an abstract; this separates it          the participants actually experience and construe
      from the previous types of validity which              the situations in the research, that they see the
      dealt in actualities – defined content. In this         situation through the actors’ eyes.
      type of validity agreement is sought on the               Campbell and Fiske (1959), Brock-Utne (1996)
      ‘operationalized’ forms of a construct, clarifying     and Cooper and Schindler (2001) suggest that
      what we mean when we use this construct.               construct validity is addressed by convergent and
      Hence in this form of validity the articulation        discriminant techniques. Convergent techniques
      of the construct is important; is the researcher’s     imply that different methods for researching the
      understanding of this construct similar to that        same construct should give a relatively high
      which is generally accepted to be the construct?       inter-correlation, while discriminant techniques
      For example, let us say that the researcher wished     suggest that using similar methods for researching
      to assess a child’s intelligence (assuming, for the    different constructs should yield relatively low
      sake of this example, that it is a unitary quality).   inter-correlations, i.e. that the construct in
      The researcher could say that he or she construed      question is different from other potentially similar
      intelligence to be demonstrated in the ability to      constructs. Such discriminant validity can also
      sharpen a pencil. How acceptable a construction of     be yielded by factor analysis, which clusters
      intelligence is this? Is not intelligence something    together similar issues and separates them from
      else (e.g. that which is demonstrated by a high        others.
      result in an intelligence test)?
         To establish construct validity the researcher
                                                             Ecological validity
      would need to be assured that his or her
      construction of a particular issue agreed with         In quantitative, positivist research variables are
      other constructions of the same underlying issue,      frequently isolated, controlled and manipulated
      e.g. intelligence, creativity, anxiety, motivation.    in contrived settings. For qualitative, naturalistic
      This can be achieved through correlations with         research a fundamental premise is that the
      other measures of the issue or by rooting the          researcher deliberately does not try to manipulate
      researcher’s construction in a wide literature         variables or conditions, that the situations in the
      search which teases out the meaning of a particular    research occur naturally. The intention here is
      construct (i.e. a theory of what that construct        to give accurate portrayals of the realities of
      is) and its constituent elements. Demonstrating        social situations in their own terms, in their
      construct validity means not only confirming the        natural or conventional settings. In education,
      construction with that given in relevant literature,   ecological validity is particularly important and
      but also looking for counter-examples which might      useful in charting how policies are actually
      falsify the researcher’s construction. When the        happening ‘at the chalk face’ (Brock-Utne 1996:
      confirming and refuting evidence is balanced, the       617). For ecological validity to be demonstrated
      researcher is in a position to demonstrate construct   it is important to include and address in the
      validity, and can stipulate what he or she takes       research as many characteristics in, and factors
      this construct to be. In the case of conflicting        of, a given situation as possible. The difficulty
      interpretations of a construct, the researcher         for this is that the more characteristics are
      might have to acknowledge that conflict and             included and described, the more difficult it
                                                                                           DEFINING VALIDITY        139

is to abide by central ethical tenets of much                 How do researchers in the target culture deal

                                                                                                                      Chapter 6
research – non-traceability, anonymity and non-               with the issues related to the research question
identifiability.                                               (including their method and findings)?
                                                              Are appropriate gatekeepers and informants
Cultural validity                                             Are the research design and research
A type of validity related to ecological validity             instruments ethical and appropriate according
is cultural validity (Morgan 1999). This is                   to the standards of the target culture?
particularly an issue in cross-cultural, intercultural        How do members of the target culture define
and comparative kinds of research, where the                  the salient terms of the research?
intention is to shape research so that it is                  Are documents and other information trans-
appropriate to the culture of the researched,                 lated in a culturally appropriate way?
and where the researcher and the researched are               Are the possible results of the research of
members of different cultures. Cultural validity              potential value and benefit to the target
is defined as ‘the degree to which a study is                  culture?
appropriate to the cultural setting where research            Does interpretation of the results include the
is to be carried out’ (Joy 2003: 1). Cultural                 opinions and views of members of the target
validity, Morgan (1999) suggests, applies at all              culture?
stages of the research, and affects its planning,             Are the results made available to members of
implementation and dissemination. It involves a               the target culture for review and comment?
degree of sensitivity to the participants, cultures           Does the researcher accurately and fairly
and circumstances being studied. Morgan (2005)                communicate the results in their cultural
writes that                                                   context to people who are not members of
                                                              the target culture?
  cultural validity entails an appreciation of the
  cultural values of those being researched. This
  could include: understanding possibly different          Catalytic validity
  target culture attitudes to research; identifying        Catalytic validity embraces the paradigm of critical
  and understanding salient terms as used in the           theory discussed in Chapter 1. Put neutrally,
  target culture; reviewing appropriate target language    catalytic validity simply strives to ensure that
  literature; choosing research instruments that are       research leads to action. However, the story does
  acceptable to the target participants; checking          not end there, for discussions of catalytic validity
  interpretations and translations of data with native     are substantive; like critical theory, catalytic
  speakers; and being aware of one’s own cultural filters   validity suggests an agenda. Lather (1986, 1991)
  as a researcher.                                         and Kincheloe and McLaren (1994) suggest that
                                      (Morgan 2005: 1)     the agenda for catalytic validity is to help
   Joy (2003: 1) presents twelve important ques-           participants to understand their worlds in order
tions that researchers in different cultural contexts      to transform them. The agenda is explicitly
may face, to ensure that research is culture-fair and      political, for catalytic validity suggests the need
culturally sensitive:                                      to expose whose definitions of the situation are
                                                           operating in the situation. Lincoln and Guba
   Is the research question understandable and of          (1986) suggest that the criterion of ‘fairness’ should
   importance to the target group?                         be applied to research, meaning that it should
   Is the researcher the appropriate person to             not only augment and improve the participants’
   conduct the research?                                   experience of the world, but also improve the
   Are the sources of the theories that the research       empowerment of the participants. In this respect
   is based on appropriate for the target culture?         the research might focus on what might be (the

      leading edge of innovations and future trends)            the research and the action-related consequences
      and what could be (the ideal, possible futures)           of the research are both legitimate and fulfilled.
      (Schofield 1990: 209).                                     Clearly, once the research is in the public domain,
         Catalytic validity – a major feature in femi-          the researcher has little or no control over the
      nist research which, Usher (1996) suggests, needs         way in which it is used. However, and this is
      to permeate all research – requires solidarity in         often a political matter, research should not be
      the participants, an ability of the research to           used in ways in which it was not intended to be
      promote emancipation, autonomy and freedom                used, for example by exceeding the capability of
      within a just, egalitarian and democratic soci-           the research data to make claims, by acting on
      ety (Masschelein 1991), to reveal the distortions,        the research in ways that the research does not
      ideological deformations and limitations that re-         support (e.g. by using the research for illegitimate
      side in research, communication and social struc-         epistemic support), by making illegitimate claims
      tures (see also LeCompte and Preissle 1993).              by using the research in unacceptable ways (e.g.
      Validity, it is argued (Mishler 1990; Scheurich           by selection, distortion) and by not acting on the
      1996), is no longer an ahistorical given, but             research in ways that were agreed, i.e. errors of
      contestable, suggesting that the definitions of            omission and commission.
      valid research reside in the academic commu-                 A clear example of consequential validity is
      nities of the powerful. Lather (1986) calls for           formative assessment. This is concerned with the
      research to be emancipatory and to empower                extent to which students improve as a result
      those who are being researched, suggesting that           of feedback given, hence if there is insufficient
      catalytic validity, akin to Freire’s (1970) notion        feedback for students to improve, or if students are
      of ‘conscientization’, should empower partici-            unable to improve as a result of – a consequence
      pants to understand and transform their oppressed         of – the feedback, then the formative assessment
      situation.                                                has little consequential validity.
         Validity, it is proposed (Scheurich 1996), is but
      a mask that in fact polices and sets boundaries
                                                                Criterion-related validity
      to what is considered to be acceptable research
      by powerful research communities; discourses of           This form of validity endeavours to relate the
      validity in reality are discourses of power to define      results of one particular instrument to another
      worthwhile knowledge.                                     external criterion. Within this type of validity
         How defensible it is to suggest that researchers       there are two principal forms: predictive validity
      should have such ideological intents is, perhaps,         and concurrent validity.
      a moot point, though not to address this area is             Predictive validity is achieved if the data acquired
      to perpetuate inequality by omission and neglect.         at the first round of research correlate highly
      Catalytic validity reasserts the centrality of ethics     with data acquired at a future date. For example,
      in the research process, for it requires researchers to   if the results of examinations taken by 16
      interrogate their allegiances, responsibilities and       year olds correlate highly with the examination
      self-interestedness (Burgess 1989).                       results gained by the same students when aged
                                                                18, then we might wish to say that the
                                                                first examination demonstrated strong predictive
      Consequential validity
      Partially related to catalytic validity is consequen-        A variation on this theme is encountered in
      tial validity, which argues that the ways in which        the notion of concurrent validity. To demonstrate
      research data are used (the consequences of the           this form of validity the data gathered from
      research) are in keeping with the capability or           using one instrument must correlate highly with
      intentions of the research, i.e. the consequences         data gathered from using another instrument. For
      of the research do not exceed the capability of           example, suppose it was decided to research a
                                                                                         TRIANGULATION        141

student’s problem-solving ability. The researcher     use of both quantitative and qualitative data.

                                                                                                                Chapter 6
might observe the student working on a problem,       Triangulation is a powerful way of demonstrating
or might talk to the student about how she is         concurrent validity, particularly in qualitative
tackling the problem, or might ask the student        research (Campbell and Fiske 1959).
to write down how she tackled the problem.               The advantages of the multi-method approach
Here the researcher has three different data-         in social research are manifold and we examine
collecting instruments – observation, interview       two of them. First, whereas the single observation
and documentation respectively. If the results        in fields such as medicine, chemistry and
all agreed – concurred – that, according to given     physics normally yields sufficient and unambiguous
criteria for problem-solving ability, the student     information on selected phenomena, it provides
demonstrated a good ability to solve a problem,       only a limited view of the complexity of human
then the researcher would be able to say with         behaviour and of situations in which human
greater confidence (validity) that the student was     beings interact. It has been observed that as
good at problem-solving than if the researcher had    research methods act as filters through which
arrived at that judgement simply from using one       the environment is selectively experienced, they
instrument.                                           are never atheoretical or neutral in representing
   Concurrent validity is very similar to its         the world of experience (Smith 1975). Exclusive
partner – predictive validity – in its core concept   reliance on one method, therefore, may bias or
(i.e. agreement with a second measure); what          distort the researcher’s picture of the particular
differentiates concurrent and predictive validity     slice of reality being investigated. The researcher
is the absence of a time element in the former;       needs to be confident that the data generated
concurrence can be demonstrated simultaneously        are not simply artefacts of one specific method
with another instrument.                              of collection (Lin 1976). Such confidence can
   An important partner to concurrent validity,       be achieved, as far as nomothetic research
which is also a bridge into later discussions of      is concerned, when different methods of data
reliability, is triangulation.                        collection yield substantially the same results.
                                                      (Where triangulation is used in interpretive
                                                      research to investigate different actors’ viewpoints,
Triangulation                                         the same method, e.g. accounts, will naturally
Triangulation may be defined as the use of two         produce different sets of data.)
or more methods of data collection in the study          Further, the more the methods contrast with
of some aspect of human behaviour. The use of         each other, the greater the researcher’s confidence.
multiple methods, or the multi-method approach        If, for example, the outcomes of a questionnaire
as it is sometimes called, contrasts with the         survey correspond to those of an observational
ubiquitous but generally more vulnerable single-      study of the same phenomena, the more the
method approach that characterizes so much of         researcher will be confident about the findings.
research in the social sciences. In its original      Or, more extreme, where the results of a rigorous
and literal sense, triangulation is a technique       experimental investigation are replicated in,
of physical measurement: maritime navigators,         say, a role-playing exercise, the researcher will
military strategists and surveyors, for example,      experience even greater assurance. If findings are
use (or used to use) several locational markers       artefacts of method, then the use of contrasting
in their endeavours to pinpoint a single spot         methods considerably reduces the chances of
or objective. By analogy, triangular techniques       any consistent findings being attributable to
in the social sciences attempt to map out, or         similarities of method (Lin 1976).
explain more fully, the richness and complexity          We come now to a second advantage: some
of human behaviour by studying it from more           theorists have been sharply critical of the limited
than one standpoint and, in so doing, by making       use to which existing methods of inquiry in the

      social sciences have been put. One writer, for              the same country or within the same subculture
      example, comments:                                          by making use of cross-cultural techniques.
                                                                  Combined levels of triangulation: this type uses
        Much research has employed particular methods or
                                                                  more than one level of analysis from the three
        techniques out of methodological parochialism or
                                                                  principal levels used in the social sciences,
        ethnocentrism. Methodologists often push particular
                                                                  namely, the individual level, the interactive
        pet methods either because those are the only ones
                                                                  level (groups), and the level of collectivities
        they have familiarity with, or because they believe
                                                                  (organizational, cultural or societal).
        their method is superior to all others.
                                                                  Theoretical triangulation: this type draws upon
                                                (Smith 1975)
                                                                  alternative or competing theories in preference
      The use of triangular techniques, it is argued,             to utilizing one viewpoint only.
      will help to overcome the problem of ‘method-               Investigator triangulation: this type engages
      boundedness’, as it has been termed; in-                    more than one observer, data are discovered
      deed Gorard and Taylor (2004) demonstrate the               independently by more than one observer
      value of combining qualitative and quantitative             (Silverman 1993: 99).
      methods.                                                    Methodological triangulation: this type uses
         In its use of multiple methods, triangulation may        either the same method on different occasions,
      utilize either normative or interpretive techniques;        or different methods on the same object of
      or it may draw on methods from both these                   study.
      approaches and use them in combination.
         Referring us back to naturalistic inquiry, Lincoln       Many studies in the social sciences are
      and Guba (1985: 315) suggest that triangulation is       conducted at one point only in time, thereby
      intended as a check on data, while member check-         ignoring the effects of social change and
      ing, and elements of credibility, are to be used as a    process. Time triangulation goes some way to
      check on members’ constructions of data.                 rectifying these omissions by making use of cross-
                                                               sectional and longitudinal approaches. Cross-
                                                               sectional studies collect data at one point in
      Types of triangulation and their                         time; longitudinal studies collect data from the
      characteristics                                          same group at different points in the time
      We have just seen how triangulation is                   sequence. The use of panel studies and trend
      characterized by a multi-method approach to              studies may also be mentioned in this connection.
      a problem in contrast to a single-method                 The former compare the same measurements
      approach. Denzin (1970b) has, however, extended          for the same individuals in a sample at several
      this view of triangulation to take in several other      different points in time; and the latter examine
      types as well as the multi-method kind which he          selected processes continually over time. The
      terms ‘methodological triangulation’:                    weaknesses of each of these methods can be
                                                               strengthened by using a combined approach to
         Time triangulation: this type attempts to take        a given problem.
         into consideration the factors of change                 Space triangulation attempts to overcome the
         and process by utilizing cross-sectional and          limitations of studies conducted within one
         longitudinal designs. Kirk and Miller (1986)          culture or subculture. As Smith (1975) says,
         suggest that diachronic reliability seeks stability   ‘Not only are the behavioural sciences culture-
         of observations over time, while synchronic           bound, they are sub-culture-bound. Yet many
         reliability seeks similarity of data gathered in      such scholarly works are written as if basic
         the same time.                                        principles have been discovered which would
         Space triangulation: this type attempts to over-      hold true as tendencies in any society, anywhere,
         come the parochialism of studies conducted in         anytime’. Cross-cultural studies may involve the
                                                                                           TRIANGULATION        143

testing of theories among different people, as in       a research setting. Observers and participants

                                                                                                                  Chapter 6
Piagetian and Freudian psychology; or they may          working on their own each have their own
measure differences between populations by using        observational styles and this is reflected in the
several different measuring instruments. We have        resulting data. The careful use of two or more
addressed cultural validity earlier.                    observers or participants independently, therefore,
   Social scientists are concerned in their research    can lead to more valid and reliable data (Smith
with the individual, the group and society. These       1975), checking divergences between researchers
reflect the three levels of analysis adopted by          leading to minimal divergence, i.e. reliability.
researchers in their work. Those who are critical          In this respect the notion of triangulation
of much present-day research argue that some            bridges issues of reliability and validity. We have
of it uses the wrong level of analysis, individual      already considered methodological triangulation
when it should be societal, for instance, or limits     earlier. Denzin (1970b) identifies two categories
itself to one level only when a more meaningful         in his typology: ‘within methods’ triangulation and
picture would emerge by using more than one             ‘between methods’ triangulation. Triangulation
level. Smith (1975) extends this analysis and           within methods concerns the replication of a study
identifies seven possible levels: the aggregative        as a check on reliability and theory confirmation.
or individual level, and six levels that are more       Triangulation between methods involves the use
global in that ‘they characterize the collective as a   of more than one method in the pursuit of a
whole, and do not derive from an accumulation of        given objective. As a check on validity, the
individual characteristics’ (Smith 1975). The six       between methods approach embraces the notion
levels include:                                         of convergence between independent measures of
                                                        the same objective (Campbell and Fiske 1959).
   group analysis: the interaction patterns of
                                                           Of the six categories of triangulation in Denzin’s
   individuals and groups
                                                        typology, four are frequently used in education.
   organizational units of analysis: units which
                                                        These are: time triangulation with its longitudinal
   have qualities not possessed by the individuals
                                                        and cross-sectional studies; space triangulation as
   making them up
                                                        on the occasions when a number of schools in
   institutional analysis: relationships within and
                                                        an area or across the country are investigated in
   across the legal, political, economic and
                                                        some way; investigator triangulation as when two
   familial institutions of society
                                                        observers independently rate the same classroom
   ecological analysis: concerned with spatial
                                                        phenomena; and methodological triangulation. Of
                                                        these four, methodological triangulation is the
   cultural analysis: concerned with the norms,
                                                        one used most frequently and the one that possibly
   values, practices, traditions and ideologies of a
                                                        has the most to offer.
                                                           Triangular techniques are suitable when a more
   societal analysis: concerned with gross factors
                                                        holistic view of educational outcomes is sought
   such as urbanization, industrialization, educa-
                                                        (e.g. Mortimore et al.’s (1988) search for school
   tion, wealth, etc.
                                                        effectiveness), or where a complex phenomenon
Where possible, studies combining several levels        requires elucidation. Triangulation is useful when
of analysis are to be preferred. Researchers are        an established approach yields a limited and
sometimes taken to task for their rigid adherence       frequently distorted picture. Finally, triangulation
to one particular theory or theoretical orientation     can be a useful technique where a researcher is
to the exclusion of competing theories. Indeed          engaged in a case study, a particular example of
Smith (1975) recommends the use of research to          complex phenomena (Adelman et al. 1980).
test competing theories.                                   Triangulation is not without its critics. For
   Investigator triangulation refers to the use         example, Silverman (1985) suggests that the very
of more than one observer (or participant) in           notion of triangulation is positivistic, and that

      this is exposed most clearly in data triangulation,        selecting appropriate instrumentation for
      as it is presumed that a multiple data source              gathering the type of data required
      (concurrent validity) is superior to a single data         using an appropriate sample (e.g. one which is
      source or instrument. The assumption that a single         representative, not too small or too large)
      unit can always be measured more than once                 demonstrating internal, external, content,
      violates the interactionist principles of emergence,       concurrent and construct validity and ‘oper-
      fluidity, uniqueness and specificity (Denzin 1997:           ationalizing’ the constructs fairly
      320). Further, Patton (1980) suggests that even            ensuring reliability in terms of stability
      having multiple data sources, particularly of              (consistency, equivalence, split-half analysis
      qualitative data, does not ensure consistency or           of test material)
      replication. Fielding and Fielding (1986) hold that        selecting appropriate foci to answer the
      methodological triangulation does not necessarily          research questions
      increase validity, reduce bias or bring objectivity        devising and using appropriate instruments:
      to research.                                               for example, to catch accurate, representative,
          With regard to investigator triangula-                 relevant and comprehensive data (King et al.
      tion, Lincoln and Guba (1985: 307) contend that            1987); ensuring that readability levels are
      it is erroneous to assume that one investigator will       appropriate; avoiding any ambiguity of
      corroborate another, nor is this defensible, particu-      instructions, terms and questions; using
      larly in qualitative, reflexive inquiry. They extend        instruments that will catch the complexity
      their concern to include theory and methodolog-            of issues; avoiding leading questions; ensuring
      ical triangulation, arguing that the search for            that the level of test is appropriate – e.g.
      theory and methodological triangulation is episte-         neither too easy nor too difficult; avoiding
      mologically incoherent and empirically empty (see          test items with little discriminability; avoiding
      also Patton 1980). No two theories, it is argued,          making the instruments too short or too long;
      will ever yield a sufficiently complete explanation         avoiding too many or too few items for each
      of the phenomenon being researched. These criti-           issue
      cisms are trenchant, but they have been answered           avoiding a biased choice of researcher or
      equally trenchantly by Denzin (1997).                      research team (e.g. insiders or outsiders as
      Ensuring validity                                         There are several areas where invalidity or bias
      It is very easy to slip into invalidity; it is both     might creep into the research at the stage of data
      insidious and pernicious as it can enter at every       gathering; these can be minimized by:
      stage of a piece of research. The attempt to build         reducing the Hawthorne effect (see the
      out invalidity is essential if the researcher is to        accompanying web site: http://www.routledge.
      be able to have confidence in the elements of the           com/textbooks/9780415368780 – Chapter 6,
      research plan, data acquisition, data processing           file 6.1.doc)
      analysis, interpretation and its ensuing judge-            minimizing reactivity effects: respondents
      ment (see              behaving differently when subjected to scrutiny
      9780415368780 – Chapter 6, file 6.3. ppt).                  or being placed in new situations, for example
         At the design stage, threats to validity can be         the interview situation – we distort people’s
      minimized by:                                              lives in the way we go about studying them
         choosing an appropriate time scale                      (Lave and Kvale 1995: 226)
         ensuring that there are adequate resources for          trying to avoid dropout rates among respon-
         the required research to be undertaken                  dents
         selecting an appropriate methodology for                taking steps to avoid non-return of question-
         answering the research questions                        naires
                                                                                      ENSURING VALIDITY        145

   avoiding having too long or too short an               avoiding unfair aggregation of data (particu-

                                                                                                                 Chapter 6
   interval between pretests and post-tests               larly of frequency tables)
   ensuring inter-rater reliability                       avoiding unfair telescoping of data (degrading
   matching control and experimental groups               the data)
   fairly                                                 avoiding Type I and/or Type II errors (see
   ensuring standardized procedures for gathering
   data or for administering tests                        9780415368780 – Chapter 6, file 6.2.doc).
   building on the motivations of the respondents
   tailoring the instruments to the concentration         A Type I error is committed where the
   span of the respondents and addressing other        researcher rejects the null hypothesis when it
   situational factors (e.g. health, environment,      is in fact true (akin to convicting an innocent
   noise, distraction, threat)                         person: Mitchell and Jolley 1988: 121); this
   addressing factors concerning the researcher        can be addressed by setting a more rigorous
   (particularly in an interview situation);           level of significance (e.g. ρ < 0.01 rather than
   for example, the attitude, gender, race,            ρ < 0.05). A Type II error is committed where
   age, personality, dress, comments, replies,         the null hypothesis is accepted when it is in
   questioning technique, behaviour, style and         fact not true (akin to finding a guilty person
   non-verbal communication of the researcher.         innocent: Mitchell and Jolley: 1988: 121). Boruch
                                                       (1997: 211) suggests that a Type II error
   At the stage of data analysis there are several     may occur if the measurement of a response
areas where invalidity lurks; these might be           to the intervention is insufficiently valid; the
minimized by:                                          measurement of the intervention is insufficiently
                                                       relevant; the statistical power of the experiment is
   using respondent validation                         too low; the wrong population was selected for the
   avoiding subjective interpretation of data (e.g.    intervention.
   being too generous or too ungenerous in the            A Type II error can be addressed by reducing
   award of marks), i.e. lack of standardization       the level of significance (e.g. ρ < 0.20 or ρ < 0.30
   and moderation of results                           rather than ρ < 0.05). Of course, the more one
   reducing the halo effect, where the researcher’s    reduces the chance of a Type I error, the more
   knowledge of the person or knowledge of other       chance there is of committing a Type II error,
   data about the person or situation exerts an        and vice versa. In qualitative data a Type I error is
   influence on subsequent judgements                   committed when a statement is believed when it is,
   using appropriate statistical treatments for        in fact, not true, and a Type II error is committed
   the level of data (e.g. avoiding applying           when a statement is rejected when it is in fact true.
   techniques from interval scaling to ordinal data       At the stage of data reporting invalidity can
   or using incorrect statistics for the type, size,   show itself in several ways; the researcher must
   complexity, sensitivity of data)                    take steps to minimize this by, for example:
   recognizing spurious correlations and extrane-
   ous factors which may be affecting the data            avoiding using data selectively and unrepresen-
   (i.e. tunnel vision)                                   tatively, for example, accentuating the positive
   avoiding poor coding of qualitative data               and neglecting or ignoring the negative
   avoiding making inferences and generaliza-             indicating the context and parameters of the
   tions beyond the capability of the data to             research in the data collection and treatment,
   support such statements                                the degree of confidence which can be placed
   avoiding the equating of correlations and              in the results, the degree of context-freedom or
   causes                                                 context-boundedness of the data (i.e. the level
   avoiding selective use of data                         to which the results can be generalized)

         presenting the data without misrepresenting its     would mean that if a test and then a retest were
         message                                             undertaken within an appropriate time span, then
         making claims which are sustainable by the          similar results would be obtained. The researcher
         data                                                has to decide what an appropriate length of time is;
         avoiding inaccurate or wrong reporting of data      too short a time and respondents may remember
         (i.e. technical errors or orthographic errors)      what they said or did in the first test situation,
         ensuring that the research questions are            too long a time and there may be extraneous
         answered; releasing research results neither too    effects operating to distort the data (for example,
         soon nor too late.                                  maturation in students, outside influences on the
                                                             students). A researcher seeking to demonstrate
      Having identified where invalidity lurks, the
                                                             this type of reliability will have to choose an
      researcher can take steps to ensure that, as far
                                                             appropriate time scale between the test and retest.
      as possible, invalidity has been minimized in all
                                                             Correlation coefficients can be calculated for the
      areas of the research.
                                                             reliability of pretests and post-tests, using formulae
                                                             which are readily available in books on statistics
      Reliability in quantitative research                   and test construction.
                                                                In addition to stability over time, reliability as
      The meaning of reliability differs in quan-            stability can also be stability over a similar sample.
      titative and qualitative research (see http://         For example, we would assume that if we were –            to administer a test or a questionnaire simulta-
      Chapter 6, file 6.4 ppt). We explore these concepts     neously to two groups of students who were very
      separately in the next two sections. Reliability in    closely matched on significant characteristics (e.g.
      quantitative research is essentially a synonym for     age, gender, ability etc. – whatever characteristics
      dependability, consistency and replicability over      are deemed to have a significant bearing, on the
      time, over instruments and over groups of respon-      responses), then similar results (on a test) or re-
      dents. It is concerned with precision and accuracy;    sponses (to a questionnaire) would be obtained.
      some features, e.g. height, can be measured pre-       The correlation coefficient on this form of the
      cisely, while others, e.g. musical ability, cannot.    test/retest method can be calculated either for the
      For research to be reliable it must demonstrate        whole test (e.g. by using the Pearson statistic or a
      that if it were to be carried out on a sim-            t-test) or for sections of the questionnaire (e.g. by
      ilar group of respondents in a similar context         using the Spearman or Pearson statistic as appro-
      (however defined), then similar results would be        priate or a t-test). The statistical significance of
      found. There are three principal types of relia-       the correlation coefficient can be found and should
      bility: stability, equivalence and internal consis-    be 0.05 or higher if reliability is to be guaranteed.
      tency (see         This form of reliability over a sample is particularly
      9780415368780 – Chapter 6, file 6.5. ppt).              useful in piloting tests and questionnaires.
                                                                In using the test-retest method, care has to be
                                                             taken to ensure (Cooper and Schindler 2001: 216)
      Reliability as stability
                                                             the following:
      In this form reliability is a measure of consistency
      over time and over similar samples. A reliable            The time period between the test and retest is
      instrument for a piece of research will yield             not so long that situational factors may change.
      similar data from similar respondents over time.          The time period between the test and retest is
      A leaking tap which each day leaks one litre is           not so short that the participants will remember
      leaking reliably whereas a tap which leaks one litre      the first test.
      some days and two litres on others is not. In the         The participants may have become interested
      experimental and survey models of research this           in the field and may have followed it up
                                                                   RELIABILITY IN QUANTITATIVE RESEARCH             147

   themselves between the test and the retest             observational data, and his method can be used

                                                                                                                      Chapter 6
   times.                                                 with other types of data.

Reliability as equivalence                                Reliability as internal consistency

Within this type of reliability there are two main        Whereas the test/retest method and the equivalent
sorts. Reliability may be achieved first through           forms method of demonstrating reliability require
using equivalent forms (also known as alternative         the tests or instruments to be done twice,
forms) of a test or data-gathering instrument.            demonstrating internal consistency demands that
If an equivalent form of the test or instrument           the instrument or tests be run once only through
is devised and yields similar results, then the           the split-half method.
instrument can be said to demonstrate this form of           Let us imagine that a test is to be administered to
reliability. For example, the pretest and post-test       a group of students. Here the test items are divided
in an experiment are predicated on this type of           into two halves, ensuring that each half is matched
reliability, being alternate forms of instrument to       in terms of item difficulty and content. Each half
measure the same issues. This type of reliability         is marked separately. If the test is to demonstrate
might also be demonstrated if the equivalent forms        split-half reliability, then the marks obtained on
of a test or other instrument yield consistent results    each half should be correlated highly with the
if applied simultaneously to matched samples (e.g.        other. Any student’s marks on the one half should
a control and experimental group or two random            match his or her marks on the other half. This can
stratified samples in a survey). Here reliability          be calculated using the Spearman-Brown formula:
can be measured through a t-test, through the                                2r
                                                            Reliability =
demonstration of a high correlation coefficient                              1+r
and through the demonstration of similar means            where r = the actual correlation between the
and standard deviations between two groups.               halves of the instrument (see http://www.
   Second, reliability as equivalence may be     –
achieved through inter-rater reliability. If more         Chapter 6, file 6.6. ppt).
than one researcher is taking part in a piece of             This calculation requires a correlation coeffi-
research then, human judgement being fallible,            cient to be calculated, e.g. a Spearman rank order
agreement between all researchers must be                 correlation or a Pearson product moment correla-
achieved, through ensuring that each researcher           tion.
enters data in the same way. This would be                   Let us say that using the Spearman-Brown
particularly pertinent to a team of researchers           formula, the correlation coefficient is 0.85; in this
gathering structured observational or semi-               case the formula for reliability is set out thus:
structured interview data where each member of
                                                                            2 × 0.85   1.70
the team would have to agree on which data would            Reliability =            =      = 0.919
                                                                            1 + 0.85   1.85
be entered in which categories. For observational
data, reliability is addressed in the training sessions   Given that the maximum value of the coefficient
for researchers where they work on video material         is 1.00 we can see that the reliability of this
to ensure parity in how they enter the data.              instrument, calculated for the split-half form of
   At a simple level one can calculate the inter-         reliability, is very high indeed.
rater agreement as a percentage:                             This type of reliability assumes that the test
                                                          administered can be split into two matched halves;
   Number of actual agreements
                                 × 100                    many tests have a gradient of difficulty or different
   Number of possible agreements                          items of content in each half. If this is the case and,
Robson (2002: 341) sets out a more sophisticated          for example, the test contains twenty items, then
way of measuring inter-rater reliability in coded         the researcher, instead of splitting the test into two

      by assigning items one to ten to one half and items       may be simply unworkable for qualitative research.
      eleven to twenty to the second half, may assign all       Quantitative research assumes the possibility of
      the even numbered items to one group and all the          replication; if the same methods are used with the
      odd numbered items to another. This would move            same sample then the results should be the same.
      towards the two halves being matched in terms of          Typically quantitative methods require a degree
      content and cumulative degrees of difficulty.              of control and manipulation of phenomena. This
         An alternative measure of reliability as internal      distorts the natural occurrence of phenomena (see
      consistency is the Cronbach alpha, frequently             earlier: ecological validity). Indeed the premises of
      referred to as the alpha coefficient of reliability, or    naturalistic studies include the uniqueness and
      simply the alpha. The Cronbach alpha provides a           idiosyncrasy of situations, such that the study
      coefficient of inter-item correlations, that is, the       cannot be replicated – that is their strength rather
      correlation of each item with the sum of all the          than their weakness.
      other relevant items, and is useful for multi-item           On the other hand, this is not to say that
      scales. This is a measure of the internal consistency     qualitative research need not strive for replica-
      among the items (not, for example, the people). We        tion in generating, refining, comparing and vali-
      address the alpha coefficient and its calculation in       dating constructs (see
      Part Five.                                                textbooks/9780415368780 – Chapter 6, file 6.7.
         Reliability, thus construed, makes several             ppt). Indeed LeCompte and Preissle (1993: 334)
      assumptions, for example that instrumentation,            argue that such replication might include repeat-
      data and findings should be controllable,                  ing
      predictable, consistent and replicable. This
                                                                   the status position of the researcher
      presupposes a particular style of research,
                                                                   the choice of informant/respondents
      typically within the positivist paradigm. Cooper
                                                                   the social situations and conditions
      and Schindler (2001: 218) suggest that, in
                                                                   the analytic constructs and premises that are
      this paradigm, reliability can be improved by
      minimizing any external sources of variation:
                                                                   the methods of data collection and analysis.
      standardizing and controlling the conditions under
      which the data collection and measurement take               Further, Denzin and Lincoln (1994) suggest that
      place; training the researchers in order to ensure        reliability as replicability in qualitative research
      consistency (inter-rater reliability); widening the       can be addressed in several ways:
      number of items on a particular topic; excluding
                                                                   stability of observations: whether the researcher
      extreme responses from the data analysis (e.g.
                                                                   would have made the same observations and
      outliers, which can be done with SPSS).
                                                                   interpretation of these if they had been
                                                                   observed at a different time or in a different
      Reliability in qualitative research                          place
                                                                   parallel forms: whether the researcher would
      While we discuss reliability in qualitative research
                                                                   have made the same observations and
      here, the suitability of the term for qualitative
                                                                   interpretations of what had been seen if he
      research is contested (e.g. Winter 2000; Stenbacka
                                                                   or she had paid attention to other phenomena
      2001; Golafshani 2003). Lincoln and Guba (1985)
                                                                   during the observation
      prefer to replace ‘reliability’ with terms such as
                                                                   inter-rater reliability: whether another observer
      ‘credibility’, ‘neutrality’, ‘confirmability’, ‘depend-
                                                                   with the same theoretical framework and
      ability’, ‘consistency’, ‘applicability’, ‘trustworthi-
                                                                   observing the same phenomena would have
      ness’ and ‘transferability’, in particular the notion
                                                                   interpreted them in the same way.
      of ‘dependability’.
         LeCompte and Preissle (1993: 332) suggest that         Clearly this is a contentious issue, for it is seeking
      the canons of reliability for quantitative research       to apply to qualitative research the canons of
                                                                   RELIABILITY IN QUALITATIVE RESEARCH           149

reliability of quantitative research. Purists might     results, in terms of process and product (Golafshani

                                                                                                                   Chapter 6
argue against the legitimacy, relevance or need for     2003: 601). These are a safeguard against the
this in qualitative studies.                            charge levelled against qualitative researchers,
   In qualitative research reliability can be           namely that they respond only to the ‘loudest
regarded as a fit between what researchers record        bangs or the brightest lights’.
as data and what actually occurs in the natural            Dependability raises the important issue of
setting that is being researched, i.e. a degree         respondent validation (see also McCormick and
of accuracy and comprehensiveness of coverage           James 1988). While dependability might suggest
(Bogdan and Biklen 1992: 48). This is not to            that researchers need to go back to respondents
strive for uniformity; two researchers who are          to check that their findings are dependable,
studying a single setting may come up with very         researchers also need to be cautious in placing
different findings but both sets of findings might        exclusive store on respondents, for, as Hammersley
be reliable. Indeed Kvale (1996: 181) suggests          and Atkinson (1983) suggest, they are not in a
that, in interviewing, there might be as many           privileged position to be sole commentators on
different interpretations of the qualitative data as    their actions.
there are researchers. A clear example of this             Bloor (1978) suggests three means by which
is the study of the Nissan automobile factory           respondent validation can be addressed:
in the United Kingdom, where Wickens (1987)
found a ‘virtuous circle’ of work organization             researchers attempt to predict what the
practices that demonstrated flexibility, teamwork           participants’ classifications of situations will
and quality consciousness, whereas the same                be
practices were investigated by Garrahan and                researchers prepare hypothetical cases and then
Stewart (1992), who found a ‘vicious circle’ of            predict respondents’ likely responses to them
exploitation, surveillance and control respectively.       researchers take back their research report to
Both versions of the same reality coexist because          the respondents and record their reactions to
reality is multilayered. What is being argued for          that report.
here is the notion of reliability through an eclectic
use of instruments, researchers, perspectives and       The argument rehearses the paradigm wars dis-
interpretations (echoing the comments earlier           cussed in the opening chapter: quantitative mea-
about triangulation) (see also Eisenhart and Howe       sures are criticized for combining sophistication
1992).                                                  and refinement of process with crudity of con-
   Brock-Utne (1996) argues that qualitative            cept (Ruddock 1981) and for failing to distin-
research, being holistic, strives to record the         guish between educational and statistical signif-
multiple interpretations of, intention in and           icance (Eisner 1985); qualitative methodologies,
meanings given to situations and events. Here the       while possessing immediacy, flexibility, authenti-
notion of reliability is construed as dependability     city, richness and candour, are criticized for being
(Lincoln and Guba 1985: 108–9; Anfara et al.            impressionistic, biased, commonplace, insignif-
2002), recalling the earlier discussion on internal     icant, ungeneralizable, idiosyncratic, subjective
validity. For them, dependability involves member       and short-sighted (Ruddock 1981). This is an arid
checks (respondent validation), debriefing by            debate; rather the issue is one of fitness for purpose.
peers, triangulation, prolonged engagement in the       For our purposes here we need to note that criteria
field, persistent observations in the field, reflexive     of reliability in quantitative methodologies differ
journals, negative case analysis, and independent       from those in qualitative methodologies. In quali-
audits (identifying acceptable processes of             tative methodologies reliability includes fidelity to
conducting the inquiry so that the results are          real life, context- and situation-specificity, authen-
consistent with the data). Audit trails enable the      ticity, comprehensiveness, detail, honesty, depth
research to address the issue of confirmability of       of response and meaningfulness to the respondents.

      Validity and reliability in interviews                 the interviewee and, thereby, on the data. Fielding
                                                             and Fielding (1986: 12) make the telling comment
      In interviews, inferences about validity are made
                                                             that even the most sophisticated surveys only
      too often on the basis of face validity (Cannell
                                                             manipulate data that at some time had to be
      and Kahn 1968), that is, whether the questions
                                                             gained by asking people! Interviewer neutrality is
      asked look as if they are measuring what they
                                                             a chimera (Denscombe 1995).
      claim to measure. One cause of invalidity is bias,
                                                                Lee (1993) indicates the problems of conducting
      defined as ‘a systematic or persistent tendency
                                                             interviews perhaps at their sharpest, where the
      to make errors in the same direction, that is,
                                                             researcher is researching sensitive subjects, i.e.
      to overstate or understate the ‘‘true value’’ of
                                                             research that might pose a significant threat
      an attribute’ (Lansing et al. 1961). One way of
                                                             to those involved (be they interviewers or
      validating interview measures is to compare the
                                                             interviewees). Here the interview might be seen as
      interview measure with another measure that has
                                                             an intrusion into private worlds, or the interviewer
      already been shown to be valid. This kind of
                                                             might be regarded as someone who can impose
      comparison is known as ‘convergent validity’. If
                                                             sanctions on the interviewee, or as someone who
      the two measures agree, it can be assumed that the
                                                             can exploit the powerless; the interviewee is in the
      validity of the interview is comparable with the
                                                             searchlight that is being held by the interviewer
      proven validity of the other measure.
                                                             (see also Scheurich 1995). Indeed Gadd (2004)
         Perhaps the most practical way of achieving
                                                             reports that an interviewee may reduce his or
      greater validity is to minimize the amount
                                                             her willingness to ‘open up’ to an interviewer
      of bias as much as possible. The sources of
                                                             if the dynamics of the interview situation are
      bias are the characteristics of the interviewer,
                                                             too threatening, taking the role of the ‘defended
      the characteristics of the respondent, and the
                                                             subject’. The issues also embrace transference and
      substantive content of the questions. More
                                                             counter-transference, which have their basis in
      particularly, these will include:
                                                             psychoanalysis. In transference the interviewees
         the attitudes, opinions and expectations of the     project onto the interviewer their feelings, fears,
         interviewer                                         desires, needs and attitudes that derive from their
         a tendency for the interviewer to see the           own experiences (Scheurich 1995). In counter-
         respondent in his or her own image                  transference the process is reversed.
         a tendency for the interviewer to seek answers         One way of controlling for reliability is
         that support preconceived notions                   to have a highly structured interview, with
         misperceptions on the part of the interviewer       the same format and sequence of words and
         of what the respondent is saying                    questions for each respondent (Silverman 1993),
         misunderstandings on the part of the                though Scheurich (1995: 241–9) suggests that
         respondent of what is being asked.                  this is to misread the infinite complexity and
                                                             open-endedness of social interaction. Controlling
         Studies have also shown that race, religion,        the wording is no guarantee of controlling the
      gender, sexual orientation, status, social class and   interview. Oppenheim (1992: 147) argues that
      age in certain contexts can be potent sources of       wording is a particularly important factor in
      bias, i.e. interviewer effects (Lee 1993; Scheurich    attitudinal questions rather than factual questions.
      1995). Interviewers and interviewees alike bring       He suggests that changes in wording, context
      their own, often unconscious, experiential and         and emphasis undermine reliability, because
      biographical baggage with them into the interview      it ceases to be the same question for each
      situation. Indeed Hitchcock and Hughes (1989)          respondent. Indeed he argues that error and
      argue that because interviews are interpersonal,       bias can stem from alterations to wording,
      humans interacting with humans, it is inevitable       procedure, sequence, recording and rapport, and
      that the researcher will have some influence on         that training for interviewers is essential to
                                                              VALIDITY AND RELIABILITY IN INTERVIEWS          151

minimize this. Silverman (1993) suggests that it       the interviewee had been a frequent complainer,

                                                                                                                Chapter 6
is important for each interviewee to understand        and the question ‘How satisfied are you with
the question in the same way. He suggests that         the new Mathematics scheme?’ assumes a degree
the reliability of interviews can be enhanced by:      of satisfaction with the scheme. The leading
careful piloting of interview schedules; training of   questions here might be rendered less leading by
interviewers; inter-rater reliability in the coding    rephrasing, for example: ‘How frequently do you
of responses; and the extended use of closed           have conversations with the headteacher?’ and
questions.                                             ‘What is your opinion of the new Mathematics
   On the other hand, Silverman (1993) argues for      scheme?’ respectively.
the importance of open-ended interviews, as this          In discussing the issue of leading questions, we
enables respondents to demonstrate their unique        are not necessarily suggesting that there is not a
way of looking at the world – their definition of       place for them. Indeed Kvale (1996: 158) makes
the situation. It recognizes that what is a suitable   a powerful case for leading questions, arguing
sequence of questions for one respondent might be      that they may be necessary in order to obtain
less suitable for another, and open-ended questions    information that the interviewer suspects the
enable important but unanticipated issues to be        interviewee might be withholding. Here it might
raised.                                                be important to put the ‘burden of denial’ onto
   Oppenheim (1992: 96–7) suggests several             the interviewee (e.g. ‘When did you stop beating
causes of bias in interviewing:                        your wife?’). Leading questions, frequently used
                                                       in police interviews, may be used for reliability
   biased sampling (sometimes created by
                                                       checks with what the interviewee has already said,
   the researcher not adhering to sampling
                                                       or may be deliberately used to elicit particular
                                                       non-verbal behaviours that give an indication of
   poor rapport between interviewer and inter-
                                                       the sensitivity of the interviewee’s remarks.
                                                          Hence reducing bias becomes more than
   changes to question wording (e.g. in attitudinal
                                                       simply: careful formulation of questions so that
   and factual questions)
                                                       the meaning is crystal clear; thorough training
   poor prompting and biased probing
                                                       procedures so that an interviewer is more aware
   poor use and management of support materials
                                                       of the possible problems; probability sampling of
   (e.g. show cards)
                                                       respondents; and sometimes matching interviewer
   alterations to the sequence of questions
                                                       characteristics with those of the sample being
   inconsistent coding of responses
                                                       interviewed. Oppenheim (1992: 148) argues, for
   selective or interpreted recording of data/
                                                       example, that interviewers seeking attitudinal
                                                       responses have to ensure that people with known
   poor handling of difficult interviews.
                                                       characteristics are included in the sample – the
   One can add to this the issue of ‘acquiescence’     criterion group. We need to recognize that the
(Breakwell 2000: 254), the tendency that               interview is a shared, negotiated and dynamic
respondents may have to say ‘yes’, regardless of       social moment.
the question or, indeed, regardless of what they          The notion of power is significant in the
really feel or think.                                  interview situation, for the interview is not simply
   There is also the issue of leading questions. A     a data collection situation but a social and
leading question is one which makes assumptions        frequently a political situation. Literally the word
about interviewees or ‘puts words into their           ‘inter-view’ is a view between people, mutually, not
mouths’, where the question influences the answer,      the interviewer extracting data, one-way, from the
perhaps illegitimately. For example (Morrison          interviewee. Power can reside with interviewer
1993: 66–7) the question ‘When did you stop                                               o
                                                       and interviewee alike (Thapar-Bj¨ rkert and Henry
complaining to the headteacher?’ assumes that          2004), though Scheurich (1995: 246) argues that,

      typically, more power resides with the interviewer:      where she instances being kept waiting, and
      the interviewer generates the questions and the          subsequently being interrupted, being patronized,
      interviewee answers them; the interviewee is             and being interviewed by the interviewee (see
      under scrutiny while the interviewer is not. Kvale       also Walford 1994d). Indeed Scheurich (1995)
      (1996: 126), too, suggests that there are definite        suggests that many powerful interviewees will
      asymmetries of power as the interviewer tends to         rephrase or not answer the question. Connell
      define the situation, the topics, and the course of       et al. (1996) argue that a working-class female
      the interview.                                           talking with a multinational director will be
         J. Cassell (cited in Lee 1993) suggests that elites   very different from a middle-class professor
      and powerful people might feel demeaned or               talking to the same person. Limerick et al. (1996)
      insulted when being interviewed by those with            comment on occasions where interviewers have
      a lower status or less power. Further, those with        felt themselves to be passive, vulnerable, helpless
      power, resources and expertise might be anxious          and indeed manipulated. One way of overcoming
      to maintain their reputation, and so will be more        this is to have two interviewers conducting each
      guarded in what they say, wrapping this up in well-      interview (Walford 1994c: 227). On the other
      chosen, articulate phrases. Lee (1993) comments          hand, Hitchcock and Hughes (1989) observe that
      on the asymmetries of power in several interview         if the researchers are known to the interviewees
      situations, with one party having more power             and they are peers, however powerful, then a
      and control over the interview than the other.           degree of reciprocity might be taking place, with
      Interviewers need to be aware of the potentially         interviewees giving answers that they think the
      distorting effects of power, a significant feature of     researchers might want to hear.
      critical theory, as discussed in Chapter 1.                 The issue of power has not been lost on fem-
         Neal (1995) draws attention to the feelings                                            o
                                                               inist research (e.g. Thapar-Bj¨ rkert and Henry
      of powerlessness and anxieties about physical            2004), that is, research that emphasizes subjec-
      presentation and status on the part of interviewers      tivity, equality, reciprocity, collaboration, non-
      when interviewing powerful people. This is               hierarchical relations and emancipatory poten-
      particularly so for frequently lone, low-status          tial (catalytic and consequential validity) (Neal
      research students interviewing powerful people;          1995), echoing the comments about research that
      a low-status female research student might find           is influenced by the paradigm of critical theory.
      that an interview with a male in a position of           Here feminist research addresses a dilemma of
      power (e.g. a university Vice-chancellor, a senior       interviews that are constructed in the dominant,
      politician or a senior manager) might turn out to        male paradigm of pitching questions that demand
      be very different from an interview with the same        answers from a passive respondent.
      person if conducted by a male university professor          Limerick et al. (1996) suggest that, in fact,
      where it is perceived by the interviewee to be more      it is wiser to regard the interview as a gift,
      of a dialogue between equals (see also Gewirtz           as interviewees have the power to withhold
      and Ozga 1993, 1994). Ball (1994b) comments              information, to choose the location of the
      that, when powerful people are being interviewed,        interview, to choose how seriously to attend
      interviews must be seen as an extension of the ‘play     to the interview, how long it will last, when
      of power’ – with its game-like connotations. He          it will take place, what will be discussed – and
      suggests that powerful people control the agenda         in what and whose terms – what knowledge is
      and course of the interview, and are usually very        important, even how the data will be analysed
      adept at this because they have both a personal                                           o
                                                               and used (see also Thapar-Bj¨ rkert and Henry
      and professional investment in being interviewed         2004). Echoing Foucault, they argue that power is
      (see also Batteson and Ball 1995; Phillips 1998).        fluid and is discursively constructed through the
         The effect of power can be felt even before           interview rather than being the province of either
      the interview commences, notes Neal (1995),              party.
                                                                 VALIDITY AND RELIABILITY IN INTERVIEWS             153

   Miller and Cannell (1997) identify some                intimate situation. Hence, telephone interviews

                                                                                                                      Chapter 6
particular problems in conducting telephone               have their strengths and weaknesses, and their use
interviews, where the reduction of the interview          should be governed by the criterion of fitness for
situation to just auditory sensory cues can be            purpose. They tend to be shorter, more focused
particularly problematical. There are sampling            and useful for contacting busy people (Harvey
problems, as not everyone will have a telephone.          1988; Miller, 1995).
Further, there are practical issues, for example,            In his critique of the interview as a research
interviewees can retain only a certain amount             tool, Kitwood (1977) draws attention to the con-
of information in their short-term memory, so             flict it generates between the traditional concepts
bombarding the interviewee with too many                  of validity and reliability. Where increased relia-
choices (the non-written form of ‘show cards’ of          bility of the interview is brought about by greater
possible responses) becomes unworkable. Hence             control of its elements, this is achieved, he argues,
the reliability of responses is subject to the            at the cost of reduced validity. He explains:
memory capabilities of the interviewee – how
                                                            In proportion to the extent to which ‘reliability’
many scale points and descriptors, for example,
                                                            is enhanced by rationalization, ‘validity’ would
can an interviewee retain about a single item?
                                                            decrease. For the main purpose of using an interview
Further, the absence of non-verbal cues is
                                                            in research is that it is believed that in an
significant, e.g. facial expression, gestures, posture,
                                                            interpersonal encounter people are more likely to
the significance of silences and pauses (Robinson
                                                            disclose aspects of themselves, their thoughts, their
1982), as interviewees may be unclear about
                                                            feelings and values, than they would in a less
the meaning behind words and statements. This
                                                            human situation. At least for some purposes, it
problem is compounded if the interviewer is
                                                            is necessary to generate a kind of conversation in
unknown to the interviewee.
                                                            which the ‘respondent’ feels at ease. In other words,
   Miller and Cannell (1997) report important
                                                            the distinctively human element in the interview is
research evidence to support the significance of
                                                            necessary to its ‘validity’. The more the interviewer
the non-verbal mediation of verbal dialogue. As
                                                            becomes rational, calculating, and detached, the less
discussed earlier, the interview is a social situation;
                                                            likely the interview is to be perceived as a friendly
in telephone interviews the absence of essential
                                                            transaction, and the more calculated the response
social elements could undermine the salient
                                                            also is likely to be.
conduct of the interview, and hence its reliability
                                                                                                 (Kitwood 1977)
and validity. Non-verbal paralinguistic cues affect
the conduct, pacing and relationships in the                 Kitwood (1977) suggests that a solution to the
interview and the support, threat and confidence           problem of validity and reliability might lie in the
felt by the interviewees. Telephone interviews can        direction of a ‘judicious compromise’.
easily slide into becoming mechanical and cold.              A cluster of problems surround the person being
Further, the problem of loss of non-verbal cues           interviewed. Tuckman (1972), for example, has
is compounded by the asymmetries of power that            observed that, when formulating their questions,
often exist between interviewer and interviewee;          interviewers have to consider the extent to which
the interviewer will need to take immediate steps         a question might influence respondents to show
to address these issues (e.g. by putting interviewees     themselves in a good light; or the extent to which
at their ease).                                           a question might influence respondents to be
   On the other hand, Nias (1991) and Miller              unduly helpful by attempting to anticipate what
and Cannell (1997) suggest that the very                  the interviewer wants to hear; or the extent to
factor that interviews are not face-to-face may           which a question might be asking for information
strengthen their reliability, as the interviewee          about respondents that they are not certain or
might disclose information that may not be                likely to know themselves. Further, interviewing
so readily forthcoming in a face-to-face, more            procedures are based on the assumption that the

      people interviewed have insight into the cause          in unfamiliar and uncongenial surroundings to
      of their behaviour. Insight of this kind may be         extended responses in the more congenial and less
      rarely achieved and, when it is, it is after long and   threatening surroundings – more sympathetic to
      difficult effort, usually in the context of repeated     the children’s everyday world. The language, argot
      clinical interviews.                                    and jargon (Edwards 1976), social and cultural
         In educational circles interviewing might be         factors of the interviewer and interviewee all exert
      a particular problem in working with children.          a powerful influence on the interview situation.
      Simons (1982) and McCormick and James (1988)               The issue is also raised here (Lee 1993) of
      comment on particular problems involved in              whether there should be a single interview
      interviewing children, for example:                     that maintains the detachment of the researcher
                                                              (perhaps particularly useful in addressing sensitive
         establishing trust
                                                              topics), or whether there should be repeated
         overcoming reticence
                                                              interviews to gain depth and to show fidelity to
         maintaining informality
                                                              the collaborative nature of research (a feature, as
         avoiding assuming that children ‘know the
                                                              was noted above, which is significant for feminist
                                                              research: Oakley 1981).
         overcoming the problems of inarticulate
                                                                 Kvale (1996: 148–9) suggests that a skilled
                                                              interviewer should:
         pitching the question at the right level
         choosing the right vocabulary                           know the subject matter in order to conduct
         being aware of the giving and receiving of              an informed conversation
         non-verbal cues                                         structure the interview well, so that each stage
         moving beyond the institutional response or             of the interview is clear to the participant
         receiving what children think the interviewer           be clear in the terminology and coverage of the
         wants to hear                                           material
         avoiding the interviewer being seen as an               allow participants to take their time and answer
         authority, spy or plant                                 in their own way
         keeping to the point                                    be sensitive and empathic, using active
         breaking silences on taboo areas and those              listening and being sensitive to how something
         which are reinforced by peer-group pressure             is said and the non-verbal communication
         seeing children as being of lesser importance           involved
         than adults (maybe in the sequence in which             be alert to those aspects of the interview which
         interviews are conducted, e.g. the headteacher,         may hold significance for the participant
         then the teaching staff, then the children).            keep to the point and the matter in hand,
                                                                 steering the interview where necessary in order
      These are not new matters. The studies by
                                                                 to address this
      Labov in the 1960s showed how students
                                                                 check the reliability, validity and consistency
      reacted very strongly to contextual matters in an
                                                                 of responses by well-placed questioning
      interview situation (Labov 1969). The language
                                                                 be able to recall and refer to earlier statements
      of children varied according to the ethnicity
                                                                 made by the participant
      of the interviewee, the friendliness of the
                                                                 be able to clarify, confirm and modify the
      surroundings, the opportunity for the children
                                                                 participants’ comments with the participant.
      to be interviewed with friends, the ease with
      which the scene was set for the interview, the            Walford (1994c: 225) adds to this the need for
      demeanour of the adult (e.g. whether the adult          interviewers to have done their homework when
      was standing or sitting) and the nature of the          interviewing powerful people, as such people could
      topics covered. The differences were significant,        well interrogate the interviewer – they will assume
      varying from monosyllabic responses by children         up-to-dateness, competence and knowledge in the
                                                           VALIDITY AND RELIABILITY IN EXPERIMENTS          155

interviewer. Powerful interviewees are usually busy   that are of greater consequence to the validity

                                                                                                              Chapter 6
people and will expect the interviewer to have read   of quasi-experiments (more typical in educational
the material that is in the public domain.            research) than to true experiments in which ran-
   The issues of reliability do not reside solely     dom assignment to treatments occurs and where
in the preparations for and conduct of the            both treatment and measurement can be more
interview; they extend to the ways in which           adequately controlled by the researcher. The fol-
interviews are analysed. For example, Lee (1993)      lowing summaries adapted from Campbell and
and Kvale (1996: 163) comment on the issue            Stanley (1963), Bracht and Glass (1968) and
of ‘transcriber selectivity’. Here transcripts of     Lewis-Beck (1993) distinguish between ‘internal
interviews, however detailed and full they            validity’ and ‘external validity’. Internal valid-
might be, remain selective, since they are            ity is concerned with the question, ‘Do the
interpretations of social situations. They become     experimental treatments, in fact, make a differ-
decontextualized, abstracted, even if they record     ence in the specific experiments under scrutiny?’.
silences, intonation, non-verbal behaviour etc.       External validity, on the other hand, asks the
The issue, then, is how useful they are to            question, ‘Given these demonstrable effects, to
researchers overall rather than whether they are      what populations or settings can they be gener-
completely reliable.                                  alized?’ (see
   One of the problems that has to be considered      9780415368780 – Chapter 6, file 6.8. ppt).
when open-ended questions are used in the
interview is that of developing a satisfactory
                                                      Threats to internal validity
method of recording replies. One way is to
summarize responses in the course of the interview.      History: Frequently in educational research,
This has the disadvantage of breaking the                events other than the experimental treatments
continuity of the interview and may result in            occur during the time between pretest and
bias because the interviewer may unconsciously           post-test observations. Such events produce
emphasize responses that agree with his or her           effects that can mistakenly be attributed to
expectations and fail to note those that do not. It      differences in treatment.
is sometimes possible to summarize an individual’s       Maturation: Between any two observations
responses at the end of the interview. Although          subjects change in a variety of ways. Such
this preserves the continuity of the interview, it       changes can produce differences that are
is likely to induce greater bias because the delay       independent of the experimental treatments.
may lead to the interviewer forgetting some of           The problem of maturation is more acute in
the details. It is these forgotten details that are      protracted educational studies than in brief
most likely to be the ones that disagree with the        laboratory experiments.
interviewer’s own expectations.                          Statistical regression: Like maturation effects,
                                                         regression effects increase systematically with
                                                         the time interval between pretests and
Validity and reliability in experiments
                                                         post-tests. Statistical regression occurs in
As we have seen, the fundamental purpose of              educational (and other) research due to the
experimental design is to impose control over            unreliability of measuring instruments and to
conditions that would otherwise cloud the true           extraneous factors unique to each experimental
effects of the independent variables upon the            group. Regression means, simply, that subjects
dependent variables.                                     scoring highest on a pretest are likely to score
   Clouding conditions that threaten to jeopardize       relatively lower on a post-test; conversely,
the validity of experiments have been identi-            those scoring lowest on a pretest are likely
fied by Campbell and Stanley (1963), Bracht and           to score relatively higher on a post-test. In
Glass (1968) and Lewis-Beck (1993), conditions           short, in pretest-post-test situations, there is

         regression to the mean. Regression effects can       a number of factors (adapted from Campbell and
         lead the educational researcher mistakenly           Stanley 1963; Bracht and Glass 1968; Hammersley
         to attribute post-test gains and losses to low       and Atkinson 1983; Vulliamy 1990; Lewis-Beck
         scoring and high scoring respectively.               1993) that jeopardize external validity.
         Testing: Pretests at the beginning of experi-
         ments can produce effects other than those due          Failure to describe independent variables explicitly:
         to the experimental treatments. Such effects            Unless independent variables are adequately
         can include sensitizing subjects to the true            described by the researcher, future replications
         purposes of the experiment and practice ef-             of the experimental conditions are virtually
         fects which produce higher scores on post-test          impossible.
         measures.                                               Lack of representativeness of available and
         Instrumentation: Unreliable tests or instru-            target populations: While those participating
         ments can introduce serious errors into ex-             in the experiment may be representative of
         periments. With human observers or judges               an available population, they may not be
         or changes in instrumentation and calibration,          representative of the population to which the
         error can result from changes in their skills and       experimenter seeks to generalize the findings,
         levels of concentration over the course of the          i.e. poor sampling and/or randomization.
         experiment.                                             Hawthorne effect: Medical research has long
         Selection: Bias may be introduced as a result           recognized the psychological effects that
         of differences in the selection of subjects             arise out of mere participation in drug
         for the comparison groups or when intact                experiments, and placebos and double-
         classes are employed as experimental or control         blind designs are commonly employed to
         groups. Selection bias, moreover, may interact          counteract the biasing effects of participation.
         with other factors (history, maturation, etc.)          Similarly, so-called Hawthorne effects threaten
         to cloud even further the effects of the                to contaminate experimental treatments in
         comparative treatments.                                 educational research when subjects realize their
         Experimental mortality: The loss of subjects            role as guinea pigs.
         through dropout often occurs in long-running            Inadequate operationalizing of dependent vari-
         experiments and may result in confounding               ables: Dependent variables that experimenters
         the effects of the experimental variables, for          operationalize must have validity in the non-
         whereas initially the groups may have been              experimental setting to which they wish to
         randomly selected, the residue that stays the           generalize their findings. A paper and pencil
         course is likely to be different from the unbiased      questionnaire on career choice, for example,
         sample that began it.                                   may have little validity in respect of the actual
         Instrument reactivity: The effects that the             employment decisions made by undergraduates
         instruments of the study exert on the people in         on leaving university.
         the study (see also Vulliamy et al. 1990).              Sensitization/reactivity to experimental conditions:
         Selection-maturation interaction: This can occur        As with threats to internal validity, pretests
         where there is a confusion between the research         may cause changes in the subjects’ sensitivity
         design effects and the variable’s effects.              to the experimental variables and thus cloud
                                                                 the true effects of the experimental treatment.
                                                                 Interaction effects of extraneous factors and
      Threats to external validity
                                                                 experimental treatments: All of the above threats
      Threats to external validity are likely to limit           to external validity represent interactions of
      the degree to which generalizations can be made            various clouding factors with treatments. As
      from the particular experimental conditions to             well as these, interaction effects may also
      other populations or settings. We summarize here           arise as a result of any or all of those factors
                                                         VALIDITY AND RELIABILITY IN QUESTIONNAIRES            157

   identified under the section on ‘Threats to           questionnaires would have given the same

                                                                                                                 Chapter 6
   internal validity’.                                  distribution of answers as did the returnees. The
   Invalidity or unreliability of instruments: The      question of accuracy can be checked by means
   use of instruments which yield data in which         of the intensive interview method, a technique
   confidence cannot be placed (see below on             consisting of twelve principal tactics that include
   tests).                                              familiarization, temporal reconstruction, probing
   Ecological validity, and its partner, the extent     and challenging. The interested reader should
   to which behaviour observed in one context           consult Belson (1986: 35-8).
   can be generalized to another: Hammersley               The problem of non-response – the issue of
   and Atkinson (1983: 10) comment on the               ‘volunteer bias’ as Belson (1986) calls it – can,
   serious problems that surround attempts to           in part, be checked on and controlled for,
   relating inferences from responses gained under      particularly when the postal questionnaire is sent
   experimental conditions, or from interviews, to      out on a continuous basis. It involves follow-
   everyday life.                                       up contact with non-respondents by means of
                                                        interviewers trained to secure interviews with
   By way of summary, we have seen that an              such people. A comparison is then made between
experiment can be said to be internally valid           the replies of respondents and non-respondents.
to the extent that, within its own confines, its         Further, Hudson and Miller (1997) suggest several
results are credible (Pilliner 1973); but for those     strategies for maximizing the response rate to
results to be useful, they must be generalizable        postal questionnaires (and, thereby, to increase
beyond the confines of the particular experiment.        reliability). They involve:
In a word, they must be externally valid
also: see also Morrison (2001b) for a critique
                                                           including stamped addressed envelopes
of randomized controlled experiments and the
                                                           organizing multiple rounds of follow-up to
problems of generalizability. Pilliner (1973) points
                                                           request returns (maybe up to three follow-ups)
to a lopsided relationship between internal and
                                                           stressing the importance and benefits of the
external validity. Without internal validity an
experiment cannot possibly be externally valid.
                                                           stressing the importance of, and benefits to,
But the converse does not necessarily follow; an
                                                           the client group being targeted (particularly if
internally valid experiment may or may not have
                                                           it is a minority group that is struggling to have
external validity. Thus, the most carefully designed
                                                           a voice)
experiment involving a sample of Welsh-speaking
                                                           providing interim data from returns to non-
children is not necessarily generalizable to a target
                                                           returners to involve and engage them in the
population which includes non-Welsh-speaking
                                                           checking addresses and changing them if
   It follows, then, that the way to good
experimentation in schools, or indeed any other
                                                           following up questionnaires with a personal
organizational setting, lies in maximizing both            telephone call
internal and external validity.
                                                           tailoring follow-up requests to individuals
                                                           (with indications to them that they are
                                                           personally known and/or important to the
Validity and reliability in questionnaires
                                                           research – including providing respondents
Validity of postal questionnaires can be seen              with clues by giving some personal information
from two viewpoints (Belson l986). First, whether          to show that they are known) rather than
respondents who complete questionnaires do                 blanket generalized letters
so accurately, honestly and correctly; and                 detailing features of the questionnaire itself
second, whether those who fail to return their             (ease of completion, time to be spent,

         sensitivity of the questions asked, length of          about the subjective and idiosyncratic nature of
         the questionnaire)                                     the participant observation study are about its
         issuing invitations to a follow-up interview           external validity. How do we know that the results
         (face-to-face or by telephone)                         of this one piece of research are applicable to
         providing encouragement to participate by a            other situations? Fears that observers’ judgements
         friendly third party                                   will be affected by their close involvement in
         understanding the nature of the sample                 the group relate to the internal validity of the
         population in depth, so that effective targeting       method. How do we know that the results of
         strategies can be used.                                this one piece of research represent the real
      The advantages of the questionnaire over inter-           thing, the genuine product? In Chapter 4 on
      views, for instance, are: it tends to be more reliable;   sampling, we refer to a number of techniques
      because it is anonymous, it encourages greater hon-       (quota sampling, snowball sampling, purposive
      esty (though, of course, dishonesty and falsification      sampling) that researchers employ as a way
      might not be able to be discovered in a question-         of checking on the representativeness of the
      naire); it is more economical than the interview in       events that they observe and of cross-checking
      terms of time and money; and there is the possibil-       their interpretations of the meanings of those
      ity that it can be mailed. Its disadvantages, on the      events.
      other hand, are: there is often too low a percentage        In addition to external validity, participant
      of returns; the interviewer is unable to answer ques-     observation also has to be rigorous in its internal
      tions concerning both the purpose of the interview        validity checks. There are several threats to
      and any misunderstandings experienced by the in-          validity and reliability here, for example:
      terviewee, for it sometimes happens in the case of           the researcher, in exploring the present, may
      the latter that the same questions have different            be unaware of important antecedent events
      meanings for different people; if only closed items          informants may be unrepresentative of the
      are used, the questionnaire may lack coverage or             sample in the study
      authenticity; if only open items are used, respon-           the presence of the observer might bring about
      dents may be unwilling to write their answers                different behaviours (reactivity and ecological
      for one reason or another; questionnaires present            validity)
      problems to people of limited literacy; and an in-           the researcher might ‘go native’, becoming too
      terview can be conducted at an appropriate speed             attached to the group to see it sufficiently
      whereas questionnaires are often filled in hurriedly.         dispassionately.
      There is a need, therefore, to pilot questionnaires
      and refine their contents, wording, length, etc. as           To address this Denzin (1970a) suggests
      appropriate for the sample being targeted.                triangulation of data sources and methodologies.
         One central issue in considering the reliability       Chapter 18 discusses the principal ways of
      and validity of questionnaire surveys is that of          overcoming problems of reliability and validity
      sampling. An unrepresentative, skewed sample,             in observational research in naturalistic inquiry.
      one that is too small or too large can easily distort     In essence it is suggested that the notion
      the data, and indeed, in the case of very small           of ‘trustworthiness’ (Lincoln and Guba 1985)
      samples, prohibit statistical analysis (Morrison          replaces more conventional views of reliability
      1993). The issue of sampling was covered in               and validity, and that this notion is devolved on
      Chapter 4.                                                issues of credibility, confirmability, transferability and
                                                                dependability. Chapter 18 indicates how these areas
                                                                can be addressed.
      Validity and reliability in observations
                                                                   If observational research is much more
      There are questions about two types of validity in        structured in its nature, yielding quantitative data,
      observation-based research. In effect, comments           then the conventions of intra- and inter-rater
                                                                       VALIDITY AND RELIABILITY IN TESTS         159

reliability apply. Here steps are taken to ensure that      situational factors: the psychological and

                                                                                                                   Chapter 6
observers enter data into the appropriate categories        physical conditions for the test – the context
consistently (i.e. intra- and inter-rater reliability)      test marker factors: idiosyncrasy and subjectiv-
and accurately. Further, to ensure validity, a              ity
pilot must have been conducted to ensure                    instrument variables: poor domain sampling,
that the observational categories themselves are            errors in sampling tasks, the realism of the
appropriate, exhaustive, discrete, unambiguous              tasks and relatedness to the experience of the
and effectively operationalize the purposes of the          testees, poor question items, the assumption or
research.                                                   extent of unidimensionality in item response
                                                            theory, length of the test, mechanical errors,
Validity and reliability in tests                           scoring errors, computer errors.

The researcher will have to judge the place and
                                                         Sources of unreliability
significance of test data, not forgetting the problem
of the Hawthorne effect operating negatively or          There are several threats to reliability in tests and
positively on students who have to undertake             examinations, particularly tests of performance
the tests. There is a range of issues which might        and achievement, for example (Cunningham
affect the reliability of the test – for example,        1998; Airasian 2001), with respect to examiners
the time of day, the time of the school year,            and markers:
the temperature in the test room, the perceived
                                                            errors in marking: e.g. attributing, adding and
importance of the test, the degree of formality
                                                            transfer of marks
of the test situation, ‘examination nerves’, the
                                                            inter-rater reliability: different markers giving
amount of guessing of answers by the students
                                                            different marks for the same or similar pieces
(the calculation of standard error which the test
                                                            of work
demonstrates feature here), the way that the test is
                                                            inconsistency in the marker: e.g. being harsh in
administered, the way that the test is marked, the
                                                            the early stages of the marking and lenient in
degree of closure or openness of test items. Hence
                                                            the later stages of the marking of many scripts
the researcher who is considering using testing
                                                            variations in the award of grades: for work that
as a way of acquiring research data must ensure
                                                            is close to grade boundaries, some markers may
that it is appropriate, valid and reliable (Linn
                                                            place the score in a higher or lower category
1993; Borsboom et al. 2004).
                                                            than other markers
   Wolf (1994) suggests four main factors that
                                                            the Halo effect: a student who is judged
might affect reliability: the range of the group that
                                                            to do well or badly in one assessment is
is being tested, the group’s level of proficiency,
                                                            given undeserved favourable or unfavourable
the length of the measure (the longer the test the
                                                            assessment respectively in other areas.
greater the chance of errors), and the way in which
reliability is calculated. Fitz-Gibbon (1997: 36)           With reference to the students and teachers
argues that, other things being equal, longer tests      themselves, there are several sources of unrelia-
are more reliable than shorter tests. Additionally       bility:
there are several ways in which reliability might be        Motivation and interest in the task have a
compromised in tests. Feldt and Brennan (1993)              considerable effect on performance. Clearly,
suggest four types of threat to reliability:                students need to be motivated if they are
   individuals: their motivation, concentration,            going to make a serious attempt at any test
   forgetfulness, health, carelessness, guessing,           that they are required to undertake, where
   their related skills (e.g. reading ability, their        motivation is intrinsic (doing something for
   usedness to solving the type of problem set, the         its own sake) or extrinsic (doing something
   effects of practice)                                     for an external reason, e.g. obtaining a

         certificate or employment or entry into higher       pupils before the actual testing session. Ideally,
         education). The results of a test completed         test instructions should be simple, direct and
         in a desultory fashion by resentful pupils          as brief as possible.
         are hardly likely to supply the students’           The Hawthorne effect, wherein, in this
         teacher with reliable information about             context, simply informing students that this
         the students’ capabilities (Wiggins 1998).          is an assessment situation will be enough to
         Motivation to participate in test-taking            disturb their performance – for the better or
         sessions is strongest when students have            the worse (either case not being a fair reflection
         been helped to see its purpose, and where           of their usual abilities).
         the examiner maintains a warm, purposeful           Distractions, including superfluous informa-
         attitude toward them during the testing session     tion, will have an effect.
         (Airasian 2001).                                    Students respond to the tester in terms of
         The relationship (positive to negative)             their perceptions of what he/she expects of
         between the assessor and the testee exerts          them (Haladyna 1997; Tombari and Borich
         an influence on the assessment. This                 1999; Stiggins, 2001).
         takes on increasing significance in teacher          The time of the day, week, month will exert
         assessment, where the students know the             an influence on performance. Some students
         teachers personally and professionally – and        are fresher in the morning and more capable of
         vice versa – and where the assessment situation     concentration (Stiggins 2001).
         involves face-to-face contact between the           Students are not always clear on what they
         teacher and the student. Both test-takers           think is being asked in the question; they may
         and test-givers mutually influence one another       know the right answer but not infer that this is
         during examinations, oral assessments and the       what is required in the question.
         like (Harlen 1994). During the test situation,      The students may vary from one question to
         students respond to such characteristics of         another – a student may have performed better
         the evaluator as the person’s sex, age and          with a different set of questions which tested
         personality.                                        the same matters. Black (1998) argues that two
         The conditions – physical, emotional, so-           questions which, to the expert, may seem to be
         cial – exert an influence on the assessment,         asking the same thing but in different ways, to
         particularly if they are unfamiliar. Wherever       the students might well be seen as completely
         possible, students should take tests in familiar    different questions.
         settings, preferably in their own classrooms un-    Students (and teachers) practise test-like
         der normal school conditions. Distractions in       materials, which, even though scores are
         the form of extraneous noise, walking about         raised, might make them better at taking tests
         the room by the examiner and intrusions into        but the results might not indicate increased
         the room, all have significant impact upon the       performance.
         scores of the test-takers, particularly when they   A student may be able to perform a specific skill
         are younger pupils (Gipps 1994). An important       in a test but not be able to select or perform it
         factor in reducing students’ anxiety and tension    in the wider context of learning.
         during an examination is the extent to which        Cultural, ethnic and gender background affect
         they are quite clear about what exactly they are    how meaningful an assessment task or activity
         required to do. Simple instructions, clearly and    is to students, and meaningfulness affects their
         calmly given by the examiner, can significantly      performance.
         lower the general level of tension in the test-     Students’ personalities may make a difference
         room. Teachers who intend to conduct testing        to their test performance.
         sessions may find it beneficial in this respect to    Students’ learning strategies and styles may
         rehearse the instructions they wish to give to      make a difference to their test performance.
                                                                    VALIDITY AND RELIABILITY IN TESTS         161

   Marking practices are not always reliable,             A single error early on in a complex sequence

                                                                                                                Chapter 6
   markers may be being too generous, marking             may confound the later stages of the sequence
   by effort and ability rather than performance.         (within a question or across a set of questions),
   The context in which the task is presented             even though the student might have been able
   affects performance: some students can perform         to perform the later stages of the sequence,
   the task in everyday life but not under test           thereby preventing the student from gaining
   conditions.                                            credit for all she or he can, in fact, do.
                                                          Questions might favour boys more than girls or
With regard to the test items themselves, there may
                                                          vice versa.
be problems (e.g. test bias):
                                                          Essay questions favour boys if they concern
   The task itself may be multidimensional,               impersonal topics and girls if they concern
   for example, testing ‘reading’ may require             personal and interpersonal topics (Haladyna
   several components and constructs. Students            1997; Wedeen et al. 2002).
   can execute a Mathematics operation in the             Boys perform better than girls on multiple
   Mathematics class but they cannot perform the          choice questions and girls perform better than
   same operation in, for example, a Physics class;       boys on essay-type questions (perhaps because
   students will disregard English grammar in a           boys are more willing than girls to guess in
   Science class but observe it in an English class.      multiple-choice items), and girls perform better
   This raises the issue of the number of contexts        in written work than boys.
   in which the behaviour must be demonstrated            Questions and assessment may be culture-
   before a criterion is deemed to have been              bound: what is comprehensible in one culture
   achieved (Cohen et al. 2004). The question of          may be incomprehensible in another.
   transferability of knowledge and skills is also        The test may be so long, in order to
   raised in this connection. The context of the          ensure coverage, that boredom and loss of
   task affects the student’s performance.                concentration may impair reliability.
   The validity of the items may be in question.
   The language of the assessment and the
                                                       Hence specific contextual factors can exert a
   assessor exerts an influence on the testee, for
                                                       significant influence on learning and this has to be
   example if the assessment is carried out in the
                                                       recognised in conducting assessments, to render
   testee’s second language or in a ‘middle-class’
                                                       an assessment as unthreatening and natural as
   code (Haladyna 1997).
   The readability level of the task can exert an
                                                          Harlen (1994: 140-2) suggests that incon-
   influence on the test, e.g. a difficulty in reading
                                                       sistency and unreliability in teacher-based and
   might distract from the purpose of a test which
                                                       school-based assessment may derive from differ-
   is of the use of a mathematical algorithm.
                                                       ences in:
   The size and complexity of numbers or
   operations in a test (e.g. of Mathematics) might
   distract the testee who actually understands the       interpreting the assessment purposes, tasks and
   operations and concepts.                               contents, by teachers or assessors
   The number and type of operations and stages           the actual task set, or the contexts and
   to a task: the students might know how to              circumstances surrounding the tasks (e.g. time
   perform each element, but when they are                and place)
   presented in combination the size of the task          how much help is given to the test-takers
   can be overwhelming.                                   during the test
   The form and presentation of questions affects         the degree of specificity in the marking criteria
   the results, giving variability in students’           the application of the marking criteria and the
   performances.                                          grading or marking system that accompanies it

         how much additional information about the               particular issue in Computer Adaptive Testing
         student or situation is being referred to in the        see chapter 19: Thissen 1990)
         assessment.                                             ensuring effective levels of item discriminabil-
                                                                 ity and item difficulty.
      Harlen (1994) advocates the use of a range of
      moderation strategies, both before and after the        Reliability has to be not only achieved but also
      tests, including:                                       seen to be achieved, particularly in ‘high stakes’
                                                              testing (where a lot hangs on the results of the test,
         statistical reference/scaling tests                  e.g. entrance to higher education or employment).
         inspection of samples (by post or by visit)          Hence the procedures for ensuring reliability must
         group moderation of grades                           be transparent. The difficulty here is that the
         post-hoc adjustment of marks                         more one moves towards reliability as defined
         accreditation of institutions                        above, the more the test will become objective,
         visits of verifiers                                   the more students will be measured as though they
         agreement panels                                     are inanimate objects, and the more the test will
         defining marking criteria                             become decontextualized.
         exemplification                                          An alternative form of reliability, which is
         group moderation meetings.                           premissed on a more constructivist psychology,
                                                              emphasizes the significance of context, the
      While moderation procedures are essentially post-       importance of subjectivity and the need to engage
      hoc adjustments to scores, agreement trials and         and involve the testee more fully than a simple test.
      practice-marking can be undertaken before the           This rehearses the tension between positivism and
      administration of a test, which is particularly         more interpretive approaches outlined in Chapter
      important if there are large numbers of scripts         1 of this book. Objective tests, as described in
      or several markers.                                     this chapter, lean strongly towards the positivist
         The issue here is that the results as well as the    paradigm, while more phenomenological and
      instruments should be reliable. Reliability is also     interpretive paradigms of social science research
      addressed by:                                           will emphasize the importance of settings, of
                                                              individual perceptions, of attitudes, in short, of
         calculating coefficients of reliability, split-half
                                                              ‘authentic’ testing (e.g. by using non-contrived,
         techniques, the Kuder-Richardson formula,
                                                              non-artificial forms of test data, for example
         parallel/equivalent forms of a test, test/retest
                                                              portfolios, documents, course work, tasks that
         methods, the alpha coefficient
                                                              are stronger in realism and more ‘hands on’).
         calculating and controlling the standard error
                                                              Though this latter adopts a view which is closer
         of measurement
                                                              to assessment rather than narrowly ‘testing’,
         increasing the sample size (to maximize the
                                                              nevertheless the two overlap, both can yield marks,
         range and spread of scores in a norm-
                                                              grades and awards, both can be formative as well
         referenced test), though criterion-referenced
                                                              as summative, both can be criterion-referenced.
         tests recognize that scores may bunch around
                                                                 With regard to validity, it is important to note
         the high level (in mastery learning for
                                                              here that an effective test will adequately ensure
         example), i.e. that the range of scores might
                                                              the following:
         be limited, thereby lowering the correlation
         coefficients that can be calculated                      Content validity (e.g. adequate and representa-
         increasing the number of observations made              tive coverage of programme and test objectives
         and items included in the test (in order to             in the test items, a key feature of domain
         increase the range of scores)                           sampling): this is achieved by ensuring that
         ensuring effective domain sampling of items             the content of the test fairly samples the class
         in tests based on item response theory (a               or fields of the situations or subject matter
                                                                    VALIDITY AND RELIABILITY IN TESTS           163

in question. Content validity is achieved by             In this respect construct validity also subsumes

                                                                                                                  Chapter 6
making professional judgements about the rel-            content and criterion-related validity. It is ar-
evance and sampling of the contents of the               gued (Loevinger 1957) that, in fact, construct
test to a particular domain. It is concerned             validity is the queen of the types of validity be-
with coverage and representativeness rather              cause it is subsumptive and because it concerns
than with patterns of response or scores. It is          constructs or explanations rather than method-
a matter of judgement rather than measure-               ological factors. Construct validity is threat-
ment (Kerlinger 1986). Content validity will             ened by under-representation of the construct,
need to ensure several features of a test (Wolf          i.e. the test is too narrow and neglects signifi-
1994): (a) test coverage (the extent to which            cant facets of a construct, and by the inclusion
the test covers the relevant field); (b) test rel-        of irrelevancies – excess reliable variance.
evance (the extent to which the test items               Concurrent validity is where the results of the
are taught through, or are relevant to, a par-           test concur with results on other tests or in-
ticular programme); (c) programme coverage               struments that are testing/assessing the same
(the extent to which the programme covers                construct/performance – similar to predictive
the overall field in question).                           validity but without the time dimension. Con-
Criterion-related validity is where a high correla-      current validity can occur simultaneously with
tion coefficient exists between the scores on the         another instrument rather than after some time
test and the scores on other accepted tests of           has elapsed.
the same performance: this is achieved by com-           Face validity is where, superficially, the test ap-
paring the scores on the test with one or more           pears – at face value – to test what it is designed
variables (criteria) from other measures or tests        to test.
that are considered to measure the same fac-             Jury validity is an important element in con-
tor. Wolf (1994) argues that a major problem             struct validity, where it is important to agree
facing test devisers addressing criterion-related        on the conceptions and operationalization of
validity is the selection of the suitable criterion      an unobservable construct.
measure. He cites the example of the difficulty           Predictive validity is where results on a test accu-
of selecting a suitable criterion of academic            rately predict subsequent performance – akin
achievement in a test of academic aptitude.              to criterion-related validity.
The criterion must be: relevant (and agreed to           Consequential validity is where the inferences
be relevant); free from bias (i.e. where external        that can be made from a test are sound.
factors that might contaminate the criterion             Systemic validity (Frederiksen and Collins
are removed); reliable – precise and accurate;           1989) is where programme activities both
capable of being measured or achieved.                   enhance test performance and enhance perfor-
Construct validity (e.g. the clear related-              mance of the construct that is being addressed
ness of a test item to its proposed con-                 in the objective. Cunningham (1998) gives an
struct/unobservable quality or trait, demon-             example of systemic validity where, if the test
strated by both empirical data and logical               and the objective of vocabulary performance
analysis and debate, i.e. the extent to which            leads to testees increasing their vocabulary,
particular constructs or concepts can give an            then systemic validity has been addressed.
account for performance on the test): this is
achieved by ensuring that performance on the             To ensure test validity, then, the test must
test is fairly explained by particular appropriate    demonstrate fitness for purpose as well as
constructs or concepts. As with content valid-        addressing the several types of validity outlined
ity, it is not based on test scores, but is more a    above. The most difficult for researchers to
matter of whether the test items are indicators       address, perhaps, is construct validity, for it
of the underlying, latent construct in question.      argues for agreement on the definition and

      operationalization of an unseen, half-guessed-at          Box 6.1
      construct or phenomenon. The community of                 Principal sources of bias in life history research
      scholars has a role to play here. For a full discussion
      of validity see Messick (1993). To conclude this
                                                                     Source:       Informant
      chapter, we turn briefly to consider validity and
                                                                     Is misinformation (unintended) given?
      reliability in life history accounts.
                                                                     Has there been evasion?
                                                                     Is there evidence of direct lying and deception?
      Validity and reliability in life histories                     Is a ‘front’ being presented?
      Three central issues underpin the quality of data              What may the informant ‘take for granted’ and
      generated by life history methodology. They are to             hence not reveal?
      do with representativeness, validity and reliability.          How far is the informant ‘pleasing you’?
      Plummer draws attention to a frequent criticism                How much has been forgotten?
      of life history research, namely that its cases                How much may be self-deception?
      are atypical rather than representative. To avoid              Source:       Researcher
      this charge, he urges intending researchers to,                Attitudes of researcher: age, gender, class, race,
      ‘work out and explicitly state the life history’s              religion, politics etc.
      relationship to a wider population’ (Plummer                   Demeanour of researcher: dress, speech, body
      1983) by way of appraising the subject on                      language etc.
      a continuum of representativeness and non-                     Personality of researcher: anxiety, need for
      representativeness.                                            approval, hostility, warmth etc.
         Reliability in life history research hinges upon            Scientific role of researcher: theory held (etc.),
      the identification of sources of bias and the                   researcher expectancy
      application of techniques to reduce them. Bias                 Source:       The interaction
      arises from the informant, the researcher, and the             The encounter needs to be examined. Is bias
      interactional encounter itself. Box 6.1, adapted               coming from:
      from Plummer (1983), provides a checklist of some              The physical setting – ‘social space’?
      aspects of bias arising from these principal sources.          The prior interaction?
         Several validity checks are available to                    Non-verbal communication?
      intending researchers. Plummer (1983) identifies                Vocal behaviour?
      the following:
         The subject of the life history may present            Source: adapted from Plummer 1983: Table 5.2, p. 103
         an autocritique of it, having read the entire
         A comparison may be made with similar                      A comparison may be made by interviewing
         written sources by way of identifying points               other informants.
         of major divergence or similarity.                        Essentially, the validity of any life history lies in
         A comparison may be made with official records          its ability to represent the informant’s subjective
         by way of imposing accuracy checks on the life         reality, that is to say, his or her definition of the
         history.                                               situation.
Part Three

Styles of educational research

It is important to distinguish between design,       are by no means exhaustive, we suggest that they
methodology and instrumentation. Too often           cover the major styles of research methodology.
methods are confused with methodology and            These take in quantitative as well as qualitative
methodology is confused with design. Part Two        research, together with small-scale and large-scale
provided an introduction to design issues and        approaches. As with the previous parts, the key
this part examines different styles, kinds of, and   here is the application of the notion of fitness for
approaches to, research, separating them from        purpose. We do not advocate slavish adherence to a
methods – instruments for data collection. We        single methodology in research; indeed combining
identify eight main styles of educational research   methodologies may be appropriate for the research
in this section, including a new chapter on the      in hand. The intention here is to shed light on the
developing field of Internet-based research and       different styles of research, locating them in the
computer usage. Although we recognize that these     paradigms of research introduced in Part One.
7        Naturalistic and ethnographic research

Elements of naturalistic inquiry                       Inquiry is value-bound.
                                                       Inquiries are influenced by inquirer values as
The social and educational world is a messy
                                                       expressed in the choice of a problem, evaluand
place, full of contradictions, richness, complexity,
                                                       or policy option, and in the framing, bounding
connectedness, conjunctions and disjunctions. It
                                                       and focusing of that problem, evaluand or
is multilayered, and not easily susceptible to the
                                                       policy option.
atomization process inherent in much numerical
                                                       Inquiry is influenced by the choice of the
research. It has to be studied in total rather
                                                       paradigm that guides the investigation into
than in fragments if a true understanding is
                                                       the problem.
to be reached. Chapter 1 indicated that several
                                                       Inquiry is influenced by the choice of
approaches to educational research are contained
                                                       the substantive theory utilized to guide the
in the paradigm of qualitative, naturalistic and
                                                       collection and analysis of data and in the
ethnographic research. The characteristics of that
                                                       interpretation of findings.
paradigm (Boas 1943; Blumer 1969; Lincoln and
                                                       Inquiry is influenced by the values that inhere
Guba 1985; Woods 1992; LeCompte and Preissle
                                                       in the context.
1993) include the following:
                                                       Inquiry is either value-resident (reinforcing
   Humans actively construct their own meanings        or congruent) or value-dissonant (conflicting).
   of situations.                                      Problem, evaluand, or policy option, paradigm,
   Meaning arises out of social situations and         theory and context must exhibit congruence
   is handled through interpretive processes (see      (value-resonance) if the inquiry is to produce                 meaningful results.
   9780415368780 – Chapter 7, file 7.1. ppt).           Research must include ‘thick descrip-
   Behaviour and, thereby, data are socially           tions’ (Geertz 1973b) of the contextualized
   situated, context-related, context-dependent        behaviour.
   and context-rich. To understand a situation         The attribution of meaning is continuous and
   researchers need to understand the context          evolving over time.
   because situations affect behaviour and             People are deliberate, intentional and creative
   perspectives and vice versa.                        in their actions.
   Realities are multiple, constructed and holistic.   History and biography intersect – we create our
   Knower and known are interactive and                own futures but not necessarily in situations of
   inseparable.                                        our own choosing.
   Only time-bound and context-bound working           Social research needs to examine situations
   hypotheses (idiographic statements) are possi-      through the eyes of the participants – the task
   ble.                                                of ethnographies, as Malinowski (1922: 25)
   All entities are in a state of mutual               observed, is ‘to grasp the point of view of the
   simultaneous shaping, so that it is impossible      native [sic], his [sic] view of the world and in
   to distinguish causes from effects.                 relation to his life’.

         Researchers are the instruments of the research        Theory emerges rather than is pre-ordinate. A
         (Eisner 1991).                                         priori theory is replaced by grounded theory.
         Researchers generate rather than test hypo-            Research designs emerge over time (and as the
         theses.                                                sampling changes over time).
         Researchers do not know in advance what they           The outcomes of the research are negotiated.
         will see or what they will look for.                   The natural mode of reporting is the case study.
         Humans are anticipatory beings.                        Nomothetic interpretation is replaced by
         Human phenomena seem to require even more              idiographic interpretation.
         conditional stipulations than do other kinds.          Applications are tentative and pragmatic.
         Meanings and understandings replace proof.             The focus of the study determines its
         Generalizability is interpreted as generalizabil-      boundaries.
         ity to identifiable, specific settings and subjects      Trustworthiness and its components replace
         rather than universally.                               more conventional views of reliability and
         Situations are unique.                                 validity.
         The processes of research and behaviour are as
         important as the outcomes.                          LeCompte and Preissle (1993) suggest that
         People, situations, events and objects have         ethnographic research is a process involving
         meaning conferred upon them rather than             methods of inquiry, an outcome and a resultant
         possessing their own intrinsic meaning.             record of the inquiry. The intention of the
         Social research should be conducted in natural,     research is to create as vivid a reconstruction
         uncontrived, real world settings with as little     as possible of the culture or groups being
         intrusiveness as possible by the researcher.        studied (p. 235). There are several purposes
         Social reality, experiences and social phe-         of qualitative research, for example, description
         nomena are capable of multiple, sometimes           and reporting, the creation of key concepts,
         contradictory interpretations and are available     theory generation and testing. LeCompte and
         to us through social interaction.                   Preissle (1993) indicate several key elements of
         All factors, rather than a limited number of        ethnographic approaches:
         variables, have to be taken into account.
         Data are analysed inductively, with constructs         Phenomenological        data      are   elicited
         deriving from the data during the research.            (LeCompte and Preissle 1993: 3).
         Theory generation is derivative – grounded             The world view of the participants is
          (Glaser and Strauss 1967) – the data suggest          investigated and represented – their ‘definition
         the theory rather than vice versa.                     of the situation’ (Thomas 1923).
                                                                Meanings are accorded to phenomena by
      Lincoln and Guba (1985: 39–43) tease out the
                                                                both the researcher and the participants; the
      implications of these axioms:
                                                                process of research, therefore, is hermeneutic,
         Studies must be set in their natural settings as       uncovering meanings (LeCompte and Preissle
         context is heavily implicated in meaning.              1993: 31–2).
         Humans are the research instrument.                    The constructs of the participants are used to
         Utilization of tacit knowledge is inescapable.         structure the investigation.
         Qualitative methods sit more comfortably than          Empirical data are gathered in their naturalistic
         quantitative methods with the notion of the            setting (unlike laboratories or in controlled
         human-as-instrument.                                   settings as in other forms of research, where
         Purposive sampling enables the full scope of           variables are manipulated).
         issues to be explored.                                 Observational techniques are used extensively
         Data analysis is inductive rather than a priori        (both participant and non-participant) to
         and deductive.                                         acquire data on real-life settings.
                                                                    ELEMENTS OF NATURALISTIC INQUIRY            169

   The research is holistic, that is, it seeks          than objective knowledge. With regard to the

                                                                                                                  Chapter 7
   a description and interpretation of ‘total           latter the authors distinguish between emic
   phenomena’.                                          approaches (as in the term ‘phonemic’, where the
   There is a move from description and                 concern is to catch the subjective meanings placed
   data to inference, explanation, suggestions of       on situations by participants) and etic approaches
   causation, and theory generation.                    (as in the term ‘phonetic’, where the intention
   Methods are ‘multimodal’ and the ethno-              is to identify and understand the objective
   grapher is a ‘methodological omnivore’               or researcher’s meaning and constructions of a
   (LeCompte and Preissle 1993: 232).                   situation) (LeCompte and Preissle 1993: 45).
                                                           Woods (1992: 381), however, argues that some
  Hitchcock and Hughes (1989: 52–3) suggest             differences between quantitative and qualitative
that ethnographies involve                              research have been exaggerated. He proposes,
   the production of descriptive cultural knowl-        for example, that the 1970s witnessed an
   edge of a group                                      unproductive dichotomy between the two, the
   the description of activities in relation to a       former being seen as strictly in the hypothetico-
   particular cultural context from the point of        deductive mode (testing theories) and the
   view of the members of that group themselves         latter being seen as the inductive method used
   the production of a list of features constitutive    for generating theory. He suggests that the
   of membership in a group or culture                  epistemological contrast between the two is
   the description and analysis of patterns of social   overstated, as qualitative techniques can be used
   interaction                                          both for generating and testing theories.
   the provision as far as possible of ‘insider            Indeed Dobbert and Kurth-Schai (1992: 94–5)
   accounts’                                            urge not only that ethnographic approaches
   the development of theory.                           become more systematic but also that they study
                                                        and address regularities in social behaviour and
  Lofland (1971) suggests that naturalistic              social structure. The task of ethnographers is to
methods are intended to address three major             balance a commitment to catch the diversity,
questions:                                              variability, creativity, individuality, uniqueness
                                                        and spontaneity of social interactions (e.g.
   What are the characteristics of a social
                                                        by ‘thick descriptions’: Geertz 1973b) with a
                                                        commitment to the task of social science to
   What are the causes of the social phenomenon?
                                                        seek regularities, order and patterns within such
   What are the consequences of the social
                                                        diversity. As Durkheim (1950) noted, there are
                                                        ‘social facts’.
In this one can observe: the environment; people           Following this line, it is possible, therefore,
and their relationships; behaviour, actions and         to suggest that ethnographic research can
activities; verbal behaviour; psychological stances;    address issues of generalizability – a tenet of
histories; physical objects (Baker 1994: 241–4).        positivist research – interpreted as ‘comparability’
   There are several key differences between the        and ‘translatability’ (LeCompte and Preissle 1993:
naturalistic approach and that of the positivists       47). For comparability the characteristics of the
to whom we made reference in Chapter 1.                 group that is being studied need to be made explicit
LeCompte and Preissle (1993: 39–44) suggest             so that readers can compare them with other
that ethnographic approaches are concerned              similar or dissimilar groups. For translatability the
more with description rather than prediction,           analytic categories used in the research as well as
induction rather than deduction, generation             the characteristics of the groups are made explicit
rather than verification of theory, construction         so that meaningful comparisons can be made with
rather than enumeration, and subjectivities rather      other groups and disciplines.

        Spindler and Spindler (1992: 72–4) put forward        were, the ‘human instrument’ in the research,
      several hallmarks of effective ethnographies:           building on their tacit knowledge in addition
                                                              to their propositional knowledge, using meth-
         Observations have contextual relevance, both
                                                              ods that sit comfortably with human inquiry,
         in the immediate setting in which behaviour is
                                                              e.g. observations, interviews, documentary anal-
         observed and in further contexts beyond.
                                                              ysis and ‘unobtrusive’ methods (Lincoln and Guba
         Hypotheses emerge in situ as the study develops
                                                              1985: 187). The advantage of the ‘human instru-
         in the observed setting.
                                                              ment’ is his or her adaptability, responsiveness,
         Observation is prolonged and often repetitive.
                                                              knowledge, ability to handle sensitive matters,
         Events and series of events are observed
                                                              ability to see the whole picture, ability to clarify
         more than once to establish reliability in the
                                                              and summarize, to explore, to analyse, to examine
         observational data.
                                                              atypical or idiosyncratic responses (Lincoln and
         Inferences from observation and various forms
                                                              Guba 1985: 193–4).
         of ethnographic inquiry are used to address
                                                                 The main kinds of naturalistic inquiry are (Ar-
         insiders’ views of reality.
                                                              senault and Anderson 1998: 121; Flick 2004):
         A major part of the ethnographic task is to elicit
         sociocultural knowledge from participants,              case study: an investigation into a specific in-
         rendering social behaviour comprehensible.              stance or phenomenon in its real-life context
         Instruments, schedules, codes, agenda for               comparative studies: where several cases are
         interviews, questionnaires, etc. should be              compared on the basis of key areas of interest
         generated in situ, and should derive from               retrospective studies: which focus on biographies
         observation and ethnographic inquiry.                   of participants or which ask participants to look
         A transcultural, comparative perspective is             back on events and issues
         usually present, although often it is an unstated       snapshots: analyses of particular situations,
         assumption, and cultural variation (over space          events or phenomena at a single point in time
         and time) is natural.                                   longitudinal studies: which investigate issues or
         Some sociocultural knowledge that affects               people over time
         behaviour and communication under study is              ethnography: a portrayal and explanation of
         tacit/implicit, and may not be known even to            social groups and situations in their real-life
         participants or known ambiguously to others. It         contexts
         follows that one task for an ethnography is to          grounded theory: developing theories to explain
         make explicit to readers what is tacit/implicit         phenomena, the theories emerging from the
         to informants.                                          data rather than being prefigured or predeter-
         The ethnographic interviewer should not                 mined
         frame or predetermine responses by the kinds            biography: individual or collective
         of questions that are asked, because the                phenomenology: seeing things as they really
         informants themselves have the emic, native             are and establishing the meanings of things
         cultural knowledge.                                     through illumination and explanation rather
         In order to collect as much live data as possible,      than through taxonomic approaches or abstrac-
         any technical device may be used.                       tions, and developing theories through the dia-
         The ethnographer’s presence should be                   logic relationships of researcher to researched.
         declared and his or her personal, social and
         interactional position in the situation should       The main methods for data collection in
         be described.                                        naturalistic inquiry (Hammersley and Atkinson
                                                              1983) are as follows:
      With ‘mutual shaping and interaction’ between
      researchers and participants taking place (Lincoln         participant observation
      and Guba 1985: 155), researchers become, as it             interviews and conversations
                                                                      PLANNING NATURALISTIC RESEARCH               171

     documents and field notes                               strains in conducting fieldwork because the

                                                                                                                     Chapter 7
     accounts                                               researcher’s own emotions, attitudes, beliefs,
     notes and memos.                                       values, characteristics enter the research;
                                                            indeed, the more this happens the less will
                                                            be the likelihood of gaining the participants’
Planning naturalistic research                              perspectives and meanings.
In many ways the issues in naturalistic research            The kinds of participation that the researcher
are not exclusive; they apply to other forms of             will undertake.
research, for example identifying the problem and           Issues of advocacy: where the researcher may be
research purposes; deciding the focus of the study;         expected to identify with the same emotions,
selecting the research design and instrumenta-              concerns and crises as the members of the
tion; addressing validity and reliability; ethical is-      group being studied and wishes to advance
sues; approaching data analysis and interpretation.         their cause, often a feature that arises at the
These are common to all research. More specifi-              beginning and the end of the research when
cally Wolcott (1992: 19) suggests that naturalistic         the researcher is considered to be a legitimate
researchers should address the stages of watching,          spokesperson for the group.
asking and reviewing, or, as he puts it, experi-            Role relationships.
encing, enquiring and examining. In naturalistic            Boundary maintenance in the research.
inquiry it is possible to formulate a more detailed         The maintenance of the balance between
set of stages that can be followed (Hitchcock               distance and involvement.
and Hughes 1989: 57–71; Bogdan and Biklen                   Ethical issues.
1992; LeCompte and Preissle 1993). These eleven             Reflexivity.
stages are presented below and are subsequently          Reflexivity recognizes that researchers are in-
dealt with later on in this chapter (see http://         escapably part of the social world that they are –              researching (Hammersley and Atkinson 1983: 14)
Chapter 7, file 7.2. ppt):                                and, indeed, that this social world is an already
 1    Locating a field of study.                          interpreted world by the actors, undermining the
 2    Addressing ethical issues.                         notion of objective reality. Researchers are in the
 3    Deciding the sampling.                             world and of the world. They bring their own
 4    Finding a role and managing entry into the         biographies to the research situation and partici-
      context.                                           pants behave in particular ways in their presence.
 5    Finding informants.                                Reflexivity suggests that researchers should ac-
 6    Developing and maintaining relations in the        knowledge and disclose their own selves in the
      field.                                              research, seeking to understand their part in, or
 7    Data collection in situ.                           influence on, the research. Rather than trying to
 8    Data collection outside the field.                  eliminate researcher effects (which is impossible,
 9    Data analysis.                                     as researchers are part of the world that they are
10    Leaving the field.                                  investigating), researchers should hold themselves
11    Writing the report.                                up to the light, echoing Cooley’s (1902) notion
                                                         of the ‘looking glass self’. As Hammersley and
These stages are shot through with a range of issues     Atkinson (1983) say:
that will affect the research:
                                                           He or she [the researcher] is the research instrument
     Personal issues: the disciplinary sympathies of       par excellence. The fact that behaviour and attitudes
     the researcher, researcher subjectivities and         are often not stable across contexts and that the
     characteristics. Hitchcock and Hughes (1989:          researcher may play a part in shaping the context
     56) indicate that there are several serious           becomes central to the analysis. . .. The theories we

        develop to explain the behaviour of the people we      Delamont 1992). Spradley (1979) sets out the
        study should also, where relevant, be applied to our   stages of: selecting a problem; collecting cultural
        own activities as researchers.                         data; analysing cultural data; formulating ethno-
                  (Hammersley and Atkinson 1983: 18–19)        graphic hypotheses; writing the ethnography. We
                                                               offer a fuller, eleven-stage model later in the
         Highly reflexive researchers will be acutely
      aware of the ways in which their selectivity,
                                                                  Like other styles of research, naturalistic
      perception, background and inductive processes
                                                               and qualitative methods will need to formulate
      and paradigms shape the research. They are
                                                               research questions which should be clear and
      research instruments. McCormick and James
                                                               unambiguous but open to change as the research
      (1988: 191) argue that combating reactivity
                                                               develops. Strauss (1987) terms these ‘generative
      through reflexivity requires researchers to monitor
                                                               questions’: they stimulate the line of investigation,
      closely and continually their own interactions with
                                                               suggest initial hypotheses and areas for data
      participants, their own reaction, roles, biases, and
                                                               collection, yet they do not foreclose the possibility
      any other matters that might affect the research.
                                                               of modification as the research develops. A balance
      This is addressed more fully in Chapter 5 on
                                                               has to be struck between having research questions
      validity, encompassing issues of triangulation and
                                                               that are so broad that they do not steer the research
      respondent validity.
                                                               in any particular direction, and so narrow that they
         Lincoln and Guba (1985: 226–47) set out ten
                                                               block new avenues of inquiry (Flick 2004: 150).
      elements in research design for naturalistic studies:
                                                                  Miles and Huberman (1994) identify two types
       1 Determining a focus for the inquiry.                  of qualitative research design: loose and tight.
       2 Determining the fit of paradigm to focus.              Loose research designs have broadly defined
       3 Determining the fit of the inquiry paradigm            concepts and areas of study, and, indeed, are open
         to the substantive theory selected to guide the       to changes of methodology. These are suitable,
         inquiry.                                              they suggest, when the researchers are experienced
       4 Determining where and from whom data will             and when the research is investigating new fields or
         be collected.                                         developing new constructs, akin to the flexibility
       5 Determining successive phases of the inquiry.         and openness of theoretical sampling of Glaser and
       6 Determining instrumentation.                          Strauss (1967). By contrast, a tight research design
       7 Planning data collection and recording                has narrowly restricted research questions and
         modes.                                                predetermined procedures, with limited flexibility.
       8 Planning data analysis procedures.                    These, the authors suggest, are useful when the
       9 Planning the logistics:                               researchers are inexperienced, when the research
             prior logistical considerations for the           is intended to look at particular specified issues,
             project as a whole                                constructs, groups or individuals, or when the
             the logistics of field excursions prior to         research brief is explicit.
             going into the field                                  Even though, in naturalistic research, issues
             the logistics of field excursions while in the     and theories emerge from the data, this does
             field                                              not preclude the value of having research
             the logistics of activities following field        questions. Flick (1998: 51) suggests three types of
             excursions                                        research questions in qualitative research, namely
             the logistics of closure and termination          those that are concerned, first, with describing
      10 Planning for trustworthiness.                         states, their causes and how these states are
                                                               sustained; second, with describing processes of
      These elements can be set out into a sequential,         change and consequences of those states; third,
      staged approach to planning naturalistic research        with how suitable they are for supporting or not
      (see, for example, Schatzman and Strauss 1973;           supporting hypotheses and assumptions or for
                                                                      PLANNING NATURALISTIC RESEARCH          173

generating new hypotheses and assumptions (the          extent to which the researcher’s prior knowledge

                                                                                                                Chapter 7
‘generative questions’ referred to above).              may be influencing the research (i.e. reflexiv-
                                                        ity); a recognition of the tentative nature of
                                                        one’s hypothesis; a willingness to use the re-
Should one have a hypothesis in qualitative
                                                        search to generate a hypothesis; and, as a more
                                                        extreme position, an acknowledgment that hav-
We mentioned in Chapter 1 that positivist ap-           ing a hypothesis may be just as much a part
proaches typically test pre-formulated hypotheses       of qualitative research as it is of quantitative
and that a distinguishing feature of naturalistic and   research.
qualitative approaches is its reluctance to enter the
hypothetico-deductive paradigm (e.g. Meinefeld          Features and stages of a qualitative study
2004: 153), not least because there is a recogni-
tion that the researcher influences the research and     An effective qualitative study has several
because the research is much more open and emer-        features (Cresswell 1998: 20–2), and these can
gent in qualitative approaches. Indeed Meinefeld        be addressed in evaluating qualitative research:
(2004), citing classic studies like Whyte’s (1955)         The study uses rigorous procedures and
Street Corner Society, suggests that it is impos-          multiple methods for data collection.
sible to predetermine hypotheses, whether one              The study is framed within the assumptions
would wish to or not, as prior knowledge cannot            and nature of qualitative research.
be presumed. Glaser and Strauss (1967) suggest             Enquiry is a major feature, and can follow one
that researchers should deliberately free them-            or more different traditions (e.g. biography,
selves from all prior knowledge, even suggesting           ethnography, phenomenology, case study,
that it is impossible to read up in advance, as            grounded theory).
it is not clear what reading will turn out to be           The project commences with a single focus on
relevant – the data speak for themselves. Theory           an issue or problem rather than a hypothesis
is the end point of the research, not its starting         or the supposition of a causal relationship of
point.                                                     variables. Relationships may emerge later, but
   One has to be mindful that the researcher’s own         that is open.
background interest, knowledge, and biography              Criteria for verification are set out, and rigour
precede the research and that though initial               is practised in writing up the report.
hypotheses may not be foregrounded in qualitative          Verisimilitude is required, such that readers
research, nevertheless the initial establishment           can imagine being in the situation.
of the research presupposes a particular area of           Data are analysed at different levels; they are
interest, i.e. the research and data for focus are         multilayered.
not theory-free; knowledge is not theory-free.             The writing engages the reader and is replete
Indeed Glaser and Strauss (1967) acknowledge               with unexpected insights, while maintaining
that they brought their own prior knowledge to             believability and accuracy.
their research on dying.
   The resolution of this apparent contradic-
                                                        Stage 1: Locating a field of study
tion – the call to reject an initial hypothesis in
qualitative research, yet a recognition that all        Bogdan and Biklen (1992: 2) suggest that research
research commences with some prior knowledge            questions in qualitative research are not framed
or theory that gives rise to the research, how-         by simply operationalizing variables as in the
ever embryonic – may lie in several fields. These        positivist paradigm. Rather, research questions are
include: an openness to data (Meinefeld 2004:           formulated in situ and in response to situations
156–7); a preparedness to modify one’s initial          observed, i.e. that topics are investigated in all
presuppositions and position; a declaration of the      their complexity, in the naturalistic context. The

      field, as Arsenault and Anderson (1998: 125) state,        He argues that in the real social world, access
      ‘is used generically in qualitative research and       to important areas of research is prohibited if
      quite simply refers to where the phenomenon            informed consent has to be sought, for example
      exists’.                                               in researching those on the margins of society or
         In some qualitative studies, the selection of the   the disadvantaged. It is to the participants’ own
      research field will be informed by the research         advantage that secrecy is maintained as, if secrecy
      purposes, the need for the research, what gave         is not upheld, important work may not be done
      rise to the research, the problem to be addressed,     and ‘weightier secrets’ (Mitchell 1993: 54) may be
      and the research questions and sub-questions. In       kept which are of legitimate public concern and
      other qualitative studies these elements may only      in the participants’ own interests. Mitchell makes
      emerge after the researcher has been immersed for      a powerful case for secrecy, arguing that informed
      some time in the research site itself.                 consent may excuse social scientists from the risk
                                                             of confronting powerful, privileged and cohesive
                                                             groups who wish to protect themselves from
      Stage 2: Addressing ethical issues
                                                             public scrutiny. Secrecy and informed consent are
      Deyle et al. (1992: 623) identify several critical     moot points. Researchers, then, have to consider
      ethical issues that need to be addressed in            their loyalties and responsibilities (LeCompte and
      approaching the research:                              Preissle 1993: 106), for example what is the public’s
         How does one present oneself in the field? As        right to know and what is the individual’s right to
      whom does one present oneself? How ethically           privacy (Morrison 1993; De Laine 2000: 13).
      defensible is it to pretend to be somebody that you       In addition to the issue of overt or covert
      are not in order to gain knowledge that you would      research, LeCompte and Preissle (1993) indicate
      otherwise not be able to acquire, and to obtain        that the problems of risk and vulnerability to
      and preserve access to places which otherwise you      subjects must be addressed; steps must be
      would be unable to secure or sustain.                  taken to prevent risk or harm to participants
         The issues here are several. First, there is        (non-maleficence – the principle of primum non
      the matter of informed consent (to participate         nocere). Bogdan and Biklen (1992: 54) extend
      and for disclosure), whether and how to gain           this to include issues of embarrassment as well
      participant assent (see also LeCompte and Preissle     as harm to those taking part. The question of
      1993: 66). This uncovers another consideration,        vulnerability is present at its strongest when
      namely covert or overt research. On the one            participants in the research have their freedom
      hand, there is a powerful argument for informed        to choose limited, e.g. by dint of their age, by
      consent. However, the more participants know           health, by social constraints, by dint of their
      about the research the less naturally they may         life style (e.g. engaging in criminality), social
      behave (LeCompte and Preissle 1993: 108), and          acceptability, experience of being victims (e.g.
      naturalism is self-evidently a key criterion of the    of abuse, of violent crime) (Bogdan and Biklen
      naturalistic paradigm.                                 1992:107). As the authors comment, participants
         Mitchell (1993) catches the dilemma for             rarely initiate research, so it is the responsibility
      researchers in deciding whether to undertake           of the researcher to protect them. Relationships
      overt or covert research. The issue of informed        between researcher and the researched are rarely
      consent, he argues, can lead to the selection          symmetrical in terms of power; it is often the
      of particular forms of research – those where          case that those with more power, information and
      researchers can control the phenomena under            resources research those with less.
      investigation – thereby excluding other kinds of          A standard protection is often the guarantee
      research where subjects behave in less controllable,   of confidentiality, withholding participants’ real
      predictable, prescribed ways, indeed where subjects    names and other identifying characteristics.
      may come in and out of the research over time.         Bogdan and Biklen (1992: 106) contrast this
                                                                     PLANNING NATURALISTIC RESEARCH             175

with anonymity, where identity is withheld              issues, time frames, artefacts and data sources. This

                                                                                                                  Chapter 7
because it is genuinely unknown. The issues are         takes the discussion beyond conventional notions
raised of identifiability and traceability. Further,     of sampling.
participants might be able to identify themselves          In several forms of research sampling is fixed
in the research report though others may not be         at the start of the study, though there may be
able to identify them. A related factor here is the     attrition of the sample through ‘mortality’ (e.g.
ownership of the data and the results, the control of   people leaving the study). Mortality is seen as
the release of data (and to whom, and when) and         problematic. Ethnographic research regards this as
what rights respondents have to veto the research       natural rather than irksome. People come into and
results. Patrick (1973) indicates this point at its     go from the study. This impacts on the decision
sharpest: as an ethnographer of a Glasgow gang,         whether to have a synchronic investigation
he was witness to a murder. The dilemma was             occurring at a single point in time, or a diachronic
clear – to report the matter (and thereby, also to      study where events and behaviour are monitored
blow his cover, consequently endangering his own        over time to allow for change, development
life) or to stay as a covert researcher.                and evolving situations. In ethnographic inquiry
   Bogdan and Biklen (1992: 54) add to this             sampling is recursive and ad hoc rather than fixed
discussion the need to respect participants as          at the outset; it changes and develops over time.
subjects, not simply as research objects to be used     Let us consider how this might happen.
and then discarded. Mason (2002: 41) suggests              LeCompte and Preissle (1993: 82–3) point out
that it is important for researchers to consider the    that ethnographic methods rule out statistical
parties, bodies, practices that might be interested     sampling, for a variety of reasons:
in, or affected by, the research and the implications
                                                           The characteristics of the wider population are
of the answer to these questions for the conduct,
reporting and dissemination of the inquiry. We
                                                           There are no straightforward boundary markers
address ethics in Chapters 2 and 5 and we advise
                                                           (categories or strata) in the group.
readers to refer to these chapters.
                                                           Generalizability, a goal of statistical methods,
                                                           is not necessarily a goal of ethnography.
Stage 3: Deciding the sampling                             Characteristics of a sample may not be evenly
                                                           distributed across the sample.
In an ideal world the researcher would be able to
                                                           Only one or two subsets of a characteristic of a
study a group in its entirety. This was the case
                                                           total sample may be important.
in Goffman’s (1968) work on ‘total institutions’,
                                                           Researchers may not have access to the whole
such as hospitals, prisons and police forces. It
was also the practice of anthropologists who were
                                                           Some members of a subset may not be drawn
able to explore specific isolated communities or
                                                           from the population from which the sampling
tribes. That is rarely possible nowadays because
                                                           is intended to be drawn.
such groups are no longer isolated or insular.
Hence the researcher is faced with the issue of         Hence other types of sampling are required. A
sampling, that is, deciding which people it will        criterion-based selection requires the researcher
be possible to select to represent the wider group      to specify in advance a set of attributes, factors,
(however defined). The researcher has to decide          characteristics or criteria that the study must
the groups for which the research questions are         address. The task then is to ensure that these
appropriate, the contexts which are important           appear in the sample selected (the equivalent
for the research, the time periods that will be         of a stratified sample). There are other forms of
needed, and the possible artefacts of interest to       sampling (discussed in Chapter 4) that are useful in
the investigator. In other words decisions are          ethnographic research (Bogdan and Biklen 1992:
necessary on the sampling of people, contexts,          70; LeCompte and Preissle 1993: 69–83), such as:

         convenience sampling: opportunistic sampling,            a particular phenomenon or phenomena (Ezzy
         selecting from whoever happens to be                     2002: 74)
         available                                                sampling according to intensity: depending on
         critical-case sampling: e.g. people who display          which features of interest are displayed or
         the issue or set of characteristics in their             occur
         entirety or in a way that is highly significant           sampling critical cases: in order to permit
         for their behaviour                                      maximum applicability to others; if the
         identifying the norm of a characteristic: then the       information holds true for critical cases (e.g.
         extremes of that characteristic are located, and         cases where all of the factors sought are
         the bearers of that extreme characteristic are           present), then it is likely to hold true for others
         selected                                                 sampling politically important or sensitive cases: to
         typical case-sampling: a profile of attributes            draw attention to the case
         or characteristics that are possessed by an              convenience sampling: saves time and money
         ‘average’, typical person or case is identified,          and spares the researcher the effort of finding
         and the sample is selected from these                    less amenable participants.
         conventional people or cases
         unique-case sampling: cases that are rare, unique     One can add to this list types of sample from Miles
         or unusual on one or more criteria are                and Huberman (1994: 28):
         identified, and sampling takes places within              homogeneous sampling: focuses on groups with
         these; here whatever other characteristics or            similar characteristics
         attributes a person might share with others, a           theoretical sampling: in grounded theory,
         particular attribute or characteristic sets that         discussed below, where participants are
         person apart                                             selected for their ability to contribute to the
         reputational-case sampling: a variant of                 developing/emergent theory
         extreme-case and unique-case sampling, where             confirming and disconfirming cases: akin to the
         a researcher chooses a sample on the recom-              extreme and deviant cases indicated by Patton
         mendation of experts in the field                         (1980), in order to look for exceptions to the
         snowball sampling: using the first interviewee to         rule, which may lead to the modification of the
         suggest or recommend other interviewees.                 rule
         Patton (1980) identifies several types of                 random purposeful sampling: when the potential
      sampling that are useful in naturalistic re-                sample is too large, a smaller subsample
      search, including                                           can be used which still maintains some
         sampling extreme/deviant cases: in order to gain         stratified purposeful sampling: to identify sub-
         information about unusual cases that may be              groups and strata
         particularly troublesome or enlightening                 criterion sampling: all those who meet some
         sampling typical cases: in order to avoid rejecting      stated criteria for membership of the group or
         information on the grounds that it has been              class under study
         gained from special or deviant cases                     opportunistic sampling: to take advantage of
         snowball sampling: one participant provides              unanticipated events, leads, ideas, issues.
         access to a further participant and so on
         maximum variation sampling: in order to               Miles and Huberman (1994) make the point that
         document the range of unique changes that             these strategies can be used in combination as well
         have emerged, often in response to the different      as in isolation, and that using them in combination
         conditions to which participants have had to          contributes to triangulation.
         adapt; useful if the aim of the research is to           We discuss below two other categories of sample:
         investigate the variations, range and patterns in     ‘primary informants’ and ‘secondary informants’
                                                                     PLANNING NATURALISTIC RESEARCH            177

(Morse 1994: 228), those who completely fulfil           sample size is not defined in advance; sampling

                                                                                                                 Chapter 7
a set of selection criteria and those who fill a         is only concluded when theoretical saturation
selection of those criteria respectively.               (discussed below) is reached.
   Lincoln and Guba (1985: 201–2) suggest                  Ezzy (2002: 74–5) gives as an example
an important difference between conventional            of theoretical sampling his own work on
and naturalistic research designs. In the former        unemployment where he developed a theory
the intention is to focus on similarities and           that levels of distress experienced by unemployed
to be able to make generalizations, whereas             people were influenced by their levels of financial
in the latter the objective is informational,           distress. He interviewed unemployed low-income
to provide such a wealth of detail that the             and high-income groups with and without debt,
uniqueness and individuality of each case can           to determine their levels of distress. He reported
be represented. To the charge that naturalistic         that levels of distress were not caused so much by
inquiry, thereby, cannot yield generalizations          absolute levels of income but levels of income in
because of sampling flaws, the writers argue that        relation to levels of debt.
this is necessarily though trivially true. In a word,      In the educational field one could imagine
it is unimportant.                                      theoretical sampling in an example thus:
   Patton (1980: 184) takes a slightly more cavalier    interviewing teachers about their morale might
approach to sampling, suggesting that ‘there are        give rise to a theory that teacher morale is
no rules for sample size in qualitative inquiry’,       negatively affected by disruptive student behaviour
with the size of the sample depending on what one       in schools. This might suggest the need to sample
wishes to know, the purposes of the research, what      teachers working with many disruptive students
will be useful and credible, and what can be done       in difficult schools, as a ‘critical case sampling’.
within the resources available, e.g. time, money,       However, the study finds that some of the teachers
people, support – important considerations for the      working in these circumstances have high morale,
novice researcher.                                      not least because they have come to expect
   Ezzy (2002: 74) underlines the notion of             disruptive behaviour from students with so many
‘theoretical sampling’ from Glaser and Strauss          problems, and so are not surprised or threatened by
(1967) in his comment that, unlike other forms          it, and because the staff in these schools provide
of research, qualitative inquiries may not always       tremendous support for each other in difficult
commence with the full knowledge of whom to             circumstances – they all know what it is like to
sample, but that the sample is determined on            have to work with challenging students.
an ongoing, emergent basis. Theoretical sampling           So the study decides to focus on teachers
starts with data and then, having reviewed these,       working in schools with far fewer disruptive
the researcher decides where to go next in              students. The researcher discovers that it is these
order to develop the emerging theory (Glaser and        teachers who experience far lower morale, and
Strauss 1967: 45). We discuss this more fully in        hypothesizes that this is because this latter group
Chapter 23.                                             of teachers has higher expectations of student
   Individuals and groups are selected for              behaviour, such that having only one or two
their potential – or hoped for – ability to offer       students who do not conform to these expectations
new insights into the emerging theory, i.e.             deflates staff morale significantly, and because
they are chosen on the basis of their                   disruptive behaviour is regarded in these schools as
significant contribution to theory generation and        teacher weakness, and there is little or no mutual
development. As the theory develops, so the             support. The researcher’s theory, then, is refined,
researcher decides whom to approach to request          to suggest that teacher morale is affected more by
their participation. Theoretical sampling does not      teacher expectations than by disruptive behaviour,
claim to know the population characteristics or         so the researcher adopts a ‘maximum variation
to represent known populations in advance, and          sampling’ of teachers in a range of schools,

      to investigate how expectations and morale are           iterative. Sampling is not decided a priori – in
      related to disruptive behaviour. In this case the        advance – but may be decided, amended, added to,
      sampling emerges as the research proceeds and            increased and extended as the research progresses.
      the theory emerges; this is theoretical sampling,
      the ‘royal way for qualitative studies’ (Flick 2004:
      151). Schatzman and Strauss (1973: 38 ff.) suggest       Stage 4: Finding a role and managing entry into the
      that sampling within theoretical sampling may            context
      change according to time, place, individuals and         This involves matters of access and permission,
      events.                                                  establishing a reason for being there, developing
         The above procedure accords with Glaser and           a role and a persona, identifying the gatekeepers
      Strauss’s (1967) view that sampling involves             who facilitate entry and access to the group being
      continuously gathering data until practical factors      investigated (see LeCompte and Preissle 1993: 100
      (boundaries) put an end to data collection, or           and 111). The issue here is complex, for the
      until no amendments have to be made to the               researcher will be both a member of the group
      theory in light of further data – their stage of         and yet studying that group, so it is a delicate
      ‘theoretical saturation’ – where the theory fits the      matter to negotiate a role that will enable the
      data even when new data are gathered. Theoretical        investigator to be both participant and observer.
      saturation is described by Glaser and Strauss            LeCompte and Preissle (1993: 112) comment that
      (1967: 61) as being reached when ‘no additional          the most important elements in securing access
      data are being found whereby the sociologist can         are the willingness of researchers to be flexible
      develop properties of the category’. That said, the      and their sensitivity to nuances of behaviour and
      researcher has to be cautious to avoid premature         response in the participants. As De Laine (2000:
      cessation of data collection; it would be too easy to    41) remarks: ‘demonstrated ability to get on with
      close off research with limited data, when, in fact,     people in the setting and a willingness to share
      further sampling and data collection might lead to       experience in ongoing activities are important
      a reformulation of the theory.                           criteria of access’.
         An extension of theoretical sampling is ‘analytic        Wolff (2004: 195–6) suggests that there are
      induction’, a process advanced by Znaniecki              two fundamental questions to be addressed in
      (1934). Here the researcher starts with a theory         considering access and entry into the field:
      (that may have emerged from the data, as in
      grounded theory) and then deliberately proceeds             How can researchers succeed in making
      to look for deviant or discrepant cases, to provide         contact and securing cooperation from
      a robust defence of the theory. This accords with           informants?
      Popper’s notion of a rigorous scientific theory              How can researchers position themselves in
      having to stand up to falsifiability tests. In analytic      the field so as to secure the necessary time,
      induction, the researcher deliberately seeks data           space and social relations to be able to carry
      which potentially could falsify the theory, thereby         out the research?
      giving strength to the final theory.
                                                               Flick (1998: 57) summarizes Wolff’s (2004) work
         We are suggesting here that, in qualitative
                                                               in identifying several issues in entering institutions
      research, sampling cannot always be decided in
                                                               for the purpose of conducting research:
      advance on a ‘once and for all’ basis. It may have
      to continue through the stages of data collection,          Research is always an intrusion and interven-
      analysis and reporting. This reflects the circular           tion into a social system, and, so, disrupts the
      process of qualitative research, in which data              system to be studied, such that the system
      collection, analysis, interpretation and reporting          reacts, often defensively.
      and sampling do not necessarily have to proceed             There is a ‘mutual opacity’ between the social
      in a linear fashion; the process is recursive and           system under study and the research project,
                                                                       PLANNING NATURALISTIC RESEARCH            179

   which is not reduced by information exchange

                                                                                                                   Chapter 7
                                                        a teacher, a researcher, an inspector, a friend, a
   between the system under study and the               manager, a provider of a particular service (e.g.
   researcher; rather this increases the complexity     extracurricular activities), a counsellor, a social
   of the situation and, hence, ‘immune reactions’.     worker, a resource provider, a librarian, a cleaner,
   Rather than striving for mutual understanding        a server in the school shop or canteen, and so on?
   at the point of entry, it is more advisable to       The issue is that one has to try to select a role that
   strive for an agreement as a process.                will provide access to as wide a range of people
   While it is necessary to agree storage rights        as possible, preserve neutrality (not being seen as
   for data, this may contribute to increasing the      on anybody’s side), and enable confidences to be
   complexity of the agreement to be reached.           secured.
   The field under study becomes clear only when            Role conflict, strain and ambiguity are to be
   one has entered it.                                  expected in qualitative research. For example, De
   The research project usually has nothing to          Laine (2002: 29) comments on the potential
   offer the social system; hence no great promises     conflicts between the researcher qua researcher,
   for benefit or services can be made by the            therapist, friend. She indicates that diverse role
   researcher, yet there may be no real reason why      positions are rarely possible to plan in advance,
   the social system should reject the researcher.      and are an inevitable part of fieldwork, giving rise
As Flick (1998: 57) remarks, the research will          to ethical and moral problems for the researcher,
disturb the system and disrupt routines without         and, in turn, requiring ongoing negotiation and
being able to offer any real benefit for the             resolution.
institution.                                               Roles change over time. Walford (2001: 62)
   The issue of managing relations is critical          reports a staged process wherein the researcher’s
for the qualitative researcher. We discuss              role moved through five phases: newcomer,
issues of access, gatekeepers and informants in         provisional acceptance, categorical acceptance,
Chapter 4. The researcher is seen as coming             personal acceptance and imminent migrant.
‘without history’ (Wolff 2004: 198), a ‘professional    Walford (2001: 71) also reports that it is almost
stranger’ (Flick 1998: 59), one who has to              to be expected that managing different roles not
be accepted, become familiar and yet remain             only throws the researcher into questioning his/her
distant from those being studied. Indeed Flick          ability to handle the situation, but also brings
(1998: 60) suggests four roles of the researcher:       considerable emotional and psychological stress,
stranger, visitor, insider and initiate. The first two   anxiety and feelings of inadequacy. This is thrown
essentially maintain the outsider role, while the       into sharp relief when researchers have to conceal
latter two attempt to reach into the institution        information, take on different roles in order to
from an insider’s perspective. These latter two         gain access, retain neutrality, compromise personal
become difficult to manage if one is dealing with        beliefs and values, and handle situations where
sensitive issues (see Chapter 5). This typology         they are seeking information from others but not
resonates with the four roles typically cited for       divulging information about themselves. Walford
observers, as shown in the diagram below.               (2001) suggests that researchers may have little
   Role negotiation, balance and trust are              opportunity to negotiate roles and manoeuvre
significant and difficult. For example, if one were       roles, as they are restricted by the expectations
to research a school, what role should one adopt:       of those being researched.

           Outsider                                                                        Insider
  ←                                                                                                      →
       Detached observer     Observer as participant    Participant as observer   Complete participant

          A related issue is the timing of the point         or group being studied. This places researchers in
      of entry, so that researchers can commence the         a difficult position, for they have to be able to
      research at appropriate junctures (e.g. before the     evaluate key informants, to decide:
      start of a programme, at the start of a programme,
                                                                whose accounts are more important than others
      during a programme, at the end of a programme,
                                                                which informants are competent to pass
      after the end of a programme). The issue goes
      further than this, for the ethnographer will need
                                                                which are reliable
      to ensure acceptance into the group, which will
                                                                what the statuses of the informants are
      be a matter of dress, demeanour, persona, age,
                                                                how representative are the key informants (of
      colour, ethnicity, empathy and identification with
                                                                the range of people, of issues, of situations, of
      the group, language, accent, argot and jargon,
                                                                views, of status, of roles, of the group)
      willingness to become involved and to take on the
                                                                how to see the informants in different settings
      group’s values and behaviour etc. (see Patrick’s
                                                                how knowledgeable informants actually are –
      (1973) fascinating study of a Glasgow gang). The
                                                                do they have intimate and expert understand-
      researcher, then, has to be aware of the significance
                                                                ing of the situation
      of ‘impression management’ (Hammersley and
                                                                how central to the organization or situation
      Atkinson 1983: 78 ff.). In covert research these
                                                                the informant is (e.g. marginal or central)
      factors take on added significance, as one slip
                                                                how to meet and select informants
      could blow one’s cover (Patrick 1973).
                                                                how critical the informants are as gatekeepers
          Lofland (1971) suggests that the field researcher
                                                                to other informants, opening up or restricting
      should attempt to adopt the role of the ‘acceptable
                                                                entry to people (Hammersley and Atkinson
      incompetent’, balancing intrusion with knowing
                                                                1983: 73)
      when to remain apart. Such balancing is
                                                                the relationship between the informant and
      an ongoing process. Hammersley and Atkinson
                                                                others in the group or situation being studied.
      (1983: 97–9) suggest that researchers have to
      handle the management of ‘marginality’: they are       Selecting informants and engaging with them
      in the organization but not of it. They comment        is problematical; LeCompte and Preissle (1993:
      that ‘the ethnographer must be intellectually          95), for example, suggest that the first informants
      poised between ‘‘familiarity’’ and ‘‘strangeness’’,    that an ethnographer meets might be self-selected
      while socially he or she is poised between             people who are marginal to the group, have a low
      ‘‘stranger’’ and ‘‘friend’’.’ They also comment        status, and who, therefore, might be seeking to
      that this management of several roles, not least       enhance their own prestige by being involved with
      the management of marginality, can engender ‘a         the research. Indeed, Lincoln and Guba (1985:
      continual sense of insecurity’ (Hammersley and         252) argue that the researcher must be careful to
      Atkinson 1983: 100).                                   use informants rather than informers, the latter
          Gaining access and entry, as we argue in           possibly having ‘an axe to grind’. Researchers who
      Chapter 5, should be regarded as a process             are working with gatekeepers, they argue, will be
      (Walford 2001: 31) that unfolds over time, rather      engaged in a constant process of bargaining and
      than a once and for all matter. Walford charts         negotiation.
      the several setbacks, delays and modifications that        A ‘good’ informant, Morse (1994: 228) declares,
      occur and have to be expected in gaining entry to      is one who has the necessary knowledge,
      qualitative research sites.                            information and experience of the issue being
                                                             researched, is capable of reflecting on that
                                                             knowledge and experience, has time to be involved
      Stage 5: Finding informants
                                                             in the project, is willing to be involved in the
      Finding informants involves identifying those          project, and, indeed, can provide access to other
      people who have the knowledge about the society        informants. An informant who fulfils all of these
                                                                     PLANNING NATURALISTIC RESEARCH             181

criteria is termed a ‘primary informant’. Morse           how to balance responsibilities to the

                                                                                                                  Chapter 7
(1994) also cautions that not all these features          community with responsibilities to other
may be present in the informants, but that they           interested parties.
may still be useful for the research, though the
researcher would have to decide how much time          Critically important in this area is the mainte-
to spend with these ‘secondary’ informants.            nance of trust and rapport (De Laine 2000: 41),
                                                       showing interest, assuring confidentiality (where
                                                       appropriate) and avoiding being judgemental. She
Stage 6: Developing and maintaining relations in the   adds to these the ability to tolerate ambiguity, to
field                                                   keep self-doubt in check, to withstand insecurity,
This involves addressing interpersonal and             and to be flexible and accommodating (De Laine
practical issues, for example:                         2000: 97). Such features are not able to be en-
                                                       capsulated in formal agreements, but they are the
   building participants’ confidence in the
                                                       lifeblood of effective qualitative enquiry. They are
                                                       process matters.
   developing rapport, trust, sensitivity and
                                                          The issue here is that the data collection process
                                                       is itself socially situated; it is neither a clean,
   handling people and issues with which the
                                                       antiseptic activity nor always a straightforward
   researcher disagrees or finds objectionable or
   being attentive and empathizing
   being discreet                                      Stage 7: Data collection in situ
   deciding how long to stay.
                                                       The qualitative researcher is able to use a variety
Spindler and Spindler (1992: 65) suggest that
                                                       of techniques for gathering information. There is
ethnographic validity is attained by having the
                                                       no single prescription for which data collection
researcher in situ long enough to see things
                                                       instruments to use; rather the issue here is of
happening repeatedly rather than just once, that
                                                       ‘fitness for purpose’ because, as was mentioned
is to say, observing regularities.
                                                       earlier, the ethnographer is a methodological
   LeCompte and Preissle (1993: 89) suggest that
                                                       omnivore! That said, there are several types of data
fieldwork, particularly because it is conducted face-
                                                       collection instruments that are used more widely
to-face, raises problems and questions that are
                                                       in qualitative research than others. The researcher
less significant in research that is conducted at a
                                                       can use field notes, participant observation, journal
distance, including:
                                                       notes, interviews, diaries, life histories, artefacts,
   how to communicate meaningfully with                documents, video recordings, audio recordings etc.
   participants                                        Several of these are discussed elsewhere in this
   how they and the researcher might be affected       book. Lincoln and Guba (1985: 199) distinguish
   by the emotions evoked in one another, and          between ‘obtrusive’ (e.g. interviews, observation,
   how to handle these                                 non-verbal language) and ‘unobtrusive’ methods
   differences and similarities between the            (e.g. documents and records), on the basis of
   researcher and the participants (e.g. personal      whether another human typically is present at
   characteristics, power, resources), and how         the point of data collection.
   these might affect relationships between parties       Field notes can be written both in situ
   and the course of the investigation                 and away from the situation. They contain
   the researcher’s responsibilities to the partici-   the results of observations. The nature of
   pants (qua researcher and member of their           observation in ethnographic research is discussed
   community), even if the period of residence in      fully in Chapter 17. Accompanying observation
   the community is short                              techniques is the use of interviews, documentary

      analysis and life histories. These are discussed       Preissle 1993: 177). At the implementation stage
      separately in Chapters 7, 15 and 16. The popularly     the conduct of the interview will be important, for
      used interview technique employed in qualitative       example, responding to interviewees, prompting,
      research is the semi-structured interview, where       probing, supporting, empathizing, clarifying,
      a schedule is prepared that is sufficiently open-       crystallizing, exemplifying, summarizing, avoiding
      ended to enable the contents to be reordered,          censure, accepting. At the analysis stage there
      digressions and expansions made, new avenues           will be several important considerations, for
      to be included, and further probing to be              example (LeCompte and Preissle 1993: 195): the
      undertaken. Carspecken (1996: 159–60) describes        ease and clarity of communication of meaning;
      how such interviews can range from the                 the interest levels of the participants; the clarity of
      interrogator giving bland encouragements, ‘non-        the question and the response; the precision (and
      leading’ leads, active listening and low-inference     communication of this) of the interviewer; how
      paraphrasing to medium- and high-inference             the interviewer handles questionable responses
      paraphrasing. In interviews the researcher might       (e.g. fabrications, untruths, claims made).
      wish to further explore some matters arising from         The qualitative interview tends to move away
      observations. In naturalistic research the canons      from a prestructured, standardized form towards
      of validity in interviews include: honesty, depth of   an open-ended or semi-structured arrangement
      response, richness of response, and commitment         (see Chapter 16), which enables respondents to
      of the interviewee (Oppenheim 1992).                   project their own ways of defining the world. It
         Lincoln and Guba (1985: 268–70) propose             permits flexibility rather than fixity of sequence
      several purposes for interviewing, including:          of discussions, allowing participants to raise and
      present constructions of events, feelings, persons,    pursue issues and matters that might not have been
      organizations, activities, motivations, concerns,      included in a pre-devised schedule (Denzin 1970b;
      claims, etc.; reconstructions of past experiences;     Silverman 1993).
      projections into the future; verifying, amending and      In addition to interviews, Lincoln and Guba
      extending data.                                        (1985) discuss data collection from non-human
         Further, Silverman (1993: 92–3) adds that           sources, including:
      interviews in qualitative research are useful for:
                                                                Documents and records (e.g. archival records,
      gathering facts; accessing beliefs about facts;
                                                                private records): these have the attraction of
      identifying feelings and motives; commenting on
                                                                being always available, often at low cost, and
      the standards of actions (what could be done
                                                                being factual. On the other hand, they may
      about situations); exploring present or previous
                                                                be unrepresentative, they may be selective,
      behaviour; eliciting reasons and explanations.
                                                                lack objectivity, be of unknown validity,
         Lincoln and Guba (1985) emphasize that the
                                                                and may possibly be deliberately deceptive
      planning of the conduct of the interview is
                                                                (see Finnegan 1996).
      important, including the background preparation,
                                                                Unobtrusive informational residues: these include
      the opening of the interview, its pacing and timing,
                                                                artefacts, physical traces, and a variety of
      keeping the conversation going and eliciting
                                                                other records. While they frequently have face
      knowledge, and rounding off and ending the
                                                                validity, and while they may be simple and
      interview. Clearly, it is important that careful
                                                                direct, gained by non-interventional means
      consideration be given to the several stages of
                                                                (hence reducing the problems of reactivity),
      the interview. For example, at the planning
                                                                they may also be very heavily inferential,
      stage, attention will need to be given to the
                                                                difficult to interpret, and may contain elements
      number (per person), duration, timing, frequency,
                                                                whose relevance is questionable.
      setting/location, number of people in a single
      interview situation (e.g. individual or group             Qualitative data collection is not hidebound to
      interviews) and respondent styles (LeCompte and        a few named strategies; it is marked by eclecticism
                                                                    PLANNING NATURALISTIC RESEARCH             183

and fitness for purpose. It is not to say that         in a school. At one level it is simply a fight

                                                                                                                 Chapter 7
‘anything goes’ but ‘use what is appropriate’ is      between two people. However, this is a common
sound advice. Mason (2002: 33–4) advocates the        occurrence between these two students as they
integration of methods, for several reasons:          are neighbours outside school and they don’t
                                                      enjoy positive amicable relations as their families
   to explore different elements or parts of a
                                                      are frequently feuding. The two households have
   phenomenon, ensuring that the researcher
                                                      been placed next door to each other by the local
   knows how they interrelate
                                                      authority because if has taken a decision to keep
   to answer different research questions
                                                      together families who are very poor at paying
   to answer the same research question but in
                                                      for local housing rent (i.e. a ‘sink’ estate). The
   different ways and from different perspectives
                                                      local authority has taken this decision because of a
   to give greater or lesser depth and breadth to
                                                      government policy to keep together disadvantaged
                                                      groups so that targeted action and interventions
   to triangulate (corroborate) by seeking
                                                      can be more effective, thus meeting the needs of
   different data about the same phenomenon.
                                                      whole communities as well as individuals.
Mason (2002: 35) argues that integration can             The issue here is: how far out of a micro-situation
take many forms. She suggests that it is necessary    does the researcher need to go to understand
for researchers to consider whether the data are      that micro-situation? This is an imprecise matter
to complement each other, to be combined,             but it is not insignificant in educational research:
grouped and aggregated, and to contribute to          for example, it underpinned: (a) the celebrated
an overall picture. She also argues that it is        work by Bowles and Gintis (1976) on schooling in
important for the data to complement each other       capitalist America, in which the authors suggested
ontologically, to be ontologically consistent, i.e.   that the hidden curricula of schools were preparing
whether they are ‘based on similar, complementary     students for differential occupational futures that
or comparable assumptions about the nature of         perpetuated an inegalitarian capitalist system,
social entities and phenomena’. Added to this         (b) research on the self-fulfilling prophecy (Hurn
Mason (2002: 36) suggests that integration must       1978), (c) work by Pollard (1985: 110) on the
be in an epistemological sense, i.e. where the data   social world of the primary school, where everyday
emanate from the same, or at least complementary,     interactions in school were preparing students
epistemologies, whether they are based on ‘similar,   for the individualism, competition, achievement
complementary or comparable assumptions about         orientation, hierarchies and self-reliance that
what can legitimately constitute knowledge of         characterize mass private consumption in wider
evidence’. Finally Mason (2002: 36) argues            society, (d) Delamont’s (1981) advocacy that
that integration must occur at the level of           educationists should study similar but different
explanation. By this she means that the data from     institutions to schools (e.g. hospitals and other
different courses and methods must be able to         ‘total’ institutions) in order to make the familiar
be combined into a coherent, convincing and           strange (see also Erickson 1973).
relevant explanation and argument.
                                                      Stage 9: Data analysis
Stage 8: Data collection outside the field
                                                      Although we devote two chapters specifically
In order to make comparisons and to suggest           to qualitative data analysis later in this book
explanations for phenomena, researchers might         (Chapters 22 and 23), there are some preliminary
find it useful to go beyond the confines of the         remarks that we make here, by way of
groups in which they occur. That this is a thorny     fidelity to the eleven-stage process of qualitative
issue is indicated in the following example. Two      research outlined earlier in the chapter. Data
students are arguing very violently and physically    analysis involves organizing, accounting for, and

      explaining the data; in short, making sense of                  Step 3: Establish relationships and linkages be-
      data in terms of participants’ definitions of the                tween the domains
      situation, noting patterns, themes, categories and              Step 4: Making speculative inferences
      regularities. Typically in qualitative research, data           Step 5: Summarizing
      analysis commences during the data collection                   Step 6: Seeking negative and discrepant cases
      process. There are several reasons for this, and                Step 7: Theory generation
      these are discussed below.
         At a practical level, qualitative research rapidly           Step 1: Establish units of analysis of the data,
      amasses huge amounts of data, and early analysis             indicating how these units are similar to and different
      reduces the problem of data overload by selecting            from each other. The criterion here is that each
      out significant features for future focus. Miles              unit of analysis (category – conceptual, actual,
      and Huberman (1984) suggest that careful data                classification element, cluster, issue) should be
      display is an important element of data reduction            as discrete as possible while retaining fidelity to
      and selection. ‘Progressive focusing’, according             the integrity of the whole, i.e. that each unit must
      to Parlett and Hamilton (1976), starts with the              be a fair rather than a distorted representation of
      researcher taking a wide-angle lens to gather                the context and other data. The creation of units
      data, and then, by sifting, sorting, reviewing and           of analysis can be done by ascribing codes to the
      reflecting on them, the salient features of the               data (Miles and Huberman 1984). This is akin
      situation emerge. These are then used as the                 to the process of ‘unitizing’ (Lincoln and Guba
      agenda for subsequent focusing. The process is               1985: 203).
      like funnelling from the wide to the narrow.                    Step 2: Create a ‘domain analysis’. A domain
         At a theoretical level a major feature of                 analysis involves grouping together items and
      qualitative research is that analysis commences              units into related clusters, themes and patterns,
      early on in the data collection process so that              a domain being a category which contains several
      theory generation can be undertaken (LeCompte                other categories. We address domain analysis in
      and Preissle 1993: 238). LeCompte and Preissle               more detail in Chapter 23.
      (1993: 237–53) advise that researchers should set               Step 3: Establish relationships and linkages
      out the main outlines of the phenomena that are              between the domains. This process ensures
      under investigation. They then should assemble               that the data, their richness and ‘context-
      chunks or groups of data, putting them together              groundedness’ are retained. Linkages can be
      to make a coherent whole (e.g. through writing               found by identifying confirming cases, by seeking
      summaries of what has been found). Then they                 ‘underlying associations’ (LeCompte and Preissle
      should painstakingly take apart their field notes,            1993: 246) and connections between data subsets.
      matching, contrasting, aggregating, comparing                   Step 4: Making speculative inferences. This is
      and ordering notes made. The intention is to                 an important stage, for it moves the research from
      move from description to explanation and theory              description to inference. It requires the researcher,
      generation.                                                  on the basis of the evidence, to posit some
         For clarity, the process of data analysis can be          explanations for the situation, some key elements
      portrayed in a sequence of seven steps which are             and possibly even their causes. It is the process
      set out here and addressed in subsequent pages.              of hypothesis generation or the setting of working
                                                                   hypotheses that feeds into theory generation.
         Step 1: Establish units of analysis of the data,             Step 5: Summarizing. This involves the
         indicating how these units are similar to and different   researcher in writing a preliminary summary
         from each other                                           of the main features, key issues, key concepts,
         Step 2: Create a ‘domain analysis’                        constructs and ideas encountered so far in the
                                                                     PLANNING NATURALISTIC RESEARCH             185

research. We address summarizing in more detail        decided, data are collected and then analysed and

                                                                                                                  Chapter 7
in Chapter 23.                                         hypotheses are supported or not supported. In
   Step 6: Seeking negative and discrepant cases. In   grounded theory a circular and recursive process
theory generation it is important to seek not only     is adopted, wherein modifications are made to the
confirming cases but to weigh the significance of        theory in light of data, more data are sought to
disconfirming cases. LeCompte and Preissle (1993:       investigate emergent issues (theoretical sampling),
270) suggest that because interpretations of the       and hypotheses and theories emerge from the data.
data are grounded in the data themselves, results         Lincoln and Guba (1985: 354–5) urge the
that fail to support an original hypothesis are        researcher to be mindful of several issues in
neither discarded nor discredited; rather, it is the   analysing and interpreting the data, including:
hypotheses themselves that must be modified to
                                                          data overload
accommodate these data. Indeed Erickson (1992:
                                                          the problem of acting on first impressions only
208) identifies progressive problem-solving as one
                                                          the availability of people and information
key aspect of ethnographic research and data
                                                          (e.g. how representative these are and how
analysis. LeCompte and Preissle (1993: 250–1)
                                                          to know if missing people and data might be
define a negative case as an exemplar which
disconfirms or refutes the working hypothesis,
                                                          the dangers of only seeking confirming rather
rule or explanation so far. It is the qualitative
                                                          than disconfirming instances
researcher’s equivalent of the positivist’s null
                                                          the reliability and consistency of the data and
hypothesis. The theory that is being developed
                                                          confidence that can be placed in the results.
becomes more robust if it addresses negative cases,
for it sets the boundaries to the theory; it modifies   These are significant issues in addressing reliability,
the theory, it sets parameters to the applicability    trustworthiness and validity in the research (see
of the theory.                                         the discussions of reliability and validity in
   Discrepant cases are not so much exceptions         Chapter 5). The essence of this approach, that
to the rule (as in negative cases) as variants         theory emerges from and is grounded in data, is
of the rule (LeCompte and Preissle 1993: 251).         not without its critics. For example, Silverman
The discrepant case leads to the modification or        (1993: 47) suggests that it fails to acknowledge
elaboration of the construct, rule or emerging         the implicit theories which guide research in its
hypothesis. Discrepant case analysis requires the      early stages (i.e. data are not theory neutral but
researcher to seek out cases for which the rule,       theory saturated) and that it might be strong
construct, or explanation cannot account or with       on providing categorizations without necessarily
which they will not fit, i.e. they are neither          explanatory potential. These are caveats that
exceptions nor contradictions, they are simply         should feed into the process of reflexivity in
different!                                             qualitative research, perhaps.
   Step 7: Theory generation. Here the theory
derives from the data – it is grounded in the
                                                       Stage 10: Leaving the field
data and emerges from it. As Lincoln and Guba
(1985: 205) argue, grounded theory must fit             The issue here is how to conclude the research,
the situation that is being researched. Grounded       how to terminate the roles adopted, how (and
theory is an iterative process, moving backwards       whether) to bring to an end the relationships that
and forwards between data and theory until the         have built up over the course of the research, and
theory fits the data. This breaks the linearity of      how to disengage from the field in ways that bring
much conventional research (Flick 1998: 41, 43)        as little disruption to the group or situation as
in which hypotheses are formulated, sampling is        possible (LeCompte and Preissle 1993: 101). De

      Laine (2000: 142) remarks that some participants            have a means of checking back for reliability
      may want to maintain contact after the research             and validity and inferences.
      is over, and not to do this might create, for them,         A fixed completion date should be specified.
      a sense of disappointment, exploitation or even
                                                              Spradley (1979) suggests nine practical steps that
      betrayal. One has to consider the after-effects of
                                                              can be followed in writing an ethnography:
      leaving and take care to ensure that nobody comes
      to harm or is worse off from the research, even if      1   Select the audience.
      it is impossible to ensure that they have benefited      2   Select the thesis.
      from it.                                                3   Make a list of topics and create an outline of
                                                                  the ethnography.
                                                              4   Write a rough draft of each section of the
      Stage 11: Writing the report
      In research literature there is a move away from        5   Revise the outline and create subheadings.
      the conduct of the research and towards the             6   Edit the draft.
      reporting of the research. It is often the case         7   Write an introduction and a conclusion.
      that the main vehicle for writing naturalistic          8   Reread the data and report to identify
      research is the case study (see Chapter 11), whose          examples.
      ‘trustworthiness’ (Lincoln and Guba 1985: 189)          9   Write the final version.
      is defined in terms of credibility, transferability,
                                                              Clearly there are several other aspects of case study
      dependability and confirmability – discussed in
                                                              reporting that need to be addressed. These are set
      Chapter 6. Case studies are useful in that they can
                                                              out in Chapter 11.
      provide the thick descriptions that are useful in
      ethnographic research, and can catch and portray
      to the reader what it is like to be involved in         Critical ethnography
      the situation (p. 214). As Lincoln and Guba
      (1985: 359) comment, the case study is the ideal        An emerging branch of ethnography that
      instrument for emic inquiry. It also builds in and      resonates with the critical paradigm outlined in
      builds on the tacit knowledge that the writer           Chapter 1 is the field of critical ethnography.
      and reader bring to the report, and, thereby, takes     Here not only is qualitative, anthropological,
      seriously their notion of the ‘human instrument’ in     participant, observer-based research undertaken,
      research, indicating the interactions of researcher     but also its theoretical basis lies in critical
      and participants.                                       theory (Quantz 1992: 448; Carspecken 1996).
         Lincoln and Guba (1985: 365–6) provide               As was outlined in Chapter 1, this paradigm
      several guidelines for writing case studies:            is concerned with the exposure of oppression
                                                              and inequality in society with a view to
         The writing should strive to be informal and to
                                                              emancipating individuals and groups towards
         capture informality.
                                                              collective empowerment. In this respect research
         As far as possible the writing should report facts
                                                              is an inherently political enterprise. Carspecken
         except in those sections where interpretation,
                                                              (1996: 4 ff.) suggests several key premises of critical
         evaluation and inference are made explicit.
         In drafting the report it is more advisable to opt
         for over-inclusion rather than under-inclusion.          Research and thinking are mediated by power
         The ethical conventions of report writing must           relations.
         be honoured, e.g. anonymity, non-traceability.           These power relations are socially and
         The case study writer should make clear the              historically located.
         data that give rise to the report, so the readers        Facts and values are inseparable.
                                                                                  CRITICAL ETHNOGRAPHY        187

   Relationships between objects and concepts           The task here is to acquire objective data and

                                                                                                                Chapter 7
   are fluid and mediated by the social relations        it is ‘monological’ in the sense that it concerns
   of production.                                       only the researchers writing their own notes to
   Language is central to perception.                   themselves. Lincoln and Guba (1985) suggest that
   Certain groups in society exert more power           validity checks at this stage will include
   than others.
                                                           using multiple devices for recording together
   Inequality and oppression are inherent
                                                           with multiple observers
   in capitalist relations of production and
                                                           using a flexible observation schedule in order
                                                           to minimize biases
   Ideological domination is strongest when
                                                           remaining in the situation for a long time in
   oppressed groups see their situation as
                                                           order to overcome the Hawthorne effect
   inevitable, natural or necessary.
                                                           using low-inference terminology and descrip-
   Forms of oppression mediate each other
   and must be considered together (e.g. race,
                                                           using peer-debriefing
   gender, class).
                                                           using respondent validation.
   Quantz (1992: 473–4) argues that research is
                                                        Echoing Habermas’s (1979; 1982; 1984) work on
inescapably value-laden in that it serves some in-
                                                        validity claims, validity here includes truth (the
terests, and that in critical ethnography researchers
                                                        veracity of the utterance), legitimacy (rightness
must expose these interests and move participants
                                                        and appropriateness of the speaker), comprehensi-
towards emancipation and freedom. The focus and
                                                        bility (that the utterance is comprehensible) and
process of research are thus political at heart, con-
                                                        sincerity (of the speaker’s intentions). Carspecken
cerning issues of power, domination, voice and
                                                        (1996: 104–5) takes this further in suggesting sev-
empowerment. In critical ethnography the cul-
                                                        eral categories of reference in objective validity:
tures, groups and individuals being studied are
                                                        that the act is comprehensible, socially legitimate
located in contexts of power and interests. These
                                                        and appropriate; that the actor has a particular
contexts have to be exposed, their legitimacy in-
                                                        identity and particular intentions or feelings when
terrogated, and the value base of the research itself
                                                        the action takes place; that objective, contextual
exposed. Reflexivity is high in critical ethnog-
                                                        factors are acknowledged.
raphy. What separates critical ethnography from
other forms of ethnography is that, in the former,
questions of legitimacy, power, values in society       Stage 2: Preliminary reconstructive analysis
and domination and oppression are foregrounded.
                                                        Reconstructive analysis attempts to uncover the
                                                        taken-for-granted components of meaning or
How does the critical ethnographer                      abstractions that participants have of a situation.
proceed?                                                Such analysis is intended to identify the value
Carspecken and Apple (1992: 512–14) and                 systems, norms, key concepts that are guiding
Carspecken (1996: 41–2) identify five stages in          and underpinning situations. Carspecken (1996:
critical ethnography, as described below.               42) suggests that the researcher goes back over
                                                        the primary record from Stage 1 to examine
                                                        patterns of interaction, power relations, roles,
Stage 1: Compiling the primary record through the       sequences of events, and meanings accorded to
collection of monological data                          situations. He asserts that what distinguishes this
At this stage researchers are comparatively             stage as ‘reconstructive’ is that cultural themes,
passive and unobtrusive – participant observers.        social and system factors that are not usually

      articulated by the participants themselves are,         leading questions at interview, reinforced by hav-
      in fact, reconstructed and articulated, making          ing peer debriefers check on this; respondent
      the undiscursive into discourse. In moving to           validation; asking participants to use their own
      higher level abstractions this stage can utilize high   terms in describing naturalistic contexts, and en-
      level coding (see the discussion of coding in this      couraging them to explain these terms.
         In critical ethnography Carspecken (1996: 141)
                                                              Stage 4: Discovering system relations
      delineates several ways of ensuring validity at this
      stage:                                                  This stage relates the group being studied to
                                                              other factors that impinge on that group, e.g.
         Use interviews and group discussions with the
                                                              local community groups, local sites that produce
         subjects themselves.
                                                              cultural products. At this stage Carspecken
         Conduct member checks on the reconstruction
                                                              (1996: 202) notes that validity checks will
         in order to equalize power relations.
                                                              include maintaining the validity requirements
         Use peer debriefing (a peer is asked to review
                                                              of the earlier stages, seeking a match between
         the data to suggest if the researcher is being
                                                              the researcher’s analysis and the commentaries
         too selective, e.g. of individuals, of data, of
                                                              that are provided by the participants and
         inference) to check biases or absences in
                                                              other researchers, and using peer debriefers and
                                                              respondent validation.
         Employ prolonged engagement to heighten the
         researcher’s capacity to assume the insider’s
         perspective.                                         Stage 5: Using system relations to explain findings
         Use ‘strip analysis’ – checking themes and
                                                              This stage seeks to examine and explain
         segments of extracted data with the primary
                                                              the findings in light of macro-social theories
         data, for consistency.
                                                              (Carspecken 1996: 202). In part, this is a matching
         Use negative case analysis.
                                                              exercise to fit the research findings within a social
      Stage 3: Dialogical data collection                        In critical ethnography, therefore, the move is
      Here data are generated by, and discussed with,         from describing a situation, to understanding it, to
      the participants (Carspecken and Apple 1992).           questioning it, and to changing it. This parallels
      The authors argue that this is not-naturalistic in      the stages of ideology critique set out in Chapter 1:
      that the participants are being asked to reflect on          Stage 1: a description of the existing situation – a
      their own situations, circumstances and lives and           hermeneutic exercise
      to begin to theorize about their lives. This is a           Stage 2: a penetration of the reasons that brought
      crucial stage because it enables the participants to        the situation to the form that it takes
      have a voice, to democratize the research. It may           Stage 3: an agenda for altering the situation
      be that this stage produces new data that challenge         Stage 4: an evaluation of the achievement of the
      the preceding two stages.                                   new situation.
         In introducing greater subjectivity by partici-
      pants into the research at this stage Carpsecken
      (1996: 164–5) proffers several validity checks,
                                                              Some problems with ethnographic and
      e.g. consistency checks on interviews that have
                                                              naturalistic approaches
      been recorded; repeated interviews with partic-         There are several difficulties in ethnographic
      ipants; matching observation with what partici-         and natural approaches (see http://www.routledge.
      pants say is happening or has happened; avoiding        com/textbooks/9780415368780 – Chapter 7, file
                              SOME PROBLEMS WITH ETHNOGRAPHIC AND NATURALISTIC APPROACHES                      189

7.3. ppt). These might affect the reliability and          critical ethnography, accepts the perspective

                                                                                                                 Chapter 7
validity of the research, and include the following.       of the participants and corroborates the status
                                                           quo. It is focused on the past and the present
 1 The definition of the situation: the participants
                                                           rather than on the future.
   are being asked for their definition of the
                                                       5   There is the difficulty of focusing on the
   situation, yet they have no monopoly on
                                                           familiar, participants (and, maybe researchers
   wisdom. They may be ‘falsely conscious’
                                                           too) being so close to the situation that
   (unaware of the ‘real’ situation), deliberately
                                                           they neglect certain, often tacit, aspects of
   distorting or falsifying information, or being
                                                           it. The task, therefore, is to make the familiar
   highly selective. The issues of reliability and
                                                           strange. Delamont (1981) suggests that this
   validity here are addressed in Chapter 6 (see
                                                           can be done by:
   the discussions of triangulation).
 2 Reactivity: the Hawthorne effect – the pres-                studying unusual examples of the same
   ence of the researcher alters the situation as              issue (e.g. atypical classrooms, timetabling
   participants may wish to avoid, impress, di-                or organizations of schools)
   rect, deny, or influence the researcher. Again,              studying examples in other cultures
   this is discussed in Chapter 6. Typically the               studying other situations that might have
   problem of reactivity is addressed by care-                 a bearing on the situation in hand (e.g.
   ful negotiation in the field, remaining in                   if studying schools it might be useful to
   the field for a considerable time, ensuring                  look at other similar-but-different organi-
   as far as possible a careful presentation of the            zations, for instance hospitals or prisons)
   researcher’s self.                                          taking a significant issue and focusing on
 3 The halo effect: where existing or given infor-             it deliberately, e.g. gendered behaviour.
   mation about the situation or participants          6   The open-endedness and diversity of the situ-
   might be used to be selective in subse-                 ations studied. Hammersley (1993) counsels
   quent data collection, or may bring about               that the drive towards focusing on specific
   a particular reading of a subsequent situation          contexts and situations might overemphasize
   (the research equivalent of the self-fulfilling          the difference between contexts and situations
   prophecy). This is an issue of reliability, and         rather than their gross similarity, their routine
   can be addressed by the use of a wide, tri-             features. Researchers, he argues, should be as
   angulated database and the assistance of an             aware of regularities as of differences.
   external observer. The halo effect commonly         7   The neglect of wider social contexts and con-
   refers to the researcher’s belief in the goodness       straints. Studying situations that emphasize
   of participants (the participants have haloes           how highly context-bound they are, might ne-
   around their heads!), such that the more neg-           glect broader currents and contexts – micro-
   ative aspects of their behaviour or personality         level research risks putting boundaries that
   are neglected or overlooked. By contrast, the           exclude important macro-level factors. Wider
   horns effect refers to the researcher’s belief in       macro-contexts cannot be ruled out of indi-
   the badness of the participants (the partic-            vidual situations.
   ipants have devils’ horns on their heads!),         8   The issue of generalizability. If situations
   such that the more positive aspects of their            are unique and non-generalizable, as many
   behaviour or personality are neglected or over-         naturalistic principles would suggest, how
   looked.                                                 is the issue of generalizability going to be
 4 The implicit conservatism of the interpretive           addressed? To which contexts will the findings
   methodology. The kind of research described             apply, and what is the role and nature of
   in this chapter, with the possible exception of         replication studies?

       9 How to write up multiple realities and           Naturalistic and ethnographic research, then, are
         explanations? How will a representative view     important but problematical research methods
         be reached? What if the researcher sees things   in education. Their widespread use signals their
         that are not seen by the participants?           increasing acceptance as legitimate and important
      10 Who owns the data, the report, and who has       styles of research.
         control over the release of the data?
8        Historical and documentary research

Introduction                                            especially on matters of detail. The act of
                                                        historical research involves the identification and
Mouly (1978) states that while historical research      limitation of a problem or an area of study;
cannot meet some of the tests of the scientific          sometimes the formulation of an hypothesis (or
method interpreted in the specific sense of              set of questions); the collection, organization,
its use in the physical sciences (it cannot             verification, validation, analysis and selection
depend, for instance, on direct observation or          of data; testing the hypothesis (or answering
experimentation, but must make use of reports           the questions) where appropriate; and writing a
that cannot be repeated), it qualifies as a scientific    research report. This sequence leads to a new
endeavour from the standpoint of its subscription       understanding of the past and its relevance to the
to the same principles and the same general             present and future.
scholarship that characterize all scientific research.      The values of historical research have been
   Historical research has been defined as the sys-      categorized by Hill and Kerber (1967) as follows:
tematic and objective location, evaluation and
                                                           It enables solutions to contemporary problems
synthesis of evidence in order to establish facts and
                                                           to be sought in the past.
draw conclusions about past events (Borg (1963).
                                                           It throws light on present and future trends.
It is an act of reconstruction undertaken in a spirit
                                                           It stresses the relative importance and the
of critical inquiry designed to achieve a faith-
                                                           effects of the various interactions that are to be
ful representation of a previous age. In seeking
                                                           found within all cultures.
data from the personal experiences and obser-
                                                           It allows for the revaluation of data in
vations of others, from documents and records,
                                                           relation to selected hypotheses, theories and
researchers often have to contend with inade-
                                                           generalizations that are presently held about
quate information so that their reconstructions
                                                           the past.
tend to be sketches rather than portraits. Indeed,
the difficulty of obtaining adequate data makes          As the writers point out, the ability of history to
historical research one of the most taxing kinds of     employ the past to predict the future, and to use
inquiry to conduct satisfactorily.1 Reconstruction      the present to explain the past, gives it a dual and
implies an holistic perspective in that the method      unique quality which makes it especially useful for
of inquiry characterizing historical research at-       all sorts of scholarly study and research.2
tempts to ‘encompass and then explain the whole            The particular value of historical research in the
realm of man’s [sic] past in a perspective that         field of education is unquestioned. Although one
greatly accents his social, cultural, economic, and     of the most difficult areas in which to undertake
intellectual development’ (Hill and Kerber 1967).       research, the outcomes of inquiry into this domain
   Ultimately, historical research is concerned         can bring great benefit to educationalists and
with a broad view of the conditions and                 the community at large. It can, for example,
not necessarily the specifics which bring them           yield insights into some educational problems
about, although such a synthesis is rarely              that could not be achieved by any other means.
achieved without intense debate or controversy,         Further, the historical study of an educational

      idea or institution can do much to help us                    Counter-Reformation or Ignatius Loyola, each
      understand how our present educational system                 of the other elements appears as a prominent
      has come about; and this kind of understanding                influence or result, and an indispensable part of
      can in turn help to establish a sound basis for               the narrative.
      further progress of change. Historical research                  For an example of historical research
      in education can also show how and why                        see Thomas (1992) and Gaukroger and Schwartz
      educational theories and practices developed. It              (1997).
      enables educationalists to use former practices to
      evaluate newer, emerging ones. Recurrent trends
      can be more easily identified and assessed from
      an historical standpoint – witness, for example,              Choice of subject
      the various guises in which progressivism in                  As with other methods we consider in this book,
      education has appeared. And it can contribute to              historical research may be structured by a flexible
      a fuller understanding of the relationship between            sequence of stages, beginning with the selection
      politics and education, between school and                    and evaluation of a problem or area of study. Then
      society, between local and central government,                follows the definition of the problem in more
      and between teacher and pupil.                                precise terms, the selection of suitable sources of
         Historical research in education may concern               data, collection, classification and processing of
      itself with an individual, a group, a movement,               the data, and finally, the evaluation and synthesis
      an idea or an institution. As Best (1970)                     of the data into a balanced and objective account
      points out, however, not one of these objects                 of the subject under investigation. There are,
      of historical interest and observation can be                 however, some important differences between
      considered in isolation. No one person can                    the method of historical research and other
      be subjected to historical investigation without              research methods used in education. The principal
      some consideration of his or her contribution                 difference has been highlighted by Borg (1963),
      to the ideas, movements or institutions of                    who suggests that in historical research, it is
      a particular time or place. These elements                    important for the student to define carefully the
      are always interrelated. The focus merely                     problem and appraise its appropriateness before
      determines the point of emphasis towards which                moving into earnest into the project, as many
      historical researchers direct their attention.                problems may not be suitable for historical research
      Box 8.1 illustrates some of these relationships               methods, while, on the other hand, other problems
      from the history of education. For example, no                may have little or no chance of yielding any
      matter whether the historian chooses to study                 significant results either because of the dearth of
      the Jesuit order, religious teaching orders, the              relevant data or because the problem is trivial.

      Box 8.1
      Some historical interrelations between men, movements and institutions

        Men                            Movements               Institutions
                                                               Type                       Specific
        Ignatius Loyola                Counter-Reformation     Religious teaching order   Society of Jesus, 1534
        Benjamin Franklin              Scientific movement;     Academy                    Philadelphia Academy, 1751
                                       Education for life
        John Dewey                     Experimentalism         Experimental school        University of Chicago
                                       Progressive education                              Elementary School, 1896

      Source: adapted from Best 1970
                                                                                         DATA COLLECTION        193

    One can see from Borg’s observations that the        be an overwhelming mass of information. Borg

                                                                                                                  Chapter 8
choice of a problem can sometimes be a daunting          (1963) observes that this requires the careful
business for the potential researcher. Once a topic      focusing, delimiting and operationalization of the
has been selected, however, and its potential and        hypothesis.
significance for historical research evaluated, the          Hill and Kerber (1967) have pointed out that
next stage is to define it more precisely, or,            the evaluation and formulation of a problem
perhaps more pertinently, delimit it so that a           associated with historical research often involve
more potent analysis will result. Too broad or too       the personality of the researcher to a greater
vague a statement can result in the final report          extent than do other basic types of research. They
lacking direction or impact. Best (1970) expresses       suggest that personal factors of the investigator
it like this: ‘The experienced historian realizes that   such as interest, motivation, historical curiosity,
research must be a penetrating analysis of a limited     and educational background for the interpretation
problem, rather than the superficial examination          of historical facts tend to influence the selection
of a broad area. The weapon of research is the           of the problem to a great extent.
rifle not the shotgun’. Various prescriptions exist
for helping to define historical topics. Gottschalk
(1951) recommends that four questions should be
                                                         Data collection
asked in identifying a topic:                            One of the principal differences between historical
                                                         research and other forms of research is that
   Where do the events take place?
                                                         historical research must deal with data that already
   Who are the people involved?
                                                         exist. Hockett (1955) argues that, as history is
   When do the events occur?
                                                         not a science which uses direct observation as
   What kinds of human activity are involved?
                                                         in chemistry or biology, the historian, like the
   As Travers (1969) suggests, the scope of a topic      archaeologist, has to interpret past events by
can be modified by adjusting the focus of any             the traces which have been left. Of course, the
one of the four categories; the geographical area        historian has to base judgements on evidence,
involved can be increased or decreased; more or          weighing, evaluating and judging the truth of
fewer people can be included in the topic; the           the evidence of others’ observations until the
time span involved can be increased or decreased;        hypothesis explains all the relevant evidence.
and the human activity category can be broadened            Sources of data in historical research may be
or narrowed. It sometimes happens that a piece of        classified into two main groups: primary sources,
historical research can only begin with a rough idea     which are the life-blood of historical research,
of what the topic involves; and that delimitation of     and secondary sources, which may be used in the
it can take place only after the pertinent material      absence of, or to supplement, primary data.
has been assembled.                                         Primary sources of data have been described
   In hand with the careful specification of              as those items that are original to the problem
the problem goes the need, where this is                 under study and may be thought of as being in
appropriate, for an equally specific and testable         two categories. First, the remains or relics of a
hypothesis (sometimes a sequence of questions            given period: although such remains and artefacts
may be substituted). As in empirical research,           as skeletons, fossils, weapons, tools, utensils,
the hypothesis gives direction and focus to data         buildings, pictures, furniture, coins and objets
collection and analysis in historical research,          d’art were not meant to transmit information to
overcoming the risk of aimless and simple                subsequent eras, nevertheless they may be useful
accretion of facts, i.e. a hypothesis informs the        sources providing sound evidence about the past.
search for, and selection of, data, a particular         Second, those items that have had a direct physical
problem if many data exist in the field. It imposes       relationship with the events being reconstructed:
a selection, a structure on what would otherwise         this category would include not only the written

      and oral testimony provided by actual participants    from earlier endeavours. The function of the
      in, or witnesses of, an event, but also the           review of the literature in historical research,
      participants themselves. Documents considered         however, is different in that it provides the
      as primary sources include manuscripts, charters,     data for research; the researchers’ acceptance or
      laws, archives of official minutes or records, files,   otherwise of their hypotheses will depend on
      letters, memoranda, memoirs, biography, official       their selection of information from the review
      publications, wills, newspapers and magazines,        and the interpretation they put on it. Borg
      maps, diagrams, catalogues, films, paintings,          (1963) has identified other differences: one is
      inscriptions, recordings, transcriptions, log books   that the historical researcher will have to peruse
      and research reports. All these are, intentionally    longer documents than the empirical researcher
      or unintentionally, capable of transmitting a first-   who normally studies articles very much more
      hand account of an event and are therefore            succinct and precise. Further, documents required
      considered as sources of primary data. Historical     in historical research often date back much
      research in education draws chiefly on the kind of     further than those in empirical research. And
      sources identified in this second category.            one final point: documents in education often
         Secondary sources are those that do not bear       consist of unpublished material and are therefore
      a direct physical relationship to the event being     less accessible than reports of empirical studies in
      studied. They are made up of data that cannot be      professional journals.
      described as original. A secondary source would          For a detailed consideration of the specific
      thus be one in which the person describing            problems of documentary research, the reader
      the event was not actually present but who            is referred to the articles by Platt (1981) where
      obtained descriptions from another person or          she considers those of authenticity, availability
      source. These may or may not have been primary        of documents, sampling problems, inference and
      sources. Other instances of secondary sources used    interpretation.
      in historical research include: quoted material,
      textbooks, encyclopedias, other reproductions of
      material or information, prints of paintings or
      replicas of art objects. Best (1970) points out
      that secondary sources of data are usually of         Because workers in the field of historical research
      limited worth because of the errors that result       gather much of their data and information from
      when information is passed on from one person to      records and documents, these must be carefully
      another.                                              evaluated so as to attest their worth for the
         Various commentators stress the importance of      purposes of the particular study. Evaluation of
      using primary sources of data where possible (Hill    historical data and information is often referred to
      and Kerber 1967). The value, too, of secondary        as historical criticism and the reliable data yielded
      sources should not be minimized. There are            by the process are known as historical evidence.
      numerous occasions where a secondary source can       Historical evidence has thus been described as that
      contribute significantly to more valid and reliable    body of validated facts and information which
      historical research than would otherwise be the       can be accepted as trustworthy, as a valid basis
      case.                                                 for the testing and interpretation of hypotheses.
         One further point: the review of the literature    Historical criticism is usually undertaken in two
      in other forms of educational research is regarded    stages: first, the authenticity of the source is
      as a preparatory stage to gathering data and serves   appraised; and second, the accuracy or worth of
      to acquaint researchers with previous research on     the data is evaluated. The two processes are known
      the topics they are studying (Travers 1969). It       as external and internal criticism respectively, and
      thus enables them to continue in a tradition,         since they each present problems of evaluation
      to place their work in context, and to learn          they merit further inspection.
                                                                          WRITING THE RESEARCH REPORT             195

External criticism                                       vanity, say, to distort or omit facts? What were

                                                                                                                    Chapter 8
                                                         the intents of the writers of the documents? To
External criticism is concerned with establishing
                                                         what extent were they experts at recording those
the authenticity or genuineness of data. It is
                                                         particular events? Were the habits of the authors
therefore aimed at the document (or other source)
                                                         such that they might interfere with the accuracy
itself rather than the statements it contains;
                                                         of recordings? Were they too antagonistic or too
with analytic forms of the data rather than the
                                                         sympathetic to give true pictures? How long after
interpretation or meaning of them in relation
                                                         the event did they record their testimonies? And
to the study. It therefore sets out to uncover
                                                         were they able to remember accurately? Finally,
frauds, forgeries, hoaxes, inventions or distortions.
                                                         are they in agreement with other independent
To this end, the tasks of establishing the age
or authorship of a document may involve tests
                                                             Many documents in the history of education