Six Sigma

							Teaching Ethics in Statistics Class




     John H. Walker
     Department of Statistics
     Cal Poly, San Luis Obispo
     jwalker@calpoly.edu
Outline
    Ethical Guidelines for Statistical Practice (ASA, 1999)

    Some thoughts on the JSM session

    Teaching ethics at Cal Poly

    A statistical pet peeve




Walker - CauseWeb - 2008   Teaching Ethics in Statistics Class   2
ASA Ethical Guidelines: Overview
      Prepared by the Committee on Professional Ethics
      Approved by the Board of Directors
      How many have read them?
      Eight sections:
     A.    Professionalism
     B.    Responsibilities to Funders, Clients, and Employers
     C.    Responsibilities in Publications and Testimony
     D.    Responsibilities to Research Subjects
     E.    Responsibilities to Research Team Colleagues
     F.    Responsibilities to Other Statistician or Practitioners
     G.    Responsibilities Regarding Allegations of Misconduct
     H.    Responsibilities of Employers



Walker - CauseWeb - 2008        Teaching Ethics in Statistics Class   3
Ethical Guidelines: Professionalism
1.  Strive for practical relevance in statistical analyses
2.  Guard against predisposition about results
3.  Remain current in statistical methodology
4.  Assure adequate statistical and subject-matter expertise
5.  Use only methodologies suitable to the data
6.  Do not join a research project unless you can expect
    valid results and your name is not used without consent
7. Understand the theory, data, and methods behind
    automated procedures
8. Recognize the implications of multiple frequentist tests
9. Respect and acknowledge the contributions of others
10. Disclose conflicts of interest and resolve them

Walker - CauseWeb - 2008   Teaching Ethics in Statistics Class   4
JSM Session: Teaching Ethics in Statistics Class
George McCabe, Purdue Univ.
“Ethics and the Introductory Statistics Course”

Patricia Humphrey, Georgia Southern Univ.
“Ethics, It’s for Everyone!”

Paul Velleman, Cornell Univ.
“Truth, Damn Truth, and Statistics”
Journal of Statistics Education
www.amstat.org/publications/jse/v16n2/velleman.html



Walker - CauseWeb - 2008   Teaching Ethics in Statistics Class   5
McCabe
    “New Course” vs. “Old Course”
    “Old course” ethics examples:
         Decide the significance level before looking at the p-value
         Make sure assumptions are satisfied. (Then what?)
         Don’t say too much!
    “Students are afraid to conclude anything”
    “New course” ethics emphasizes:
         Question formulation
         Correct choice of method
         Focusing on the data


    Are we teaching the “new course” with “old” ethics?


Walker - CauseWeb - 2008       Teaching Ethics in Statistics Class      6
Humphrey
    Data ethics
         Much more than “have we avoided bias”


    Institutional Review Boards
         Confidentiality
         Informed Consent


    Case studies are great ways to teach ethics!

    Continuous reinforcement throughout the class
         Project data collection
         Labs
         Exams

Walker - CauseWeb - 2008       Teaching Ethics in Statistics Class   7
Velleman
    Statistics is the honest search for truth about the world

    Good statistics requires judgment

    “The best analysis often arises from the Darwinian
     competition among alternative models.”
          Survival of the best fit
          In the end, there may not be a single “best” model


    Public mistrust of statistics
          “The problem isn’t that another sample may give a different
          answer, but that another statistician working with the same
          sample may give a different answer.”

Walker - CauseWeb - 2008         Teaching Ethics in Statistics Class    8
Teaching Ethics in the Cal Poly Statistics Major
    First quarter (“Concepts and Controversies” level course)
         Debates on ethical issues (animal testing, informed consent)
         Complete NIH online ethics training
              http://researchethics.od.nih.gov


    Late 1st/Early 2nd year (2 course Applied Stat sequence)
         Choice of statistical method
         Data collection
         Type I & Type II errors, power
         Multiple comparison methods
         Assumptions and alternative methods
         Limits of a statistician
              Importance of subject matter knowledge
              Practical significance

Walker - CauseWeb - 2008           Teaching Ethics in Statistics Class   9
Teaching Ethics in the Cal Poly Statistics Major
    Third & fourth year (Electives, e.g. Survey Sampling)
         Nonresponse in survyes
         Class project
              Institutional Review Board / Human Subjects Committee


    Last quarter (Capstone: Communication and Consulting)
         Team projects
              Mock consulting sessions
              “Unaided” choice of statistical method
              Dealing with “pushy” clients
         ASA Ethical Guidelines




Walker - CauseWeb - 2008           Teaching Ethics in Statistics Class   10
Teaching Ethics: Conclusions
    Emphasize “judgment points” in data analysis
    Discuss alternative choices and consequences
         Study design
              Observational study vs. designed experiment
              Was the data collected ethically?
         Assumption checking and reexpression
              What is the possible effect of a violation? Of reexpression?
              All assumptions are not created equal. Some are more important.
         Multiple comparisons
              How many tests did you run? Each p-value you look at is a test.
         Outliers and influential observations
              Identify and gather information. Are they real or errors?
              How do they affect the results?
              Disclose any changes to your data.


Walker - CauseWeb - 2008           Teaching Ethics in Statistics Class           11
Teaching Ethics: Conclusions
    Focus on the data: an analytical outline
         Look at the data. (Make a graph.)
         Analyze the data.
         Draw conclusions.
         Look at the data again. Reevaluate conclusions.


    Reporting results
         State conclusions with authority (within reason).
         Don’t confuse causation and association.
         Report statistical significance (p-value).
         Report practical significance (effect size and direction).
         When possible report intervals, not just point estimates.
         Report any unresolved problems and possible consequences.


Walker - CauseWeb - 2008     Teaching Ethics in Statistics Class       12
My (Current) Statistical Pet Peeve
    Remember Ethical Guideline #8:
     Recognize the implications of multiple frequentist tests


    Problem: Uneven application!

    Why do we usually talk about multiple tests only when
     we teach ANOVA?

    Classic example:
     One-way ANOVA with 4 factor levels
     6 pairwise comparisons
     Standard multiple comparison methods control overall error rate


Walker - CauseWeb - 2008     Teaching Ethics in Statistics Class       13
What about…
    Multiple regression
    Multifactor ANOVA
    Multiple response variables
    Multiple regression or multifactor ANOVA with several
     response variables in the study

    Canned multiple comparison methods do not control the
     overall Type I error rate in these situations.

    Do we tell our students enough about this problem?




Walker - CauseWeb - 2008   Teaching Ethics in Statistics Class   14
Examples
    A multiple regression with 10 predictor variables
     Each predictor tested at a = .05
     10 tests. No control on overall Type I error rate


    A three-factor experiment, each factor with 4 levels
     Each term tested at a = .05 with multiple comparisons at a = .05
     3 main effects, 3 two-way interactions, 1 three-way interaction
     Canned MC methods will adjust for comparisons within each factor,
        but not across the different terms in the model.
     7 tests. No control on overall Type I error rate


    The above design with 5 univariate responses
     35 tests. Yikes!


Walker - CauseWeb - 2008     Teaching Ethics in Statistics Class     15
Solutions
    Bonferroni adjustment
         Controls overall Type I error rate, but very conservative
         Simple enough to use everyday—even in intro classes
         Higher level classes could use more powerful step down versions


    What about power?
         Who says you have to have a 5% overall Type I error rate?
         Before analysis, just choose a higher overall Type I error rate.


    A pseudo-Bonferroni adjustment (working backwards)
         Don’t like weird fractional individual significance levels?
         Use a small, rounded comparison-wise rate.
         Back compute the upper-bound on the overall Type I error rate.

Walker - CauseWeb - 2008       Teaching Ethics in Statistics Class           16
What To Tell Students
    Be aware of the problem: Ignorance is not bliss!

    Discuss the consequences of different approaches
         Exploratory vs. confirmatory analyses


    Balancing power and overall Type I error rate
         What to do may be a judgment call
         If you adjust, understand the power implications
         If not, count the tests, then compute and report the Type I error bound


    Stand up to clients who don’t want to adjust
         Guiding principle: The honest search for truth about the world


    Then, make sure we practice what we teach!


Walker - CauseWeb - 2008         Teaching Ethics in Statistics Class                17
                           Thank you!




Walker - CauseWeb - 2008   Teaching Ethics in Statistics Class   18