Ask Good Questions - ROSSMANCHANCE.COM

Document Sample
Ask Good Questions - ROSSMANCHANCE.COM Powered By Docstoc
					Ask Good Questions

  Allan J. Rossman
  Dept of Statistics
  Cal Poly – San Luis Obispo (USA)

  8th International Conference on Teaching Statistics
Frank Sinatra

   What’s the secret to
    success as a singer?
     “Sing Good Songs”

Rossman           Ask Good Questions (ICOTS-8)   2
My similarly succinct suggestion

   What’s the secret to success at
    teaching statistics?

   “Ask Good Questions”

Rossman          Ask Good Questions (ICOTS-8)   3
No hidden message

   My take-home message
     Ask Good Questions

   Quiz at end of presentation
     Q: What was my point?

     A: Ask Good Questions

   I may email you in 10 years
         Q: What do you remember?
         A: Ask Good Questions

Rossman                Ask Good Questions (ICOTS-8)   4
George Cobb

   “Judge a textbook by its
    exercises, and you
    cannot go far wrong.”

Rossman               Ask Good Questions (ICOTS-8)   5
My corollary

   “Judge a teacher by the questions that he/she asks
    of students, and you cannot go far wrong.”

Rossman               Ask Good Questions (ICOTS-8)       6
What kinds/purposes of questions?

   Guide students to develop their understanding
    and skills
         Learning activities
   Assess how well students have learned
         Assessment questions

Rossman                   Ask Good Questions (ICOTS-8)   7
What makes a question “good”?

   I respectfully decline to answer
         For now
   Instead I’ll present some examples of both
    kind of questions (learning activities,
    assessment questions)
         Taken from “Stat 101” (introductory, algebra-
          based, service course at tertiary level)
   Then I’ll revisit this question

Rossman                   Ask Good Questions (ICOTS-8)    8
Four examples (learning activities)

Rossman        Ask Good Questions (ICOTS-8)   9
George Cobb (again)

   “Shorn of all subtlety
    and led naked out of
    the protective fold of
    educational research
    literature, there comes
    a sheepish little fact:
    lectures don’t work
    nearly as well as many
    of us would like to

Rossman               Ask Good Questions (ICOTS-8)   10
Example 1: TVs and life expectancy

   Is there an association between a country’s
    life expectancy and its number of televisions
    per person?

Rossman             Ask Good Questions (ICOTS-8)    11
Example 1: TVs and life expectancy

   Does the scatterplot reveal an association?
    Which direction? How strong? Linear?
   Based on these data, would you conclude
    that sending TVs to Haiti would cause
    Haitians to start living longer?
   Identify a confounding variable that explains
    the association
   Does a strong association between variables
    imply a cause/effect relationship?
Rossman             Ask Good Questions (ICOTS-8)    12
Example 2: Reading cancer pamphlets

   Are pamphlets containing information for
    cancer patients written at an appropriate level
    that cancer patients can understand?

Rossman             Ask Good Questions (ICOTS-8)   13
Example 2: Reading cancer pamphlets

   Explain why the mean cannot be calculated
    for patients’ reading levels
   Determine median reading level of patients,
    median readability level of pamphlets
   Are the pamphlets’ readability levels well-
    matched to the patients’ reading levels?
   What proportion of patients have reading
    level below that of simplest pamphlet?

Rossman            Ask Good Questions (ICOTS-8)   14
Example 2: Reading cancer pamphlets


                        0.2                 pamphlets














                                                                                           above 12
                              under 3


Rossman                                     Ask Good Questions (ICOTS-8)                              15
Example 3: Sleep deprivation

   Does sleep deprivation have harmful effects
    on cognitive functioning three days later?
                  21 subjects; random assignment
          sleep condition

                                           -16   -8    0         8        16     24   32   40

   Identify type of study, observational units,
    explanatory and response variables

Rossman                                               Ask Good Questions (ICOTS-8)              16
Example 3: Sleep deprivation

   Students use simulation (tactile, then
    computer-based) to investigate core question
    of statistical inference:
         Is such an extreme difference unlikely to occur by
          chance (random assignment) alone (if there were
          no treatment effect)?

Rossman                   Ask Good Questions (ICOTS-8)     17
Example 3: Sleep deprivation

   Summarize conclusion, and explain
    reasoning process that supports your

Rossman            Ask Good Questions (ICOTS-8)   18
Example 4: Which tire?

   Legendary campus story
         Which tire would you pick?
   Research question: Do people tend to pick
    right front tire more often than random
         Again, students investigate the question of how
          surprising the observed class result would be
          under the null model with ¼ probability

Rossman                   Ask Good Questions (ICOTS-8)      19
Example 4: Which tire?

   What if 32% of a random sample selects right
    front? Is this a significant/convincing result?
         Or, what additional information would you need?
         Make prediction for how p-value will change (if at
          all) as sample size increases
         Calculate p-value for many different sample sizes
         Summarize how sample size affects p-value,
          strength of evidence
             Explain why this makes intuitive sense (in hindsight, if
              not in foresight)

Rossman                        Ask Good Questions (ICOTS-8)              20
Five examples (assessment items)

Rossman       Ask Good Questions (ICOTS-8)   21
Joan Garfield, Beth Chance

   Because students learn
    to value what they
    know they will be tested
    on, we should assess
    what we value.” - JG
   “The number one
    mantra to remember
    when designing
    instruments is: Assess
    what you value.” - BC

Rossman               Ask Good Questions (ICOTS-8)   22
Assessment example 1

For each of the following quantities, indicate whether it can
  NEVER be negative or can SOMETIMES be negative.

a) Sample size                              b) Sample proportion
c) Standard deviation                       d) Inter-quartile range
e) Difference in sample means               f) Odds ratio
g) Total sum of squares                     h) Slope coefficient
i) Coefficient of determination             j) Correlation coefficient
k) ANOVA F-test statistic                   l) p-value

Rossman                Ask Good Questions (ICOTS-8)                      23
Assessment example 2

The U.S. has slightly more than 300 million
  residents. In order to estimate the proportion of
  U.S. residents who have a facebook account to
  within about 3 percentage points with 95%
  confidence, about how many people should be
  randomly sampled?

    100          1000                             10,000
    100,000      1,000,000                        10,000,000

Rossman            Ask Good Questions (ICOTS-8)                24
Assessment example 3

Are people more likely to lie with email than with pencil-
  and-paper communication? A recent study involved 48
  graduate students in business at a particular university
  who participated in a “bargaining” game. Researchers
  kept track of whether the student lied about the amount
  of money involved when negotiating with another player.
  Some of the participants were randomly assigned to use
  email for their communication, while others used paper-
  and-pencil. It turned out that 24 of 26 who used email
  were guilty of lying about the amount of money involved,
  compared to 14 of 22 who used paper-and-pencil.

Rossman               Ask Good Questions (ICOTS-8)       25
Assessment example 3 (cont.)

Use a simulation analysis to investigate whether
 these data provide strong evidence in support of
 the research conjecture in the first sentence
 above. Summarize the conclusions that you
 draw from your analysis. Be sure to address
 issues of statistical significance, causation, and
 generalizability. Also explain the reasoning
 process and justification for your conclusions.

Rossman            Ask Good Questions (ICOTS-8)   26
Assessment example 4

State a research question, and describe a data
  collection plan to address it, for which a two-
  proportion z-test would be the relevant
  inference procedure. Identify the explanatory
  and response variables, and also state the
  relevant hypotheses, defining the parameter
  values appropriately. Be sure to indicate
  whether the data collection plan involves
  random sampling or random assignment.

Rossman           Ask Good Questions (ICOTS-8)      27
Assessment example 5 (from 2009 AP
Statistics exam)
   Consider the statistic mean / median. What
    values of this statistic might indicate that the
    population distribution is skewed to the right?
   Consider simulation results for values of
    mean / median, based on a normal
    population, and also the observed value of
    mean / median for given sample data. Do the
    simulation results suggest that the underlying
    population is skewed to the right? Explain.

Rossman             Ask Good Questions (ICOTS-8)   28
Assessment example 5 (from AP
Statistics, cont.)

   Use only statistics in the five-number
    summary (min, Q1, median, Q3, max) to
    propose your own measure of skewness.
    Indicate values of this statistic that would
    suggest skewness to the right. Explain.

Rossman              Ask Good Questions (ICOTS-8)   29
Oh, by the way ….

   Notice that this talk is trying to model
    question-based pedagogy
         From these specific questions/examples we can
          extract principles of “good” questions

Rossman                  Ask Good Questions (ICOTS-8)     30
So, what makes a question “good”?

   Makes students think
         Goes beyond their starting point
         Challenges without overwhelming students
   Engages students’ interest
         To put forth effort to learn
         To care about the course material being studied
   Addresses important ideas
         Indicates to students what’s valued

Rossman                   Ask Good Questions (ICOTS-8)      31
What else makes a question “good”?

   Fits within logical progression
         Builds on students’ knowledge
         Asks students to make/check predictions
         Confronts students’ misconceptions
   Inspires students to ask their own questions
         About course material
         About “real-world” phenomena that can be
          investigated using intellectual skills being learned

Rossman                    Ask Good Questions (ICOTS-8)          32
One vehicle for students posing and
investigating own questions
   Students use gapminder software (Hans Rosling) to
         Pose research questions about the world
         Investigate those questions with animated graphics
         Write report of their findings

Rossman                     Ask Good Questions (ICOTS-8)       33
Some questions from students’ projects
   Are students more likely to recycle water bottles depending on the
    proximity of recycle bin and waste basket?
   Can people better answer math problems if they are presented with
    Roman letter notation as opposed to Greek letter notation?
   Are people who walk into a clothing store more likely to purchase
    something when the weather is rainy as opposed to sunny?
   Is balsa wood less elastic after it has been immersed in water?
   Are students’ reaction times affected by whether they’ve just
    completed exercising?
   Are faculty more likely to drive a foreign car than students?
   Do guests at a dinner respond more positively to spaghetti sauce
    that has been sweetened or not sweetened?
   Do college students spend more money at a local fast food
    restaurant if they are under the influence of alcohol?

Rossman                    Ask Good Questions (ICOTS-8)                  34
More unsolicited advice

   ICOTS: Teaching Statistics
         My experience at introductory tertiary level
   Any advice for teachers of younger students?
         Ask Good Questions
   Any advice for teachers of graduate students?
         Ask Good Questions
   Any advice for statistics education
         Ask Good Questions

Rossman                   Ask Good Questions (ICOTS-8)   35
Examples of interest to me

   Does presenting activities/examples based
    on real data from genuine studies
         Improve student learning of statistical ideas?
         Increase student interest in statistics?
         Enhance transferability of skills?
   Do students learn more if tactile simulations
    are presented before technology ones?
   Does it matter whether students construct or
    simply consume simulation results?
Rossman                   Ask Good Questions (ICOTS-8)     36
Some final questions for you

   Have you ever attended a presentation titled
         Ask Bad Questions, or
         Don’t Ask Questions?
   Of course not! So, …
         Why am I wasting your time offering such obvious
          advice as “Ask Good Questions”??

Rossman                  Ask Good Questions (ICOTS-8)    37
Yeah, why am I wasting your time?

   I respectfully suggest that the next time you
    or I feel compelled to invest some of our
    valuable time, thought, energy, and creativity
    on …
         Preparing crystal-clear lectures
         Writing lucid paragraphs of exposition
         Developing software illustrations
         Crafting beautiful presentations

Rossman                   Ask Good Questions (ICOTS-8)   38
What should we do?

Rossman      Ask Good Questions (ICOTS-8)   39
What should we do?

   Resist this temptation!

Rossman             Ask Good Questions (ICOTS-8)   40
So, what do I suggest instead?

   Instead we should invest these precious
    commodities (time, thought, energy,
    creativity) on
         Developing engaging classroom activities
         Preparing thought-provoking assignments
         Writing lab or project expectations
         Crafting effective assessment items
   In other words, we should focus more of our
    attention on making sure that we …

Rossman                  Ask Good Questions (ICOTS-8)   41
What’s my point again?

Ask Good Questions!

Rossman      Ask Good Questions (ICOTS-8)   42
Thanks very much!


Rossman       Ask Good Questions (ICOTS-8)   43

Shared By: