Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Quantitative Literacy Hoelter Alter by eEDv5N0


									Quantitative Literacy:
 Assessment & Practice
    Virtual OR Meeting, 2009

Flora McMartin, Broad-based
      Knowledge, Inc.
Corrine Taylor, Wellesley College &
      National Numeracy Network

                                      Page 1

• Flora McMartin
  President, Broad-based Knowledge, Inc.

• Corri Taylor
  Director, Quantitative Reasoning
  Program, Wellesley College;
  President, National Numeracy Network

                                    Page 2
Overview of the Presentation

• What is “quantitative literacy” (QL)?

• Some background on “assessment”

• Examples of QL assessment in practice

                                      Page 3
QL is defined as

• the ability to understand and use
  quantitative measures and inferences that
  allow one to function as a responsible
  citizen, productive worker, and discerning
  consumer. (Bernie Madison)

• the ability to identify, understand and use
  quantitative arguments in everyday
  contexts. Quantitative literacy describes a
  habit of mind rather than a set of topics
  of a list of skills. (Deborah Hughes Hallett)

                                          Page 4
MAA’s QL Competencies
• Reading and understanding quantitative info in
    graphs, tables, etc.
•   Interpreting quantitative info and drawing
    appropriate inferences
•   Solving problems using logic, math, statistics
•   Estimating answers and checking for
•   Communicating quantitative info – verbally,
    graphically, numerically
•   Recognizing the limitations of mathematical or
    statistical models

                                             Page 5
QL in the Social Sciences

• Nicely captured in the NCED book
  Mathematics and Democracy: The
  Case for Quantitative Literacy by
  Lynn Steen
• Statistics for data analysis in all
  social sciences
• Basic QR skills to be an informed
  citizen, voter, etc.
                                               Page 6
Assessing QL: The Ws and the H

• What to assess?
  – Students’ attitudes, behaviors; skills
  – Course materials, modules
  – Program effectiveness
• Why assess?
  – For student placement
  – For enhancement of course materials
  – For planning and accreditation
  – To test innovation             Page 7
Assessing QL: The Ws and the H
• When to assess?
  – Upon college entry and exit and later
  – Before, during, and after particular courses
• How to assess?
  – Using various kinds of “tests”
  – Using “portfolios” and other tools
  – Ensuring the assessment tools match the
    purpose and the level of QL being assessed
  – Evaluating with clear rubrics

                                           Page 8
   Some practical questions when
   considering assessment
• Is the assessment feasible (timing, logistics,
• What is the impact of the type of institution?
   – What is the skill level of students?
• What course level(s) will be involved? (lower division,
  upper division)
• What type of administrative support is available?
   – collecting/storing/analyzing data
• What is the scope of the assessment? (Course to
• Is IRB review necessary?
                                                    Page 9
Assessing Attitudes & Behaviors

• Student Attitude Assessment –
  developed at Dartmouth by J. Korey–
  Four scales: utility, personal growth,
  ability, & interest
• Office of Institutional Research –
  surveys and analyses of students’
  course-taking behavior
• Continuous feedback from
  quantitative faculty
                                   Page 10
Assessing Quantitative Skills:
Four Examples in Practice
• QR Placement Test for incoming students
  (Wellesley College)
• Test of two QR learning objectives for
  students who completed gen. ed.
  requirements (James Madison University)
• Rubric for QR in rhetoric; in sophomore
  writing portfolios (Carleton College)
• Rubric for evaluating QL skills gained over
  college career; in electronic portfolios
  (AAC&U’s QL VALUE project)
                                         Page 11
Wellesley’s QR Placement

• Assesses incoming students’ QR
  skills; weakest performers on test
  need to take basic QR course before
  enrolling in quantitative courses
• 18 open-ended questions
• Booklet with info available at

                                                               Page 12
JMU’s QR Test
• Measures two specific learning objectives…
  How well students who completed gen ed:
   – Use graphical, symbolic, and numerical
     methods to analyze, organize, and interpret
     natural phenomena
   – Discriminate between association and
     causation; how to establish causation
• Test is 26 computerized, multiple-choice

                                                                      Page 13
Carleton’s QR in rhetoric
• Examine papers in sophomore writing
  portfolios for evidence of QR
• Rubric:
• For each paper, examines
   – Potential relevance of QR (none, peripheral,
   – Extent of QR (scale: 1 to 3)
   – Overall quality of implementation,
     interpretation, and communication (scale: 1-4)
   – Problems (ambiguous words rather than
     numbers; fail to describe data collection
     methods; fails to provide comparisons)

                                                                 Page 14
AAC&U’s QL rubric

• For institutional use in evaluating and
  discussing student learning over all the
  college years
• Rubric (scale: 1-4) applied to components
  of students’ e-portfolios
• Six QL skill areas for evaluation:
  interpretation, representation, calculation,
  application/analysis, assumptions,

                                                               Page 15
Infusing QL throughout the
Social Science Curriculum
• NSF-CCLI funded partnership between ICPSR and Social
  Science Data Analysis Network (SSDAN)
• Purpose: Transform undergrad instruction by improving
  the teaching of QR and the way students understand
  research in the social sciences
• Activities
   – Study the impact of SSCAN and ICPSR teaching modules
     with regards to QL
   – Develop new teaching resources (American Community
   – Provide new instructional materials linked to ICPSR
   – Build national social science community of users of these

                                                          Page 16
Assessment Focus of “Infusing QL”
• Assessment tool/process development
  and testing
  – 8 Faculty from diverse institutions
    developing and testing tools
  – Mainly lower division introductory classes
  – Tools map to QL student learning
• Survey of faculty use/adoption of QL

                                        Page 17
Building on the AAC&U QL Rubric

 •   Interpretation             • Communication
 •   Representation             • Find/Identify/
 •   Calculation                  Generate Data
 •   Analysis                   • Research design
 •   Method selection           • Confidence
 •   Estimation/                • Content learning
     Reasonableness               outcomes

           Items in blue: direct map to AACU Rubric
                 Items in green: indirect map

                                                      Page 18
Using Rubrics to Plan for
Teaching and Assessing
                Design      Experience     Assess if
               Learning        and       Students’ met
                Activity   demonstrate   expectations

                                           Page 19
Sample Assessment Plan*

                                                    Page 20
      * Adapted from Jill Bouma’s Assessment Plan
A Short Digression
• A short digression to Bloom’s
  taxonomy: a thinking skills framework
        Lower Order          Higher Order
        Remembering            Analyzing
        Understanding          Evaluating
           Applying             Creating

• Why is this important?
  – Practical: course level, student level,
  – Methodological: qualitative/quantitative
                                              Page 21
Assessment Tools
Tool must collect data appropriate to level (Bloom)
and Learning Outcome (Rubric)
     Lower-order                 Higher-order
• Quiz                     •   Report/in-depth paper
    – Multiple choice      •   Chart/outline
    – Definition
                           •   Debate, panel
    – List
                           •   Investigation
•   Problems
                           •   Evaluation
•   Examples
                           •   Media product
•   Simulations
                           •   Art work
•   Performance
                                              Page 22
 Sample Rubric for a Paper*
   Outcome           Excellent            Competent         Needs Work
Interpretation:   Consistently          Uses correct       Incorrect
reading and       uses correct          numbers but        numbers and
reporting         numbers; uses         makes some         frequent
frequencies;      %’s from the          mistakes with      mistakes in
properly          proper cell;          language usage     language
interpreting      correctly adds        Uses %’s from      Uses wrong %’s
numbers in        across or down        correct cell;      and can not
bivariate table   different             some problems      combine
                  categories            when               categories
                                        combining          correctly

              * Adapted from Jill Bouma’s grading rubric
                                                                 Page 23
Final Thoughts

• Wrap-up
• Questions from you

                       Page 24

To top