Between a rock and a hard place with literacy rate statistics

Document Sample
Between a rock and a hard place with literacy rate statistics Powered By Docstoc
					focus on IALS

Between a rock and a hard place
with literacy rate statistics
When it comes to using literacy rate statistics,                               by Susan          Sussman
advocates and policy makers are stuck between
a rock and a hard place – we are damned if we
don’t and damned if we do use statistics to help            Who suffers from it and how? Who gains from it?
us do our jobs.                                             What causes it? Population literacy surveys, statistics
                                                            that derive from them, and interpretations of that
Damned if we don’t                                          data all influence the way literacy is framed as a
                                                            policy issue. For example, reports on the
   Where dozens of social and economic issues               International Adult Literacy Survey (IALS), issued by
compete every day for media attention, public               the OECD and Statistics Canada, frame literacy as a
awareness, political support and a bigger share of          human capital issue, crucial to the economic
public funding, nearly every social, political and          performance of industrialized nations in an
economic argument is backed up by statistical               increasingly competitive global economy.
research. In the month of August 2003 alone,                   Forecasting is used in policy development to help
statistical data supported reports in the Toronto Star      decision makers make better decisions. Where literacy
and Globe and Mail on baby boomers’ retirement              is regarded as a public policy concern, literacy rate
worries, childhood obesity, crime rates, drug               data is bound to show up in the forecasts. For
trafficking, the economic integration of immigrants in      example, consider the following notes from a
Canada, the effects of family income on university          presentation on the federal government’s Skills and
attendance, gender differences in performance on            Learning Agenda (1999).
standardized reading tests, homelessness, home-                 • Canada’s demographics mean there will be
schooling, infant mortality, major depression and                  fewer new workers.
other mental illnesses among Canadians, and seniors             • Shortages of skilled workers could restrain
requiring home care services.                                      future growth and innovation.
                                                                • 8 million working age Canadians have low liter-
                                                                   acy skills (IALS levels 1 and 2) by international
       Most advocates find it impossible                           standards.
        to move literacy issues into the                        • An additional 6.5 million (IALS level 3) will
         spotlight and onto the public                             need continuous upgrading/lifelong learning to
       policy agenda without referring                             participate effectively in the knowledge-based
            to literacy rate statistics.                           economy.
                                                                Goal-setting and decision-making about the
   Journalists, decision makers and corporate leaders       allocation of public resources are also essential for
sit up straighter and take more notice when presented       policy development. These usually involve a priority-
with statistics that point to staggering numbers of         setting process in which problems, goals, services,
Canadians with literacy difficulties. Most advocates        geographic areas and/or specific population groups
find it impossible to move literacy issues into the         are ranked. Data from IALS has been used to make
spotlight and onto the public policy agenda without         the case for placing adult literacy at the top of the
referring to literacy rate statistics. And the influence    priority lists of industrialized nations, primarily as a
of literacy rate statistics doesn’t end there; it is seen   labour force development issue. Most Canadian
throughout the policy development process.                  literacy advocates acknowledge that reports from IALS
   Once decision makers have bought into the                have been instrumental in preserving or increasing
notion that literacy warrants a public policy               funding for literacy.
response, the issue literacy has to be framed further.          Once policy goals are determined, options for
What is the specific problem in need of attention?          achieving them are identified, evaluated and selected.

6   fall 2003 LITERACIES #2
                                                                                                     focus on IALS

Some argue that literacy assessment data from large-
scale surveys must be used to inform decisions about
how best to address literacy problems.

      Without such data it is difficult to deter-
      mine what types of educational programs
      are needed and where funding should be
      channeled. For example, national data
      can be used to determine where English
      literacy programs and native language                   Meanwhile, most research experts agree that all
      literacy services or bi-lingual literacy             existing methods of estimating literacy rates have
      services are needed. (Wiley)                         significant conceptual and technical limitations. For
                                                           starters, there’s no broad consensus about what it
   Finally, policy outcomes are reviewed and               means to be literate. This is a big problem. Logic
evaluated. Expectations identified earlier in the policy   dictates that a shared understanding of the word
development process are commonly the                       literacy is a prerequisite for a shared understanding
benchmark against which outcomes are                               of how to measure it.
measured. When expectations have been                                                    Even those who define
expressed in terms of changes in literacy                                                      literacy in the same way
rates, this data becomes a yardstick                                                           may disagree about if
used to evaluate policy outcomes. For                                                         and/or how it should be
example, the 2001 Throne Speech                                                              measured. Words used in
included the goal of “significantly                                                          a definition are one thing;
increasing the proportion of adults                                                         the phenomenon captured
with higher-level skills.”                                                                 by a particular assessment
   In a nutshell, the issue of adult                                                       method is another. Many
literacy likely never would have                                                          researchers question the
gained the public and political                                                          validity of all assessment
recognition it enjoys in Canada                                                          methods used so far, arguing
today were it not for the startling                                                     that they fail to reflect how
statistical results of national                                                        real people use literacy skills
literacy surveys like IALS. To a                                                      in their real lives.
considerable extent, the design                                                           The science of statistics
of current policies and                                                              complicates the problem
programs has been influenced                                                        further when the results of
by interpretation of literacy                                                      thousands of individual literacy
rate data.                                                                        assessments are aggregated into
                                                                                  population literacy rate estimates.
        “Statistics are human beings                                             Population statistics tend to
          with the tears wiped off.”                                             diminish the complex realities of
                                                           individuals in service of creating mathematical
                                                           findings. This problem led one pundit to say,
Damned if we do                                            “Statistics are human beings with the tears wiped off.”
                                                              Different experts can legitimately draw different
   Few people, including those Canadians involved in       conclusions from the same data. A skilled
adult literacy work, have ever formally studied the        mathematician who tortures numbers long enough
sciences of statistics or research design. Thus few can    can make them confess to almost anything. The
confidently examine statistical research findings and      pliable nature of statistics led Mark Twain to say,
interpret them independently. People working in the        “There are three kinds of lies – lies, damned lies and
literacy field who do have a research background           statistics.” Many of our colleagues are wary of
rarely have the time or mandate to critically consider     statistics believing that numbers can and will be
new research. As far as many people are concerned,         manipulated to prove whatever a researcher wants
statistics are alien and alienating.                       them to show.

                                                                                                  LITERACIES #2 fall 2003   7
focus on IALS

    Numbers that quantify Canada’s literacy challenges may engage us, but they may
        also enrage us. They may reveal a lot, but they may conceal as much.

   Thus literacy advocates and policy makers are           the will and the ability to become critical consumers
damned if we do and damned if we don’t use                 of literacy rate statistics. Towards these ends, I offer
literacy statistics to help us do our jobs. Without        four recommendations.
them we seem unable to gain and keep public and
political support for broad-based strategies to                1. Given the potential influence of literacy rate
improve literacy levels. While the statistics can help            data on the development of policies and
put the issue on the map, they don’t always lead us               programs, alternative plausible interpretations
in the right direction. Thus numbers that quantify                of literacy rate data should always be identified
Canada’s literacy challenges may engage us, but they              and considered before that data is used to
may also enrage us. They may reveal a lot, but they               inform policy decisions.
may conceal as much. While discourse on literacy is            2. Researchers from various professional affilia-
influenced by the numbers at the policy level,                    tions and theoretical persuasions should be
literacy rate estimates don’t necessarily reflect                 involved in interpreting literacy rate data as
literacy problems as learners and practitioners in                early as possible in data-analysis and policy-
programs understand them.                                         development processes.
                                                               3. Consumers of research should be encouraged
                                                                  and supported to reflect critically on literacy
    We need to understand statistics, and                         rate data and related interpretations.
     when, how and why they are used.                          4. Training for advocates, policy-makers and other
                                                                  interested parties in how to critically review
   Tensions between the opportunities and challenges              literacy rate research should be included in
associated with using literacy statistics generate noise          initiatives to build Canada’s literacy-related
and confusion in Canada’s system of literacy                      research capacity.
initiatives. What gets said about literacy in public
awareness campaigns (e.g. “22 per cent of adult
Canadians have serious difficulty dealing with print”)
may be highly relevant to literacy rate statistics but     Susan Sussman                               has worked in literacy since
not entirely relevant to what goes on in literacy          1993, in the position of Executive Director of the Ontario Literacy
programs, which tend to be shaped more by realities        Coalition, as the President and board member of the Movement for
                                                           Canadian Literacy, and as an independent researcher. Sussman holds a
and needs of individual learners and practitioners.        master’s degree from Columbia University's Teachers College and a
Many of the important gains made in literacy               bachelor's degree from the City University of New York.
programs may never show up in literacy rate statistics.
Thus we run the risk of winning public and political
support today because of what the numbers show,            SOURCES:
only to lose it tomorrow because of what the               Skills and Learning Agenda 1999. Notes from a presentation by a representative from
numbers won’t show.                                            Human Resources Development Canada, at a meeting of invited literacy
                                                               stakeholders convened by Senator Joyce Fairbairn, to gather input on the
                                                               proposed Skills and Learning A genda. Ottawa.
What to do?                                                Speech from the Throne to Open the Fir st Session of the Thirty-Seventh Parliament of
                                                              Canada, Ottawa, January 30, 2001. Her Excellency the Right Honourable
    For better or worse, literacy rate statistics will        Adrienne Clarkson.

continue to be used wherever literacy policy decisions     Wiley, Terrence G. (1994) “Estimating Literacy in the Multicultural United States:
                                                              Issues and Concerns”. Washington D.C.: ERIC Digests.
are being made. Those of us who want to influence   
Canada’s literacy policies cannot simply ignore the
numbers. Instead, we need to understand them and
when, how and why they are used. We don’t                    This article borrows heavily from Susan Sussman’s report,
necessarily need to become statisticians but we do           Moving the Markers: New Perspectives on Adult Literacy,
need to know what questions to ask about large scale         recently published by Movement for Canadian Literacy.
literacy assessment research, and we need to develop         The report is a vailable online at

8   fall 2003 LITERACIES #2