Docstoc

Dont Confuse Me with The Facts; Explaining Research-Based

Document Sample
Dont Confuse Me with The Facts; Explaining Research-Based Powered By Docstoc
					Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based
Listeners
By Cybèle Elaine Werts
cybelew@aol.com
Skype Handle: supertechnogirl



One of the interesting things about working
in special education is that it‟s a bit of a
lightening rod these days. The main reason
for this is the No Child Left Behind
(NCLB) act which has, more than any
previous legislation, brought students with
disabilities to the forefront of awareness
among Americans. The result is that I‟m
often asked questions like “why is it that
there are more kids with disabilities than
when we were young?” It‟s a legitimate
question, because it does seem that way
sometimes.

The Personal Approach
I usually start by talking about when I was
in elementary school and the only kid with
disabilities I knew was a student who was
learning impaired, who we made fun of
mercilessly (and I‟m so sorry now). Did
our school have kids who were blind, deaf, autistic, wheelchair bound, or have behavior and learning
issues? Probably. But this was circa 1960‟s and those students could have been in separated classrooms,
down in the basement somewhere, in different schools entirely, or frankly, not being educated at all. In
some cases they were put in mental institutions. You see, back before the Education of all Handicapped
Children Act was passed by congress in 1975, kids with special needs were often forgotten or even
abused, because disabilities were something that were not very well understood, so there weren‟t a lot of
best practices as to how to help them learn. Diagnosis has also changed radically, so conditions like
autism, which was misunderstood and frequently misdiagnosed as mental retardation thirty years ago,
are now far more often correctly diagnosed, as well as taught with much advanced techniques.

Data & Cultural Issues
Another problem is cultural because there was, and still is, a great deal of shame and lack of
understanding about disabilities. This fear can cause students with disabilities to avoid other students,
and sadly the rest of us to avoid them as well. Fortunately, this has changed in radical ways as the
population of people with disabilities has become empowered and spoken out in their own voice. The
bottom line however, is that thirty years ago those kids were often not diagnosed, educated, or even
seen. Today, they are none of those things, and thanks to NCLB they have to pass the same tests that
non-disabled kids do, which of course makes sense because being in a wheelchair doesn‟t by definition
either increase or decrease your I.Q.. When you put all these issues together, it certainly can seem like
there are more students with disabilities.
Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners
By Cybèle Elaine Werts                                                                                 1
I tell you all this because the other day I was getting an ultrasound on my knee, and the technician asked
me this very question about the increase in students with disabilities. I delivered much the same spiel as
I did above, but at the end she said something like “but I still think that there are more.” In other words,
“don‟t confuse me with the facts!” So I‟m sitting there thinking that I could have saved my breath and
read a good mystery novel for having wasted my time explaining things. Also, I realized that I often
have this problem explaining evidence and research-based information to people who believe that their
personal experience is the equivalent of data gathering. In other words, people who work in data and
statistics, or generally have an analytical turn of mind, tend to understand that while anecdotal
information is useful to fill out a picture, it does not and cannot represent a broad base of a population.

Most people of course do not work in education or data, and create their sense of the world based on
their personal experience. So my ultrasound technician ultimately felt that her personal experience with
kids with disabilities trumped any factual information I could provide, despite my credibility in the field.
This happens to me fairly often and I realized that there is a need to be able to explain research or data-
driven issues to people who are experience-oriented in terms of their listening, as well as their ability to
integrate information.

Looking for the Underlying Question
In thinking about this, I developed a modified approach which includes offering up the facts as I have
done above, but doing that not as the first line of attack. You see, it‟s difficult to know where a person is
coming from when they ask a broad question like “why is it that there are more kids with disabilities
than when we were young?” I realized that the first thing to listen for is what judgment lies behind the
question. My sister always tells me that people‟s questions belay what they‟re really trying to ask. In this
case, the real question is often something like “why are my school taxes higher because there are more
kids with disabilities than there were before?” or “why are those special needs kids taking time away
from my kid in the classroom?” Sometimes the underlying question is more overt, as when someone
asks “I read that American student‟s scores in math and science are some of the lowest in the world, so
why are our teachers doing such a crappy job?” or “The local school district just cut our music and art
budgets. This will make our kids grow into adults who can read and write, but who have no cultural
awareness.” These are examples of weighted questions; that is, you can hear what the speaker is really
wanting to rant about beneath the reasoned question.

Now, presuming you‟ve received a less weighted question, you‟ll still need to do a bit of sleuthing to
find out what‟s behind that question. The speaker could be dogmatically positioned and just want to
carry on. They could be concerned about their brother‟s child who was recently diagnosed with
Attention Deficit Hyperactivity Disorder (ADHD) and are becoming aware of the complex issues
around that. Or maybe they‟ve been reading about how their local school can‟t get a budget passed
because the percentage of the budget dedicated to students with disabilities is a larger amount than in
years past.

Finding out why the person is asking about this issue has turned out to be critical in answering their
question, so I‟ve learned to go a little deeper with people about their inquiries. Quite often, in chatting
with them, I discover that there‟s often an irrational foundation to the question that I can easily
straighten out; no spiel required. For example, when most people think of students with disabilities, the
kid in their mind is one who is learning impaired, just like the only one I knew when I was young. In
fact, students who are mentally retarded are only .8% of the entire population of Students, with the many
other disabilities in far larger proportions as you can see in the chart below. But of course as I mentioned
Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners
By Cybèle Elaine Werts                                                                                 2
earlier, for most of the history of American schools, all these other kids were not seen – literally or
figuratively – for one reason or another. Being able to see this data in a way that is indisputable makes it
easy to show how “those mentally retarded kids” are not causing any schools to fail. My hope is that if
people know the fact, (at least in theory) they‟ll know better.

I‟ve included a chart below which has some data about students with disabilities alphabetically by state
along with the different kinds of disabilities. I included the chart in part to educate you about these kids,
but also to explain what I mean when I talk about evidence-based information. It‟s important to have
good facts at hand when you‟re trying to get people to blast irrational ideas out of their heads (not that it
always works as I‟ve noted, but it definitely works better than no facts). You can see in the yellow
column the percentage of students with disabilities in each state and the U.S. average which is 9.15% of
all students. The other columns show the different kinds of disabilities by type. For example the Mental
Retardation column is highlighted in orange, and you can see at the bottom that this group is actually
fairly small, just .81% of all the students. In comparison, note that the Specific Learning Disabilities is
4.14%, a much larger group. Learning disabilities are disabilities which affect a student‟s ability to read,
write, calculate, or process information.


Table 1-12. Students ages 6 through 21 served under IDEA, Part B, as a percentage of populationª, by disability category and state: Fall 2005


                                                            Specific     Speech or
                                                   All      learning      language    Mental     Emotional Multiple Hearing
                                              disabilities disabilities impairments Retardation disturbance disabilities impairments
State                                             (%)         (%)            (%)        (%)         (%)        (%)          (%)
Alabama                                          8.52         4.21          1.71       0.95         0.22       0.14         0.10
Alaska                                           9.10         4.50          2.00       0.42         0.43       0.23         0.10
Arizona                                          8.13         4.37          1.49       0.63         0.59       0.16         0.13
Arkansas                                         9.35         3.74          1.91       1.62         0.13       0.21         0.10
California                                       7.14         3.66          1.58       0.45         0.32       0.06         0.12
Colorado                                         7.07         2.96          1.49       0.33         0.82       0.29         0.13
Connecticut                                      8.35         3.20          1.66       0.39         0.86       0.30         0.09
Delaware                                         9.53         5.21          0.95       1.17         0.48            .       0.15
Florida                                          10.01        4.90          2.15       1.00         0.95            .       0.10
Georgia                                          8.62         2.65          1.78       1.25         1.13            .       0.08
USA                                              9.15         4.14          1.74       0.81         0.72       0.20         0.11


                                                                          Other                                         Traumatic
                                                   All      Orthopedic    health     Visual                   Deaf-        brain  Developmental
                                              disabilities impairments impairments impairments    Autism    blindness     injury      delay
State                                             (%)          (%)         (%)        (%)          (%)         (%)          (%)        (%)
Alabama                                          8.52          0.06        0.63       0.04         0.19        0.00         0.03       0.24
Alaska                                           9.10          0.04        0.55       0.02         0.22        0.01         0.04       0.54
Arizona                                          8.13          0.05        0.39       0.04         0.24            .        0.03           .
Arkansas                                         9.35          0.03        1.32       0.03         0.23            .        0.03           .
California                                       7.14          0.15        0.43       0.05         0.31            .        0.02           .
Colorado                                         7.07          0.84            .      0.03         0.13            .        0.04           .
Connecticut                                      8.35          0.02        1.42       0.04         0.37            .        0.02           .
Delaware                                         9.53          0.27        0.95       0.02         0.28        0.02             .          .
Florida                                          10.01         0.11        0.53       0.03         0.22            .        0.02           .
Georgia                                          8.62          0.05        1.20       0.03         0.28            .        0.02       0.14
USA                                              9.15          0.10        0.85       0.04         0.29        0.00         0.04       0.12
SOURCE: https://www.ideadata.org/index.html




Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners
By Cybèle Elaine Werts                                                                                                                    3
Do they Really Want an Answer, or Just to Rant?
Another thing you want to find out is the level of the questioner‟s interest. You might ask: “On a scale
of one to ten, how open are you to learning more about this subject?” Some people just want a quick
answer, and others really want to learn more and would appreciate some articles on the subject. Lucky
for them I‟m an information specialist! You could also ask how they learn best so you can tailor those
materials to the way they take in information, such as articles for readers, podcasts for listeners and so
on.

Once you‟ve found out where the speaker is coming from, then you can tailor your factual response – if
in fact any is appropriate – to their actual question. I‟m guessing that the reason my response with my
ultrasound technician didn‟t make any headway was because I answered the question which she may
have asked, but which never really was her real question.

All Data Is Hooey Anyway!
Finally, I‟d like to address that group of people who dismiss data entirely. I‟m guessing they feel it‟s
okay to dismiss because they‟ve read Darrell Huff‟s book How To Lie With Statistics, which also just
happens to be one of my favorite books. I actually dated a guy who said he thought data was all hooey
because it could so easily be manipulated (I seriously reconsidered the
future of our relationship at that point). This is not an uncommon
attitude, and being a lover of data myself, it‟s an attitude I‟ve faced
surprisingly often. The challenge is that although it is indeed easy to lie
with statistics, the correlation does not necessarily follow that all
statistics are lies. On the flip side is the fear issue, which my friend and
evaluation specialist Patricia Mueller addresses when she notes: “Often
it's a fear of the unknown, and for many, data is that „unknown,‟ so folks
tend to rely on their experiences versus looking at the cold, hard facts.”
Helping people get over that fear, or even admit that they might not be
familiar enough with how data works might be a longer conversation
than you planned for, but might be worth it if your mission is a critical
one.

When I took some time to talk to that guy about why he dismissed data and statistics out of hand, it
turned out that it was because he had two false beliefs. One was that data meant “studies,” as in when a
medical company does a study to see if a new drug is effective against diabetes. I explained to him that
there‟s lots of different kinds of data. For example, I‟ve been working with some data from the Office of
Special Education Programs (OSEP) which shows how much money is allocated to each state for kids
with disabilities. Since we know how many students with disabilities are in each state, a simple division
tells us how much is being spent per student, although I will note our data specialist reminds me that this
“simple” division can be used only in the broadest sense as the federal government allocates money in a
far more complex way than this. In any case, below is a chart below showing the first ten states
alphabetically. You can see that for the most part, allocations range from about $1,300 to $1,800 with
the US average being $1,500. In short, most states are pretty close to the average. I‟d say that this data is
pretty valid based on the fact that it‟s consistent among these states. There‟s actually a fair bit of data
that can be looked at and internally verified like this, meaning that in this case all the states counted their
students and came up with numbers that were in the same ballpark. Like they say in law enforcement: a
good shooting. Like I might say: It‟s good data.



Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners
By Cybèle Elaine Werts                                                                                  4
                                                                   Total number of       Total Amount of Money
                                                                    Students with         Allocated per Student
                                         Total Federal Grant      Disabilities in that     (Grant/Number of
                      State                 to the State*              State**                  students)
                      Alabama                167,634,539                92,635                   1,809.62

                      Alaska                  32,451,580                17,997                   1,803.17

                      Arizona                162,327,526               124,504                   1,303.79

                      Arkansas               103,400,423                67,314                   1,536.09

                      California            1,130,940,237              676,318                   1,672.20

                      Colorado               137,481,329                83,498                   1,646.52

                      Connecticut            122,566,945                71,968                   1,703.08

                      Delaware                29,741,783                18,857                   1,577.23
                      Florida                580,456,790               398,916                   1,455.09
                      Georgia                285,369,440               197,596                   1,444.21

                      United States        $10,582,960,540             6,720,400                 1,574.75


                      **Fiscal Year 2006 Allocations - Grants To States- Individuals With Disabilities Education
                      Act - Part B, Section 611. Data is from IDEAdata.org Source: U.S. Dpt of Ed, Office of
                      Special Education Programs, Data Analysis System (DANS), OMB# 1820-0043 =CHAR(34)
                      &"Children with Disabilities Receiving Special Education Under Part B of the Individuals"


                      **Table 1-1. Children and students served under IDEA, Part B, by age group and state: Fall
                      2005 Age 3-21. Data is from OSEP Grant Award Letters and Funding Pages Website
                      http://www.ed.gov/fund/data/award/idea/index.html




Another misunderstanding he had was that data manipulation by definition was a bad thing, and because
it occurred – all statistics were colored in the negative by it. He told me about a study he‟d read about
where the medical company tested a new drug and there were some participants who had bad reactions
to the medication. Those outliers were removed from the data because it was assumed that they were not
part of the bell curve of the majority of people who would respond in the most normal way to the
medication. This turned out to be a very bad idea, because when administered to enough people, the
number of people (outliers) who had a negative reaction became quite significant. The result was that
the medication eventually had to be recalled. In this case, my friend was correct because in medical
studies particularly, ignoring outlier data can result in sickness or even death.

On the other hand, I suggested he consider an internal employee review survey done in a very small
company, so small that one person could very much affect the results one way or another. One person
who didn‟t have their requisite cup of java that morning and rated everyone in the survey with terrible
ratings would completely skew the results. In this case, a good data analyst would probably want to
remove this outlier because the entry was clearly not within the bounds of the normal data curve and
would adversely affect the results. In short, you cannot dismiss a statistical technique like removing
outliers without carefully considering the specific parameters of the particular data you‟re looking at.



Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners
By Cybèle Elaine Werts                                                                                             5
Of course it‟s just as easy to find invalid data too, but that‟s a different article, indeed a different book.
In the mid range you might find the All Theories Proven with One Graph chart particularly amusing, as
well as interesting for its wry humor about data and the impossibility of showing everything we want to
in a graphical format. It comes to us from the Journal of Irreproducible Results, an entertaining
magazine on science and humor. I‟m also not here to convince anyone that data is the greatest thing on
the planet (although it may well be). Rather, that in the quest to use data in a way that reaches people
effectively, just spewing it out is rarely going to do it. As we information specialists and Alex Trebec of
Jeopardy know, finding out the real question is really what the work is all about.

RESOURCES




                  How to Lie With Statistics
                  by Darrell Huff (Author), Irving Geis (Illustrator)
                  Available on www.amazon.com




                  Creating Effective Graphs: Solutions for a Variety of Evaluation Data
                  By Gary T. Henry, Editor
                  *for those of you who like a more serious approach
                  Available on www.amazon.com



The Journal of Irreproducible Results
A Science Humor magazine
http://www.jir.com/home.html




                             Cybèle Elaine Werts is an information specialist for a national non-profit
                             research and development agency and is co-editor of Education Libraries, the
                             peer-reviewed journal of the Education Division of the Special Libraries
                             Association (SLA). She can be reached at cybelew@aol.com. Cybèle's
                             articles, interviews, and podcasts can be found on her website at
                             www.supertechnogirl.com.



Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners
By Cybèle Elaine Werts                                                                                  6

				
DOCUMENT INFO