VIEWS: 3 PAGES: 6 POSTED ON: 2/26/2010
Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners By Cybèle Elaine Werts firstname.lastname@example.org Skype Handle: supertechnogirl One of the interesting things about working in special education is that it‟s a bit of a lightening rod these days. The main reason for this is the No Child Left Behind (NCLB) act which has, more than any previous legislation, brought students with disabilities to the forefront of awareness among Americans. The result is that I‟m often asked questions like “why is it that there are more kids with disabilities than when we were young?” It‟s a legitimate question, because it does seem that way sometimes. The Personal Approach I usually start by talking about when I was in elementary school and the only kid with disabilities I knew was a student who was learning impaired, who we made fun of mercilessly (and I‟m so sorry now). Did our school have kids who were blind, deaf, autistic, wheelchair bound, or have behavior and learning issues? Probably. But this was circa 1960‟s and those students could have been in separated classrooms, down in the basement somewhere, in different schools entirely, or frankly, not being educated at all. In some cases they were put in mental institutions. You see, back before the Education of all Handicapped Children Act was passed by congress in 1975, kids with special needs were often forgotten or even abused, because disabilities were something that were not very well understood, so there weren‟t a lot of best practices as to how to help them learn. Diagnosis has also changed radically, so conditions like autism, which was misunderstood and frequently misdiagnosed as mental retardation thirty years ago, are now far more often correctly diagnosed, as well as taught with much advanced techniques. Data & Cultural Issues Another problem is cultural because there was, and still is, a great deal of shame and lack of understanding about disabilities. This fear can cause students with disabilities to avoid other students, and sadly the rest of us to avoid them as well. Fortunately, this has changed in radical ways as the population of people with disabilities has become empowered and spoken out in their own voice. The bottom line however, is that thirty years ago those kids were often not diagnosed, educated, or even seen. Today, they are none of those things, and thanks to NCLB they have to pass the same tests that non-disabled kids do, which of course makes sense because being in a wheelchair doesn‟t by definition either increase or decrease your I.Q.. When you put all these issues together, it certainly can seem like there are more students with disabilities. Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners By Cybèle Elaine Werts 1 I tell you all this because the other day I was getting an ultrasound on my knee, and the technician asked me this very question about the increase in students with disabilities. I delivered much the same spiel as I did above, but at the end she said something like “but I still think that there are more.” In other words, “don‟t confuse me with the facts!” So I‟m sitting there thinking that I could have saved my breath and read a good mystery novel for having wasted my time explaining things. Also, I realized that I often have this problem explaining evidence and research-based information to people who believe that their personal experience is the equivalent of data gathering. In other words, people who work in data and statistics, or generally have an analytical turn of mind, tend to understand that while anecdotal information is useful to fill out a picture, it does not and cannot represent a broad base of a population. Most people of course do not work in education or data, and create their sense of the world based on their personal experience. So my ultrasound technician ultimately felt that her personal experience with kids with disabilities trumped any factual information I could provide, despite my credibility in the field. This happens to me fairly often and I realized that there is a need to be able to explain research or data- driven issues to people who are experience-oriented in terms of their listening, as well as their ability to integrate information. Looking for the Underlying Question In thinking about this, I developed a modified approach which includes offering up the facts as I have done above, but doing that not as the first line of attack. You see, it‟s difficult to know where a person is coming from when they ask a broad question like “why is it that there are more kids with disabilities than when we were young?” I realized that the first thing to listen for is what judgment lies behind the question. My sister always tells me that people‟s questions belay what they‟re really trying to ask. In this case, the real question is often something like “why are my school taxes higher because there are more kids with disabilities than there were before?” or “why are those special needs kids taking time away from my kid in the classroom?” Sometimes the underlying question is more overt, as when someone asks “I read that American student‟s scores in math and science are some of the lowest in the world, so why are our teachers doing such a crappy job?” or “The local school district just cut our music and art budgets. This will make our kids grow into adults who can read and write, but who have no cultural awareness.” These are examples of weighted questions; that is, you can hear what the speaker is really wanting to rant about beneath the reasoned question. Now, presuming you‟ve received a less weighted question, you‟ll still need to do a bit of sleuthing to find out what‟s behind that question. The speaker could be dogmatically positioned and just want to carry on. They could be concerned about their brother‟s child who was recently diagnosed with Attention Deficit Hyperactivity Disorder (ADHD) and are becoming aware of the complex issues around that. Or maybe they‟ve been reading about how their local school can‟t get a budget passed because the percentage of the budget dedicated to students with disabilities is a larger amount than in years past. Finding out why the person is asking about this issue has turned out to be critical in answering their question, so I‟ve learned to go a little deeper with people about their inquiries. Quite often, in chatting with them, I discover that there‟s often an irrational foundation to the question that I can easily straighten out; no spiel required. For example, when most people think of students with disabilities, the kid in their mind is one who is learning impaired, just like the only one I knew when I was young. In fact, students who are mentally retarded are only .8% of the entire population of Students, with the many other disabilities in far larger proportions as you can see in the chart below. But of course as I mentioned Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners By Cybèle Elaine Werts 2 earlier, for most of the history of American schools, all these other kids were not seen – literally or figuratively – for one reason or another. Being able to see this data in a way that is indisputable makes it easy to show how “those mentally retarded kids” are not causing any schools to fail. My hope is that if people know the fact, (at least in theory) they‟ll know better. I‟ve included a chart below which has some data about students with disabilities alphabetically by state along with the different kinds of disabilities. I included the chart in part to educate you about these kids, but also to explain what I mean when I talk about evidence-based information. It‟s important to have good facts at hand when you‟re trying to get people to blast irrational ideas out of their heads (not that it always works as I‟ve noted, but it definitely works better than no facts). You can see in the yellow column the percentage of students with disabilities in each state and the U.S. average which is 9.15% of all students. The other columns show the different kinds of disabilities by type. For example the Mental Retardation column is highlighted in orange, and you can see at the bottom that this group is actually fairly small, just .81% of all the students. In comparison, note that the Specific Learning Disabilities is 4.14%, a much larger group. Learning disabilities are disabilities which affect a student‟s ability to read, write, calculate, or process information. Table 1-12. Students ages 6 through 21 served under IDEA, Part B, as a percentage of populationª, by disability category and state: Fall 2005 Specific Speech or All learning language Mental Emotional Multiple Hearing disabilities disabilities impairments Retardation disturbance disabilities impairments State (%) (%) (%) (%) (%) (%) (%) Alabama 8.52 4.21 1.71 0.95 0.22 0.14 0.10 Alaska 9.10 4.50 2.00 0.42 0.43 0.23 0.10 Arizona 8.13 4.37 1.49 0.63 0.59 0.16 0.13 Arkansas 9.35 3.74 1.91 1.62 0.13 0.21 0.10 California 7.14 3.66 1.58 0.45 0.32 0.06 0.12 Colorado 7.07 2.96 1.49 0.33 0.82 0.29 0.13 Connecticut 8.35 3.20 1.66 0.39 0.86 0.30 0.09 Delaware 9.53 5.21 0.95 1.17 0.48 . 0.15 Florida 10.01 4.90 2.15 1.00 0.95 . 0.10 Georgia 8.62 2.65 1.78 1.25 1.13 . 0.08 USA 9.15 4.14 1.74 0.81 0.72 0.20 0.11 Other Traumatic All Orthopedic health Visual Deaf- brain Developmental disabilities impairments impairments impairments Autism blindness injury delay State (%) (%) (%) (%) (%) (%) (%) (%) Alabama 8.52 0.06 0.63 0.04 0.19 0.00 0.03 0.24 Alaska 9.10 0.04 0.55 0.02 0.22 0.01 0.04 0.54 Arizona 8.13 0.05 0.39 0.04 0.24 . 0.03 . Arkansas 9.35 0.03 1.32 0.03 0.23 . 0.03 . California 7.14 0.15 0.43 0.05 0.31 . 0.02 . Colorado 7.07 0.84 . 0.03 0.13 . 0.04 . Connecticut 8.35 0.02 1.42 0.04 0.37 . 0.02 . Delaware 9.53 0.27 0.95 0.02 0.28 0.02 . . Florida 10.01 0.11 0.53 0.03 0.22 . 0.02 . Georgia 8.62 0.05 1.20 0.03 0.28 . 0.02 0.14 USA 9.15 0.10 0.85 0.04 0.29 0.00 0.04 0.12 SOURCE: https://www.ideadata.org/index.html Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners By Cybèle Elaine Werts 3 Do they Really Want an Answer, or Just to Rant? Another thing you want to find out is the level of the questioner‟s interest. You might ask: “On a scale of one to ten, how open are you to learning more about this subject?” Some people just want a quick answer, and others really want to learn more and would appreciate some articles on the subject. Lucky for them I‟m an information specialist! You could also ask how they learn best so you can tailor those materials to the way they take in information, such as articles for readers, podcasts for listeners and so on. Once you‟ve found out where the speaker is coming from, then you can tailor your factual response – if in fact any is appropriate – to their actual question. I‟m guessing that the reason my response with my ultrasound technician didn‟t make any headway was because I answered the question which she may have asked, but which never really was her real question. All Data Is Hooey Anyway! Finally, I‟d like to address that group of people who dismiss data entirely. I‟m guessing they feel it‟s okay to dismiss because they‟ve read Darrell Huff‟s book How To Lie With Statistics, which also just happens to be one of my favorite books. I actually dated a guy who said he thought data was all hooey because it could so easily be manipulated (I seriously reconsidered the future of our relationship at that point). This is not an uncommon attitude, and being a lover of data myself, it‟s an attitude I‟ve faced surprisingly often. The challenge is that although it is indeed easy to lie with statistics, the correlation does not necessarily follow that all statistics are lies. On the flip side is the fear issue, which my friend and evaluation specialist Patricia Mueller addresses when she notes: “Often it's a fear of the unknown, and for many, data is that „unknown,‟ so folks tend to rely on their experiences versus looking at the cold, hard facts.” Helping people get over that fear, or even admit that they might not be familiar enough with how data works might be a longer conversation than you planned for, but might be worth it if your mission is a critical one. When I took some time to talk to that guy about why he dismissed data and statistics out of hand, it turned out that it was because he had two false beliefs. One was that data meant “studies,” as in when a medical company does a study to see if a new drug is effective against diabetes. I explained to him that there‟s lots of different kinds of data. For example, I‟ve been working with some data from the Office of Special Education Programs (OSEP) which shows how much money is allocated to each state for kids with disabilities. Since we know how many students with disabilities are in each state, a simple division tells us how much is being spent per student, although I will note our data specialist reminds me that this “simple” division can be used only in the broadest sense as the federal government allocates money in a far more complex way than this. In any case, below is a chart below showing the first ten states alphabetically. You can see that for the most part, allocations range from about $1,300 to $1,800 with the US average being $1,500. In short, most states are pretty close to the average. I‟d say that this data is pretty valid based on the fact that it‟s consistent among these states. There‟s actually a fair bit of data that can be looked at and internally verified like this, meaning that in this case all the states counted their students and came up with numbers that were in the same ballpark. Like they say in law enforcement: a good shooting. Like I might say: It‟s good data. Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners By Cybèle Elaine Werts 4 Total number of Total Amount of Money Students with Allocated per Student Total Federal Grant Disabilities in that (Grant/Number of State to the State* State** students) Alabama 167,634,539 92,635 1,809.62 Alaska 32,451,580 17,997 1,803.17 Arizona 162,327,526 124,504 1,303.79 Arkansas 103,400,423 67,314 1,536.09 California 1,130,940,237 676,318 1,672.20 Colorado 137,481,329 83,498 1,646.52 Connecticut 122,566,945 71,968 1,703.08 Delaware 29,741,783 18,857 1,577.23 Florida 580,456,790 398,916 1,455.09 Georgia 285,369,440 197,596 1,444.21 United States $10,582,960,540 6,720,400 1,574.75 **Fiscal Year 2006 Allocations - Grants To States- Individuals With Disabilities Education Act - Part B, Section 611. Data is from IDEAdata.org Source: U.S. Dpt of Ed, Office of Special Education Programs, Data Analysis System (DANS), OMB# 1820-0043 =CHAR(34) &"Children with Disabilities Receiving Special Education Under Part B of the Individuals" **Table 1-1. Children and students served under IDEA, Part B, by age group and state: Fall 2005 Age 3-21. Data is from OSEP Grant Award Letters and Funding Pages Website http://www.ed.gov/fund/data/award/idea/index.html Another misunderstanding he had was that data manipulation by definition was a bad thing, and because it occurred – all statistics were colored in the negative by it. He told me about a study he‟d read about where the medical company tested a new drug and there were some participants who had bad reactions to the medication. Those outliers were removed from the data because it was assumed that they were not part of the bell curve of the majority of people who would respond in the most normal way to the medication. This turned out to be a very bad idea, because when administered to enough people, the number of people (outliers) who had a negative reaction became quite significant. The result was that the medication eventually had to be recalled. In this case, my friend was correct because in medical studies particularly, ignoring outlier data can result in sickness or even death. On the other hand, I suggested he consider an internal employee review survey done in a very small company, so small that one person could very much affect the results one way or another. One person who didn‟t have their requisite cup of java that morning and rated everyone in the survey with terrible ratings would completely skew the results. In this case, a good data analyst would probably want to remove this outlier because the entry was clearly not within the bounds of the normal data curve and would adversely affect the results. In short, you cannot dismiss a statistical technique like removing outliers without carefully considering the specific parameters of the particular data you‟re looking at. Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners By Cybèle Elaine Werts 5 Of course it‟s just as easy to find invalid data too, but that‟s a different article, indeed a different book. In the mid range you might find the All Theories Proven with One Graph chart particularly amusing, as well as interesting for its wry humor about data and the impossibility of showing everything we want to in a graphical format. It comes to us from the Journal of Irreproducible Results, an entertaining magazine on science and humor. I‟m also not here to convince anyone that data is the greatest thing on the planet (although it may well be). Rather, that in the quest to use data in a way that reaches people effectively, just spewing it out is rarely going to do it. As we information specialists and Alex Trebec of Jeopardy know, finding out the real question is really what the work is all about. RESOURCES How to Lie With Statistics by Darrell Huff (Author), Irving Geis (Illustrator) Available on www.amazon.com Creating Effective Graphs: Solutions for a Variety of Evaluation Data By Gary T. Henry, Editor *for those of you who like a more serious approach Available on www.amazon.com The Journal of Irreproducible Results A Science Humor magazine http://www.jir.com/home.html Cybèle Elaine Werts is an information specialist for a national non-profit research and development agency and is co-editor of Education Libraries, the peer-reviewed journal of the Education Division of the Special Libraries Association (SLA). She can be reached at email@example.com. Cybèle's articles, interviews, and podcasts can be found on her website at www.supertechnogirl.com. Don’t Confuse Me with The Facts; Explaining Research-Based Information To Experience-Based Listeners By Cybèle Elaine Werts 6
"Dont Confuse Me with The Facts; Explaining Research-Based "