Using Data: Locating, Analyzing, and Benefiting from Existing Program, Client, and Community Information* Definitions
Information = "data that have been recorded, classified, organized, related, or interpreted within a framework so that meaning emerges" Data = "facts or figures from which conclusions can be drawn" Statistics = “a type of information obtained through mathematical operations on numerical data,” and a way of providing useful information by means of numbers. “Statistical information can be presented in a variety of ways such as graphs, tables, or illustrations.”
Source: Statistics Canada, Statistics: Power from Data!
Introduction Information is powerful. HIV/AIDS prevention, testing, and care programs throughout the country need systematic information for planning and decision making. Both the Centers for Disease Control and Prevention (CDC) and the HIV/AIDS Bureau within the Health Resources and Services Administration (HRSA) spend a lot of time helping health departments and community-based HIV/AIDS prevention and care providers to collect, analyze, report, and use data. They require community needs assessment and epidemiologic profiles of HIV/AIDS, want funded services of demonstrated effectiveness based on research and evaluation, and increasingly ask for detailed client and program data reports. Yet for many community-based organizations (CBOs) engaged in HIV/AIDS prevention, testing, and care, data and research are seen primarily as funder requirements rather than beneficial activities. While executives and staff may recognize the value of data – and may have in their organizations a great deal of valuable information – much of that data is likely to go unused or minimally used. Uses of Data Information – quantitative data sets as well as qualitative information that can’t easily be measured – helps community-based AIDS prevention and service providers in nearly everything they do. Most organizations recognize these uses, even if they do not take full advantage of the possibilities. For example, data: Shows you whether and how your organization and its programs are making a difference for clients and the community Provides documentation of your work and accomplishments, so others will better understand and value the organization Over time, enables you to demonstrate results and program impact to funders and other stakeholders Improves your own understanding of your programs and clients
*
Prepared by Emily Gantz McKay, President of Mosaica, for the Pfizer Foundation’s ConnectHIV Conference, Philadelphia, PA, May 2008.
Mosaica for AED/Pfizer Foundation
Page 1
Provides a morale booster and motivator for staff, Board, and volunteers, by showing that their work makes a difference Shows changes in client needs and services over time Enables you to make program changes and other decisions from a sound base of information and knowledge
Types of Data Many HIV/AIDS groups (like other CBOs) have a lot of data sitting somewhere in their organizations. Sometimes it is in a computerized data set. Perhaps more often, it is in “hard copy” – sets of client evaluations of training programs or conferences, written summaries of focus groups, client folders, annual reports on client demographics that together show trends in the local epidemic. Among the most frequently available types of data are the following: Community needs assessments Program data – services, outputs (number of clients served, sessions provided, etc.) Client demographics, including trends Client services – e.g., who got what services, changes in demand Client outcomes – short, medium, and longer term (based on internal or external evaluation or simply from client records) Quality measures – training assessments, quality management/quality improvement (QM/QI) results, external monitoring results Anecdotal information – stories from clients or staff Data on the impact of policies or regulations Current and Potential Uses of Data If HIV/AIDS organizations were to review, analyze, and use this information, what could they do with it? The possibilities are almost endless. For example, the data can be used to: Improve Programs: Examples of Data Use Determine community/client needs A prevention program’s review of client Document and review program demographics and risk factors over a period activities of five years shows a large reported Diagnose successes and problems – increase in heterosexual risk behavior. The what worked, what didn’t, and why organization decides to increase targeting Identify best practices of women and refine its prevention model. Provide feedback to staff (individually A comparison of client evaluations for Prevention 101 sessions two years apart or by project or function) indicates that recent sessions are less Use for program refinement effective in convincing participants of the Use for planning need to change certain risky behaviors. The Demonstrate value to stakeholders: analysis leads the Project Director to Show quality, results, unique aspects of observe several sessions – and she realizes that new staff are not using some services to funders of the interactive exercises. After more Inform clients, partners, community and training on interactive methods, they begin other stakeholders about your work and fully implementing the curriculum – and its value evaluations go back to their previous high Educate funders and policy makers level. about the importance of your work Educate the public through media work
Mosaica for AED/Pfizer Foundation
Page 2
Enable others to learn from your work and use your models: Document program models Share knowledge for replication Get published in journals or less formal publications Influence public policies: Do public policy analysis to demonstrate needed policy changes or the impact of proposed policies or regulations on your program, clients, or community Testify before legislative bodies Provide information to the media about policy issues
Barriers to Using Data Why aren’t available data sets and information being more fully used? Community-based organizations face many barriers to the systematic, ongoing use of data. Among them: Lack of time Limited training or experience in handling Innumeracy = a person’s inability data to make sense of numbers; a lack of Fear of data/”innumeracy” (see box) numerical or quantitative literacy Lack of knowledge about where to start Math anxiety = a feeling of intense Lack of knowledge about inexpensive sources frustration or helplessness about of assistance one's ability to do math or use data Low priority given heavy workloads Questions about the value of data/reports, and the relative importance of using data, particularly in comparison with other priority activities Getting Started Suppose your organization recognizes the value of your existing information and data sets, and wants to use them. Here are some practical ideas for getting started. Assess and address barriers 1. Understand important barriers affecting your organization 2. Discuss/address personal barriers like math anxiety and “innumeracy” 3. Get initial advice on “what is possible and practical” in terms of overcoming identified barriers and using available data 4. Form a task force of staff, perhaps with Board members and/or volunteers, to recommend a plan 5. Identify key analyses of existing information that would provide direct and immediate benefits for the organization – and might help demonstrate the value of data to staff 6. Consider some initial training for staff – often you can arrange low- or no-cost training through a public funding agency, area university, or sister organization Choose your first project and product 7. Systematically identify and list the data sets and other types of information you have and the ways it might be used 8. Identify key analyses and findings – those that provide immediate value with limited effort 9. Review data sets – do a quick analysis/assessment of the quantity and quality of the data 10. Choose initial data/information sets to work with, based on your assessment
Mosaica for AED/Pfizer Foundation
Page 3
11. Be very specific about the products you want and how you will use them – keep focused 12. Start small – with a single data set and defined analysis Find help 13. Get technical assistance from a funder – private or public – to help explore your data and possible uses with you 14. Explore internal interest and capacity – among staff, Board members, and/or volunteers 15. Explore external sources of free or low-cost assistance, e.g., from a college or university 16. See if a current funder will add a supplemental grant for data analysis or a publication 17. Consider possible new sources of funding for data use 18. See if an expert volunteer or your task force will help prepare a funding proposal 19. Explore partnering with another organization – perhaps one that provides different services to a similar population Implement! 20. Carry out the first project – with careful monitoring 21. Debrief and build on the experience 22. Develop a one-year or multi-year data use plan to guide future analysis 23. Build data use into future proposals and work plans Conclusion Many HIV/AIDS organizations have a potential goldmine of data sitting, largely unused, on their computers or in their program or client files. If reviewed and analyzed, the information would help support sound planning and decision making and meet a variety of organizational needs. If the organization has been providing HIV prevention, education, testing, and/or care for a considerable period, it may well have data that could help document its model for sharing. Using data is often challenging because it requires time, energy, and some specific skills – but it also adds important visibility and value to HIV/AIDS work.
Mosaica for AED/Pfizer Foundation
Page 4