Docstoc

Key Consumer Run Organization Stakeholder Satisfaction with Self

Document Sample
Key Consumer Run Organization Stakeholder Satisfaction with Self Powered By Docstoc
					Goal Achievement in Kansas Consumer Run Organizations September, 2003

Summary: Objective: This study analyzes three years of quarterly reports from twenty-one mutual support oriented consumer run non-profit organizations with the intent of understanding the types of goals that have been set, rates of goal achievement and predictors of goal achievement. Results: Organizations set an average of 9.4 objectives and achieved an average of 6.4 objectives each quarter. Overall, objectives were achieved with average success rate of 68%. Organizations with many people involved in running the organization (leadership size), many years of experience (age of organization), and many days of operation each quarter (days open) are able to achieve the most objectives. These three individually significant variables explain 41% of the variability in objective achievement. Implications: Consumer run organizations are generally able to accomplish what they set out to do. With time organizations get much better at accomplishing their objectives. Having a large leadership base is critical to the success of these organizations. Previous research has also shown that individual involvement in running the organization is the best predictor of positive outcomes for members. Introduction: Over the last three decades people with mental illness have increasingly become involved in developing their own initiatives to supplement the mental health system, often with an emphasis on self-help or mutual support (McLean, 1995; Chamberlin, 1990). These initiatives have gained popularity among mental health consumers because they provide opportunities for leadership, helper status and other valued social roles. Early in the history of the consumer movement, people with mental illnesses would work together in small informal groups. As this movement has grown, so has the size and sophistication of the groups. Today many of these groups have become non-profit organizations that provide an array of services including drop-in centers, support groups, peer counseling, consumer advocacy and educational activities. Along with growth has come the challenge of staying focused on the original vision of each individual organization and keeping track of progress towards this vision. To meet this challenge, twenty-one consumer run organizations in Kansas have created goals in their yearly grant applications to the state and kept track of progress on these goals in their quarterly reports. This study analyzes three years of quarterly reports from these organizations with the intent of understanding the types of goals that have been set, rates of goal achievement and predictors of goal achievement. There has not been any research in the area of organizational goal achievement with consumer run organizations, although the operations and activities of these organizations have previously been studied. Mutual support, cultural activities, advocacy, consumer knowledge development, skills training, and public as well as professional education have all been popular areas of focus for these organizations (Trainor, Shepherd, Boydell, Leff & Crawford, 1997). A study of six organizations in Michigan by Mowbray & Tan (1993) indicated that the consumer run organizations were able to meet their main program objectives of providing mutual support and informing members about community resources. Another study of nine organizations in Pennsylvania by Kaufmann et al. (1993) found that successful organizations exhibited effective
Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

leadership, organizational skills, a core group of volunteers, interdependent relationships with providers, efficient financial management, planned social activities, and the ongoing recruitment of new members. Consumer run organizations across North America have a variety of primary functions, including the operation of businesses, drop-in centers, homeless support services, and case management (Mowbray, Robinson & Holter, 2002; Mowbray, Chamberlin, Jennings, & Reed, 1988; Segal, Hardman, & Hodges, 2002; Chinman, Weingarten, Stayner, & Davidson, 2001). All the consumer run organizations in this study offer a drop-in center where consumers can build their social skills and social networks. Additionally, all organizations emphasize the provision of leadership opportunities to members. There are many different ways members can get involved such as joining the board of directors, organizing activities, and helping maintain the building. There is quite a bit of variation in the other activities that these organizations pursue but some of the most common are educational activities, support groups, public presentations about mental illness, and fundraising. Whatever consumer run organizations decide to pursue, setting goals and tracking progress can help guide these organizations towards success. Creating and tracking progress on goals is a time honored tradition which has been studied extensively in various forms. Management by Objectives is one such goal oriented philosophy that has dominated management in large corporations and in government (Odiorne, 1979; Rodgers & Hunter, 1991). Goal Attainment Scaling is another established method of goal setting and tracking that has been used for both organizational and individual evaluation (Fiester & Fort, 1978; Kiresuk, Smith, & Cardillo, 1994; Austin et al., 1976). Both of these techniques have been extensively researched and many studies have documented their efficacy (Rodgers & Hunter, 1991; Evans, 1981). The establishment and follow-up of goals has been shown to reap many rewards for those who pursue this strategy (Rodgers & Hunter, 1991; Harkins & Lowe, 2000). Organizational productivity is enhanced because goal tracking helps: (1) facilitate planning; (2) maintain a future orientation instead of getting lost in daily crisis; (3) create a shared understanding of the organizational mission and the logic behind each task; (4) enhance individual motivation; (5) provide corrective feedback (Kiresuk & Lund, 1978; Rodgers & Hunter, 1991). Goal tracking is a particularly useful strategy for consumer run mutual support organizations because it allows them to define problems and set goals that stay true to their values. These organizations are based in the self-help philosophy, which emphasizes: (1) the promotion of inner strengths, (2) a reliance on internal resources, (3) a rejection of hierarchy and impersonal rules, (4) sense of community, (5) empowerment and participation, (6) selfacceptance and openness towards individual differences (Riessman and Carroll, 1995). Setting individual goals allows these organizations to deal with problems internally, relying on their own strengths. Additionally, the goals are agreed upon by any member of the organization who is interested in giving their input rather than passed down from the executive director. This allows for a participatory process that is empowering to everyone. The current study does not examine the benefits of using goal tracking as these have already been documented. Instead this study focuses on describing the types of goals consumer run mutual support organizations set for themselves and how frequently these organizations are able to accomplish what they have set out to do. Additionally, this study investigates how consumer run organizations can improve their current rate of goal achievement.
Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

Method
Participant Organizations The 21 consumer run mutual support organizations in this study are diverse in terms of community size, operating budget, length of existence, and membership size. The organizations are spread throughout the state of Kansas, approximately proportional to the distribution of the Kansas population. While some programs exist in communities of less than 4,000, others are part of metropolitan areas greater than one million. The diversity in number of members is just as broad, ranging from 9 to 325, with an average of 74 members. Operating budgets ranged from $5,600 to $132,000 with an average of $31,000. The primary funding agency is the Kansas Department of Social and Rehabilitation Services, although several of the organizations have multiple funding partners. Although consumer run organizations have been in Kansas for more than 25 years, over half of the 21 organizations supported by the state began operating in the past five years. The average age of all the organizations is seven years, with one as old as 27 years and two with less than one year of history. When these organizations receive a grant from the Kansas Department of Social and Rehabilitations Services they are required to submit quarterly reports. This study analyzed three years of quarterly reports from twenty-one consumer run organizations. Of the 224 quarterly reports that were supposed to be submitted, 184 (82%) were included in this study. Quarterly reports are missing because they were either never submitted or they were misplaced after being received. The leadership at each consumer run organizations is responsible for completing the quarterly reports. The work may fall solely on the shoulders of the executive director but many organizations make it a team effort with contributions from members of the board of directors or paid staff. Analysis of Quarterly Reports The quarterly reports contain quantitative information about finances, attendance rates, hours of operation and the number of people contributing to voluntary leadership. The attendance and budgetary information reported above is from these reports. Additionally, there is a narrative describing progress made on the goals and objectives set by the organization at the beginning of the year in their grant application. The goals are typically long term outcomes and several measurable short term objectives are written to address each goal. An example goal is, “to reduce social isolation among persons with mental illness” and some accompanying objectives are, “to increase the number of hours open for operation each quarter to 200” and “to organize five fun activities each quarter”. Based on the qualitative data analysis process of Miles and Huberman (1994), the objectives were reviewed looking for common themes and coded into categories. Whether each objective was achieved during the quarter was also recorded. Success on each objective was usually easily determined because the objectives were designed to be very concrete. In situations where there was not enough information to judge achievement, non achievement was assumed.

Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

Findings
Organizations set an average of 9.4 objectives and achieved an average of 6.4 objectives each quarter. Overall, objectives were achieved with average success rate of 68%. The 21 consumer run organizations were quite productive over the past three years, setting a total of 1553 objectives and achieving 1057 of them. Table 1 describes the frequency with which different types of objectives were set and how often each type of objective was achieved. Of the 184 quarterly reports there were 19 (10%) that did not report on progress made towards objectives and were therefore excluded from all analysis related to objective achievement. Predictors of Objective Achievement Standard multiple regression was employed to explore the predictors of objective achievement. To create a continuous dependent variable that represents degree of achievement the total number objectives achieved in each quarterly report was computed. Of the 165 reports available for this analysis four were excluded. Two did not report the number of people who contribute to the operation of the organization (leadership size), one did not report the number of days open per quarter and one had an outlying residual value. The leadership size variable had six outliers that were more than three standard deviations above the mean. These outliers were assigned the value of the highest non outlier. Independent variables available for regression analysis were leadership size, yearly budget, hours open for the quarter, days open for the quarter, number of members participating for the quarter, average daily attendance, and the age of the organization at the time of the report. All of these variables have significant positive relationships with objective achievement. Table 2 shows the strength of each relationship along with the mean and standard deviation of each variable in this multiple regression. After entering all variables into the prediction equation the suppressor variables and non significant variables were removed to simplify interpretation. Leadership size, days open, and age of organization were the only variables individually significant in the regression equation. Together, these three variables explain 41% of the variability in objective achievement. Age of the organization was the strongest individual predictor with a beta weight of .343. Leadership size was the second strongest with a beta weight of .295 and days open was third with a beta weight of .243. Table 3 displays the correlations between the predictor variables, the unstandardized regression coefficients (B), the standardized regression coefficients (β), the t value, R, R², and adjusted R².

Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

Table 1 Breakdown of Organizational Goals and Rates of Successful Achievement (N = 166) Type of Goal % reports Success/Attempt a) Objectives to achieve goal with goal ratio Decrease social isolation among members. a) Increase or maintain days of operation 69% 60/116 = 52% b) Provide recreational and social activities 69% 103/116 = 89% c) Increase or maintain hours of operation 62% 54/105 = 51% d) Celebrate member/organizational milestones 42% 63/71 = 89% e) Provide support groups 22% 30/37 = 81% f) Provide peer counseling or a warm line 21% 27/36 = 75% Increase member involvement in organization a) Increase or maintain total membership 72% 79/121 = 65% b) Increase volunteerism 63% 74/107 = 69% c) Increase or maintain average daily attendance 58% 44/98 = 45% d) Provide member recognition 30% 43/50 = 86% e) Provide transportation to members 28% 40/48 = 83% Educate public/providers on the organization/mental illness a) Make community presentations 63% 65/106 = 61% b) Get media exposure for the organization 40% 43/68 = 63% c) Publish a newsletter 30% 34/51 = 67% d) Have members attend conferences 18% 28/30 = 93% Secure additional funding a) Conduct fundraising activities 49% 57/82 = 70% b) Submit and receive funding from additional grants 27% 28/46 = 61% c) Contract with KU to collect survey information 25% 34/43 = 79% Other goals and objectives a) Conduct member satisfaction survey 58% 48/98 = 49% a) Provide educational opportunities for members 30% 46/50 = 92% b) Develop community partnerships with joint activities 25% 38/42 = 90% c) Other (does not fit into category) 22% 32/42 = 76% Table 3 Standard Regression of Objective Achievement (N = 161) Age of Num. of Variables Organization Leaders B β t Intercept 2.175 3.91* Age of .189 .343 4.75* Organization Number of r = .285* .047 .295 4.64* Leaders Days Open r = .490* r = .137 .038 .236 3.38* *p < .001 Model Summary: R = .646*, R² = .417, adjusted R² = .406
Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

Table 2 Significant relationships with objective achievement (N = 161) Correlation with Objective Standard Variable Achievement Mean Deviation Number of Objectives r = 1.00 6.46 3.76 Achieved Yearly Budget r = .26* $31,385 $19,445 Hours Open For Quarter Days Open For Quarter Members Served for Quarter Average Daily Attendance Leadership Size For Quarter Age of Organization During Quarter *p < .01, **p < .001 r = .40** r = .44** r = .41** r = .22* r = .43** r = .54** 240 47.41 58 12.7 25.5 6.7 146 23.14 60 7.5 23.6 6.8

Implications and Considerations: This study has used documentation on goal achievement to determine what consumer run organizations are trying to do and if they are successful in their pursuits. The findings on what consumer run organizations are trying to do are largely consistent with previous research. Trainor et al. (1997) found that 93% of consumer run organizations are involved in mutual support, 73% are involved in public education and 70% are involved in educating professionals. Similarly, this study found that 93% of the reports set objectives with the end goal of increasing mutual support among members and 81% attempted to educate the public or service providers. One notable difference between the activities of the organizations in each study is with respect to advocacy. 80% of the organizations in the Trainor et al. study emphasized advocacy while almost none in this study did because they are not allowed to use state grant money for advocacy. The 69% success rate on achieving objectives set at the beginning of the year suggest that consumer run organizations are competent and generally able to produce intended results. Unfortunately, it is not possible to make valid comparisons between the success rates found in this study and the success rates of organizations in other studies. This is because of the intentionally relativistic framework created by setting individual goals and objectives. One organization that achieves 70% of their difficult goals will look the same as another organization that achieves 70% of their easy goals. Studies on the individual impacts of participation in consumer run organizations suggest that these organizations are making a difference in the lives of participants but the results remain highly speculative (Solomon & Draine, 2001). The findings on objective achievement suggest that these organizations are at least able to accomplish the tasks that are intended to promote positive outcomes. The question of whether these tasks do actually promote positive
Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

outcomes remains to be seen. One of the major limitations of this study’s findings is that objective achievement is judged through information that is self-reported by each organization. Although gross exaggerations on progress would face serious reproach, the quarterly reports are susceptible to self-report bias as everybody wants their organization to look good. Predictors of Objective Achievement Results demonstrate that there are at least three major factors that are related to the achievement of goals. Number of members involved in running the organization (leadership size), the organization’s length of existence, and the number of days the organization is open each quarter account for 41% of the variance in objective achievement. It seems that organizations with many people involved in running the organization, many years of experience, and many days of operation each quarter are able to achieve the most objectives. It should be noted that number of objectives achieved is being predicted and not the achievement success rate. This is because number of objectives achieved is thought to be a better indicator of organizational effectiveness. Achieving many objectives requires the organization to be involved in many different areas with many different activities. If achievement success rate were predicted then the organization that sets and achieves one objective will look the same as the organization that sets and achieves 15 objectives. It is encouraging that length of existence is the most powerful predictor of objective achievement (β = .343). This suggests that with practice and experience consumer run organizations become more productive and more successful in their endeavors. Number of days open for the quarter is another significant predictor of objective achievement (β = .236) that is highly correlated with age (r = .49). This finding may be indicative of a developmental process in consumer run organizations similar to what happens with years of experience. When these organizations first open they have the challenge of getting their drop-in center up and running. Over time the days when the drop-in center is open grow until they meet demand and the leaders master the initial challenge of keeping a drop-in center open. Once this foundation for the organization is established the leadership can branch out into others areas of interest, above and beyond what is required of them. The third significant predictor of objective achievement is the number of people who are contributing voluntary leadership and helping with the daily operations of the organization (β = .295). This suggests that having a strong leadership base is critical to the success of consumer run organizations. Burnout has been found as a common problem among the leadership of selfhelp groups and shared leadership is strategy that has been successful in preventing it (Meissen, 1997). Because consumer run non-profits are largely volunteer driven they face many of the same challenges as self-help groups. A strong leadership base may allow the organization to grow and achieve more without putting all the weight on one or two paid staff. A strong leadership base may be both beneficial to the organization and to the individuals who constitute the leadership base. Previous research by Segal & Silverman (2002) suggests that involvement in operating a consumer run organization is the best predictor of positive outcomes for members of such agencies. A major limitation of the previous interpretations is that the findings on the predictors of objective achievement only measure strength of relationship between the variables and do not imply causality. Although the results do support pre-existing notions on the enhancement of objective achievement, any number of plausible interpretations could be made from the data. Objective achievement may be causing leadership size, days open and length of existence instead of vice versa.
Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

References Austin, N.K., Liberman, R.P., King, L.W., & DeRisi, W.J. (1976). A comparative evaluation of two day hospitals. Journal of Nervous and Mental Disease, 163, 253-262. Chamberlin, J. (1990). The ex-patients’ movement: Where we’ve been and where we’re going. Journal of Mind and Behavior, 11, 323-336. Chinman, M.J., Weingarten, R., Stayner, D., & Davidson, L. (2001). Chronicity reconsidered: Improving the person-environment fit through a consumer-run service. Community Mental Health Journal, 37, 215-229. Evans, H.M. (1981). Psychiatric patient participation in goal setting as related to goal attainment. Dissertation Abstracts International, 41 4658B. (University Microfilms No. 1111691) Fiester, A., & Fort, D. (1978). A method of evaluating the impact of services at a comprehensive community mental health center. American Journal of Community Psychology, 6, 291302. Harkins S.G., & Lowe, M.D. (2000). The Effects of Self-Set Goals on Task Performance. Journal of Applied Social Psychology, 30, 1-40. Kaufmann, C.L., Ward-Colasante, C., & Farmer, J. (1993). Development and evaluation of dropin centers operated by mental health consumers. Hospital and Community Psychiatry, 44, 675-678. Kiresuk, T.J., & Lund, S.H. (1978). Goal Attainment Scaling. In C.C. Attkisson, W.A. Hargreaves, M.J. Horowitz, & J.E. Sorensen (Eds.), Evaluation of Human Service Programs. New York: Academic Press. Kiresuk, T.J., Smith, A., & Cardillo, J.E. (Eds.). (1994). Goal attainment scaling: Applications, theory, and measurement. Hillsdale, NJ: Lawrence Erlbaum. McLean, A. (1995). Empowerment and the psychiatric consumer/ex-patient movement in the United States: Contradictions, crisis and change. Social Science & Medicine, 40, 10531071. Medvene, L.J., Volk, F.A, & Meissen G.J. (1997) Communal orientation and burnout among self-help group leaders. Journal of Applied Social Psychology, 27, 262-278. Miles, M.B., and Huberman, M.A. (1994). Qualitative Data Analysis. Thousands Oaks, CA: Sage. Mowbray, C.T., Chamberlin, P., Jennings, M., & Reed, C. (1988). Consumer-run mental health services: Results from five demonstration projects. Community Mental Health Journal. 24, 151-156. Mowbray, C.T., Robinson, E.A., & Holter, M.C. (2002). Consumer drop-in centers: operations, services, and consumer involvement. Health and Social Work. 27, 248-261. Mowbray, C.T. & Tan, C. (1993). Consumer-operated drop-in centers: Evaluation of operations and impact. Journal of Mental Health Administration, 20, 8-19. Odiorne, G.S. (1979). MBO II: A system of managerial leadership for the 80s. Belmont, CA: Fearon Pitman. Riessman, F., & Carroll, D. (1995). Redefining self-help. San Francisco: Jossey-Bass. Rodgers, R. & Hunter, J.E. (1991). Impact of management by objectives on organizational productivity. Journal of Applied Psychology, 76, 322-336. Segal, S.P., Hardman, E.R., & Hodges, J.Q. (2002). Characteristics of new clients at self-help and community mental health agencies in geographic proximity. Psychiatric Services, 53, 1145-1152. Segal, S.P. & Silverman, C. (2002). Determinants of client outcomes in self-help agencies. Psychiatric Services, 53, 304-309.
Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu

Soloman, P., & Draine, J. (2001). The state of knowledge of the effectiveness of consumer provided services. Psychiatric Rehabilitation Journal, 25, 20-27. Trainor, J., Shepherd, M., Boydell, K.M., Leff, A., & Crawford, E. (1997). Beyond the service paradigm: The impact and implications of consumer/survivor initiatives. Psychiatric Rehabilitation Journal. 21, 132-140.

Self-Help Network Center for Community Support and Research Fairmount College of Liberal Arts and Sciences/Wichita State University, Department of Psychology In Kansas 800.445.0116 • Ph 316.978.3843 • Fax 316.978.3593 • Wichita, Kansas 67260-0034 www.selfhelpnetwork.wichita.edu