Get documents about "Case scenario for Seniors program
Document Sample
Seniors Support Program
1
Program Context
Agency A.B.C is a community based organization with a strong commitment to improving
the health and well-being of seniors. The agency continues to serve seniors in the
community and is a strong advocate for seniors’ independent living in their communities for
as long as possible. Recently, the agency was awarded a three year funding to develop a
program that seeks to improve the health of seniors by promoting physical activity and
reducing social isolation. The agency thinks promoting physical activity and reducing
isolation among seniors in the community can contribute to falls prevention which in turn
reduces the incidence of hospitalization. As the agency’s program team (i.e. staff
members, volunteers, students) responsible for the implementation of the program, the
team was tasked to engage stakeholders in the development of the evaluation process for
this program.
The program team used a facilitated process to go through the development of the
evaluation processes for the program. The team began with this basic assumption:
1. Seniors have multiple pressing issues (e.g. mobility issues, limited family/community
support, safety concerns) and other systemic issues that require ongoing support. If
these aspects of their lives are not taken into account, any supports they receive are
not likely to lead to independent and health living in their community.
First, the program team identified who has a stake in the issue or who cares about seniors’
independent and healthy living in the community?
The program team identified the following key stakeholders:
The program team
Agency’s Senior management and the programs’ committee of the board
Funder
Potential participants/clients (Seniors)
2
Step 1
After identifying the key stakeholders, the team reviewed the background information and
began the program development and planning process by discussing, agreeing on and
getting specific about long-term goals for the program. The team also clarified program
rationale; assumptions about the program; its strategies; individuals that will be served;
and what will change as a result.
1. The team agreed that the program’s main focus will be to ensure seniors are
physically active and are less isolated. This will then contribute to falls prevention
as well as independent and healthy life in the community. Answering explicitly the
program’s purpose helped the team arrive at the same understanding of what the
program is trying to change.
2. The team revisited the question of why the agency is doing this work and what
causes the issue by reviewing the agency’s annual community needs assessment
and the funding proposal to further understand the issue and its causes. The team
thought this is an issue that is affecting not just the seniors, but their families, the
community and the broader healthcare and other social systems. The team
thought:
Lack of appropriate and adequate community based support further isolates seniors
and affects their ability to be safe at home and to live independent and healthy life in
the community.
In addition, there has been more focus on institutional long-term care and creating a
network of community support for seniors is only gaining new ground. Some of the
programs that provide support to seniors are not coordinating their services.
3. The team ensured that the program is clear and agreed that those who are affected
by the issue are seniors from diverse backgrounds, male and female, but the
program team agreed the program will focus on seniors between the ages of 70 to
75 years.
4. The team revisited the issue and the clients involved; reviewed what was learned
from needs assessment; day-to-day experience engaging with marginalized youth
and other community members; and the report on aging in community, released by
the local university.
3
Step 2
After the team agreed on what the program will be addressing or changing, they used the
following questions to help them with the development of a logic model for the seniors
support program.
1. The team needed to be specific in terms of the program’s target group that will
benefit from the program. It was clear that the program cannot support all seniors in
the community. Therefore, the team agreed based on their needs assessment and
other information to focus on seniors between the ages of 70 to 75 years.
2. The team discussed how long the duration of the program will be; whether or not the
program will be ongoing for the duration of the funding or implemented differently.
The team agreed that the program would be ongoing for the duration of the funding.
3. The team discussed how the goal of the program could be achieved and started to
visualize the steps that would bring about the long-term goal or the ultimate
outcome and intermediate and immediate outcomes that will contribute to the
ultimate outcome. The team also discussed what they expect to change with the
program and for which outcomes they want to be held accountable. The team then
identified the following outcomes:
Ultimate Outcome:
Seniors maintain healthy and independent life in the community (e.g. live at home or
in community with community support)
Intermediate Outcome:
Improved safety at home (e.g. reduced incidence of hospitalization resulting from
falling at home)
Immediate Outcomes:
1. Seniors become physically active.
2. Seniors establish trusting relationships with ‘friendly visitor’ and others in the
community (seniors are less isolated).
3. Seniors learn practical skills on falls prevention
The team made assumptions about the immediate and intermediate outcomes as
necessary to achieving the ultimate outcome based on research/community needs
assessment, experience and/or ongoing interactions with seniors and their families. In
other words, the team agreed that for the ultimate outcome to occur, the immediate and
intermediate outcomes must be achievable:
Assumptions (Ultimate Outcome)
1. Seniors are interested and prefer to live in the community for as long as possible
2. There is a genuine interest among community stakeholders to advocate for aging in
the community
4
Assumptions (Intermediate & Immediate Outcomes)
1. Seniors are interested in ‘trusting relationship’ with others
2. Seniors are willing and interested to learn new skills
3. Seniors are interested in physical activity
The team is aware of the importance of documenting assumptions and justifications
moving forward and agreed to continue to ask themselves questions when they add and
change outcomes and explain how and why they are necessary.
4. The team identified the following strategies/activities to achieve the program’s
intended outcomes:
Strategies/Activities
Outreach strategy is necessary to promote the program
Screening participants to track participants’ readiness and profile
Set up support systems to facilitate participation
Conduct training sessions (falls prevention learning sessions)
Organize and facilitate physical exercise (i.e. certified trainer)
Recruit ‘friendly visitors’
Some Outputs
Seniors attend falls prevention learning session, physical exercise with a certified
trainer
Seniors are matched with a ‘friendly visitor’
Identifying Indicators
After identifying and agreed on the outcomes and strategies, the team started to identify
the indicators that will tell them the occurrence of the outcomes and implementation of the
strategies. The team noted the indicators stage is when details are added to the program
theory. This stage focuses on how to measure the implementation and effectiveness of the
program. By collecting data on each outcome, the initiative can identify what is or isn’t
happening and find out why.
To identify indicators the team asked the following:
Who is changing? (Seniors attending the program)
How many do we expect to benefit from the program? (May be 70% of Youth attending
program)
How much is reasonable? (50% of those who attend attain intended outcomes)
What is the minimum for success: (20% of those who attend maintain healthy and
independent life in the community after the end of the program)
The team noted that the indicator is what is being measured, such as learning, physical
activity and relationships; target population is the group being measured, such as program
participants (Seniors); minimum for success is the minimum for the outcome to be
considered a success.
5
Indicators for Seniors Support Program:
Ultimate Outcome Seniors maintain healthy and independent life in the community (e.g.
live at home or in community with community support)
Indicator Physical health, Confidence and Function
Population Program participants (Seniors)
Minimum 20% maintain healthy and independent life by the end of 3rd year
achievement
Intermediate Improved safety at home
Outcome
Indicator Incidence of hospitalization resulting from falling at home
Population Program participants (Seniors)
Minimum 5% reduction in incidence of hospitalization due to falling at home
achievement
Immediate Outcome 1. Seniors become physically active. 2. Seniors establish trusting
relationship with ‘friendly visitors’ and others in the community. 3.
Seniors learn practical skills on falls prevention
Indicator 1. Physical activity 2. Relationship building 3. Skills (on falls
prevention)
Population Program participants (Seniors)
Minimum 50% of seniors who attend learn practical falls prevention skills, build
achievement relationship, physically active
Program Expectation Seniors attend program and participate in activities
Indicator Attendance & Participation
Population Program participants (Seniors)
Minimum Seniors attend at least 70% of all activities
achievement
Strategies/Activities
a) Outreach strategy to promote program as the seniors may not be able to hear about
the program without an outreach strategy. The outreach strategies are to be led by
Program staff/volunteers/students and are responsible for how well it is designed
and implemented.
b) Outreach strategy needs to be measured to evaluate the strategies’ effectiveness.
c) To achieve its implementation objectives and outcomes for participants, seniors who
are attending the program; the program team and staff members of collaborating
organizations must work together and ensure that they are doing the job right.
5. The team identified the following resources to implement the program:
Staff and volunteer/student time
Space for the program/gym
6
Certified trainer (for physical activities)
Support systems to facilitate seniors’ participation (e.g. transportation)
Friendly visitors
Supplies and equipment for outdoor leadership
6. The team discussed some of the external factors that could influence the program’s
success and identified the following:
Due to the nature of the client group program participation may be
inconsistent and can affect the success of the program
Current economic situation may increase the demand for the program and
possibilities of shift in focus may occur
7. The team ensured that the intended outcomes of the program respond to the overall
mission or goal of the agency. The agency’s mandate is to support seniors to
maintain healthy and independent life in the community for as long as possible.
This program intends to support seniors between the ages of 70 to 75 years to learn
practical falls prevention skills; build trusting relationship with ‘friendly visitors’ and
others in the community; and become physically active. The team thinks, this will
contribute to reduced incidence of falling at home and maintaining healthy and
independent life in the community.
8. Finally, the team reflected on the planning process and checked if the resources
they have for the program; the strategies/activities identified; the nature of clients’
issues; and the issues the program is trying to address are realistic and meaningful
to achieve the intended outcomes. Also, checked whether the outcomes the team
identified seem realistic and meaningful? The team agreed that the program
outcomes, strategies/activities and resources are feasible given their understanding
of clients they are intending to support. However, they agreed to continue to check
on their assumptions with seniors who will be attending the program.
7
Logic model for the Seniors Support Program.
Support seniors to maintain healthy and independent life in the Seniors Timeline
community. Support 3 years
Program
Input Activities Outputs Short-term Intermediate Long-term
Outcomes Outcomes Outcomes
-Staff & -Outreach -# of participants -Seniors become -Improved -Seniors maintain
volunteer -Learning attending physically active safety at home healthy and
time sessions -# learning -Seniors build independent life
-Certified -Friendly sessions trusting in the community
trainer visitor -# of physical relationships with
-Program matching activities ‘friendly visitor’ ad
space/gym -Organizing -# of friendly others in the
-Support support visitors recruited community
systems systems -# of clients and -Seniors learn
-Friendly -Conducting friendly visitors practical skills on
visitors physical satisfied falls prevention
-Supplies & activities
equipment
Evaluations
8
Step 3
After the team has completed mapping out the program plan and agreed on a common
understanding of how the program will work, the team started to prepare the ground for the
development of the program’s evaluation plan by reflecting on the following questions:
1. First, the team identified some broad evaluation questions they thought the program
should address. The broad evaluation questions include:
Are the resources allocated sufficient to operate the program?
To what extent has the program met its implementation objectives?
What is the overall impact of the program (how many participants took on
leadership roles)?
Are clients and other stakeholders satisfied?
Have program participants attained the intended outcomes (what did
participants learn)?
What was the quality and quantity of the services provided?
Did participants receive what was promised in our program brochure/flyer
2. The team also clarified who the evaluation is for and how it will be used. The
evaluation is both for internal and external audiences.
Ongoing data will be shared and discussed during team meetings with agency
staff/volunteers/students
The program committee of the board will receive regular updates
Agency board will receive quarterly progress report
The Funder will receive bi-annual progress report
Participants will also be engaged and progress will be shared with them through the
monthly bulletin board
3. The team also discussed and identified evaluation committee that will be involved
with the ongoing evaluation process, which includes facilitating and coordinating the
evaluation process. The evaluation committee will consist of the program team and
possibly past program participant and a member of the board’s program committee.
The evaluation committee will ensure everyone is informed about the program’s
progress in a timely manner.
4. The team identified how the evaluation will be used and reiterated the importance of
ensuring what is learned will be shared and used to make decisions. The team
agreed the following to be the purpose of evaluating this program:
To assess the overall impact of the program on participants
To assess the quality and quantity of service provided
To assess the extent to which the program has met its implementation objective
To assess the attainment of outcomes for participants
5. The team discussed some of the potential implications of evaluating this program
and identified the following:
The program may become extremely successful and may expand as the
9
result posing challenges of having to hire new personnel to meet the needs
Due to the nature of the client group the program serves, the program may
experience ongoing changes to outcomes and this can frustrate
staff/volunteers and may affect their morale
Program may not become successful and the learnings may suggest a shift
in focus or termination of the program which can affect staff/volunteers and
clients.
10
Step 4
After the team has prepared the ground for the development of our evaluation plan, they
turned their attention to focusing and prioritizing what will be evaluated by reflecting on the
following questions and developing the program’s evaluation plan:
1. After engaging key stakeholders on what evaluation questions will the team focus
on as well as assessing the resources the program has to collect, track, monitor
data, the team prioritized on the following key evaluation questions to be addressed
in year one:
What has been the overall impact of the program?
Did program participants attain the intended outcomes (what did participants
learn)?
To what extent has the program met its implementation objectives?
Are clients and other stakeholders satisfied?
2. The team agreed to track and measure the following indicators:
Attendance & Participation
Quality and quantity of service provided
Physical activity, falls prevention skills, relationship building,
Safety at home
Independent and healthy living in the community
3. The team had a lengthy discussion about data sources; how data will be collected;
and when and who will be collecting data. The team identified data sources to be
clients/participants, ‘friendly visitors’, program staff, certified trainer, family
members/caregivers. The data collection methods will include staff observation,
program records, and informal interviews and focus groups. The team also
identified staff/volunteers/students who are facilitating the program will collect data
and a member of the board program committee will also be involved in data
collection. Data will be collected at various intervals in the life of the program.
4. The team agreed to ensure data will be shared. Data will be analyzed as it is
collected and mechanisms, such as team/committee meetings, board meetings will
be used to share results of ongoing evaluation.
11
This is what the evaluation plan for the drop-in program look like.
Evaluation Indicators Data Data Who When to Reporting
Questions source collection collects collect data
method data
To what extent # of seniors Program Program Program Ongoing Team
has the program served team record team
met its # of falls Board
implementation prevention committee
objective? classes
offered
# of support
offered
# of ‘friendly
visitors’
recruited
Are participants -Attendance Participants Informal Program Ongoing Board
satisfied? & participant team committee
participation ‘Friendly interview
-Quality & visitors’ Team
quantity of observation
service
Did participant -Skills (on Participants Informal Program End of year Board
attain intended falls participant team one committee
outcomes? prevention) Program interview
-Physical team A member Team
activity Observation of the
-Relationship board’s Funder
building (with Focus group program
‘friendly committee
visitor’ and
others)
What is the -Physical Participants Interview Program End of year Board
overall impact of health team two report
the program on -Independent Caregivers
participants? living Funder
final report
12
Step 5
After the team has developed the program’s evaluation plan, and identified appropriate
data collection methods, the team discussed about the development of data collection
questions.
1. The team reviewed the key evaluation questions they have identified during the
process of developing their evaluation plan. The key evaluation questions the team
chose as a priority are the following:
What is the overall impact of the program on participants?
Did program participants attain intended outcomes?
To what extent has the program met its implementation objectives?
Are clients and other stakeholders satisfied?
2. The team explored together how and where they would start to develop data
collection questions. The team quickly reviewed the tools they decided to use to
gather data. They were, staff observation, program records, focus group and
informal interviews. The team reiterated that the questions they develop need to
speak to the key evaluation questions they identified in their evaluation plan. The
team agreed to review questions from similar programs, questions the agency used
for a similar program in the past and conduct internet search for sample questions.
The team also agreed to draft some questions.
3. After reviewing questions from other programs, conducting internet search and
drafting questions, the team identified a list of questions that will address key
evaluation questions identified in the evaluation plan. The list included the following
questions that the team agreed to further review:
What has been the overall impact of the program?
1. In what ways has the program impacted your ability to live independently?
2. Did the program help you establish trusting relationships with ‘friendly visitor’ and
others in the community? How so?
3. Did the program help you to function well on your own? If yes, please give examples
4. What difference did you see in the overall improvement of this senior’s physical
health? Please give some examples (question to caregivers/care providers)?
Did program participants attain the intended outcomes (what participants learning)?
1. What did you learn from participating in the program?
2. What specific things did you learn about falls prevention? Did you use what you
learned at home?
3. In what way has the program helped you build relationships with ‘friendly visitor’ and
others in the community? Please give examples.
4. Please tell us how often you repeat the physical activities you learned at home?
To what extent has the program met its implementation objectives?
1. Did program reach the appropriate target group? (internal process)
13
2. How did you learn about the program?
3. Was the program marketed or promoted properly? (internal process)
4. Did program implement the activities as planned? (internal process)
5. Did you receive the support that was promised in program brochure/flyer?
Are clients and other stakeholders satisfied?
1. How knowledgeable are staff members about the issues you face?
2. Did the program meet your needs? How so?
3. How would you rate the quality, quantity of support you received?
4. How would you rate the program overall?
14
Step 6
After the team has gone through the program and evaluation planning process,
implementation of the program has began and the team used the following questions to
help them learn from the data they started gathering.
1. As program progressed, the team discussed about what they are learning about the
scope and reach of the program. They continued to monitor if, the program is
reaching the right target group by asking participants informally and gathering
information about participants’ profile. The team started to learn about their decision
to provide support, such as transportation to facilitate participation given
participants’ needs. The team found that many participants needed transportation
support to participate and learned about resource implication. The team also
learned about how the transportation support reduced participants’ anxiety TCC
waiting time. The team had lengthy discussion about how to continue with limited
resources for facilitating transportation support for participants.
2. The team also inquired about what they are learning about program implementation
and asked specific questions about outreach strategies, promotion of the program,
program delivery, program quality & quantity, as well as program’s relevance and
sensitivity to participants’ needs. Through staff observation and informal
client/participant interview, the team started to learn about program quality and
quantity, relevance and sensitivity and delivery. The team made relevant changes
to program where possible in response to learnings from data gathered.
3. The team also inquired about what they are learning about their internal capacity to
deliver the program, such as sufficiency of resources, staff/volunteer capacity, and
level and sensitivity of engagement with clients/participants. The team quickly
learned that program staff/volunteers had high level of commitment, capacity and
intent to serve clients/participants, however they noted that staff/volunteer need to
be systematic about documenting learnings.
4. The team also inquired about what they are learning about what is changing for
clients/participants and continued to track participants’ physical activity, relationship
building, knowledge (on falls prevention). The team quickly learned that tracking
participants’ physical activity is relatively easier, but tracking relationship building
and knowledge (on falls prevention topic) is challenging, but not impossible. As the
team tracked these through observation and informal interviews and focus group,
they started to see some progress.
5. The team also found the learnings emerging from the ongoing process of asking
questions about the program, participants and the environment under which the
program was operating very helpful as well as ongoing reflection on the following:
Making sense of the data they are collecting as they go and making course
corrections as needed
Monitoring and tracking trends and patterns as they emerge
15
Timely sharing of progress with stakeholders (e.g. using team meetings to
share and discuss data)
Tracking benefits for participants throughout the program
Checking contribution of outcomes to long-term agency goals
16
Step 7
The team made a concerted effort to making sense of data at various stages of the
program and documented learnings along the way. The team discussed data collected
through staff observation, informal interviews and the focus group at the end of year one is
telling them. To make sense of the overall data gathered through focus groups and
ongoing staff observation, informal interviews, the team scheduled three 2 hour meetings
to go through the data and analyze it. After analyzing data, the team reflected on what the
data is telling them; what major learnings are emerging; and what it all means. The team
learned the following in year one:
Providing transportation has facilitated great participation and was beneficial, but
has resource implications for the program
The program has reached the intended population, but the program may have
served some participants who did not fit the criteria (e.g. age). Further review of the
criteria is needed.
The number of participants has been great and the level of participation was very
high
Commitment and good intention to serve clients has been great by all
Staff/volunteers involved, however, systematic documentation of learnings needs to
be improved
Majority of participants have achieved the intended outcomes
More will be learned as further data on safety at home and independent living is
gathered and analyzed
The team shared learnings from the evaluation in a team meeting and made presentation
to the board’s program committee. For each of the learnings, a set of recommendations
were made for decision to be made.
17
Step 8
As the program continues, the team agreed to use the same process to engage
stakeholders in the ongoing evaluation of the program to ensure learnings will be used to
improve the program and thereby improving participants’ safety at home and maintain
healthy and independent life in the community.
Finally, the team reflected on the evaluation process (i.e. program and evaluation planning
process, and implemented the program). The team used the following questions to reflect
and think about the various stakeholder perspectives around evaluation processes and
what the evaluation process means to the various stakeholders.
1. What did we learn about the evaluation process (i.e. program development,
planning and implementation)?
2. Can we think of some benefits and challenges of the evaluation process as a tool for
engaging stakeholders?
3. Can we think of some benefits and challenges of the evaluation process as a tool for
capacity building among stakeholders?
4. Can we think of some of the implications of evaluation process for our agency?
5. As we continue to engage stakeholders in identifying issues, setting program goals,
identify evaluation priorities, develop evaluation tools, implement program and
collect data at various stages of the program, what challenges and opportunities
come to mind?
6. How do we build a culture of reflective practice across our agency?
18
