Get documents about "Member record books are useful tools for evaluating 4-H club programs
Document Sample
RESEARCH ARTICLE
t
Member record books are useful tools for
evaluating 4-H club programs
by Larry Forero, Katherine E. Heck, Pat Weliver,
Ramona M. Carlos, Thi Nguyen and Audra Lane
We used data from 4-H record books
to evaluate the 4-H programs in
Shasta and Trinity counties. These
books are completed annually by
youth participants throughout
California to describe and quantify
their experiences in the program
and reflect on their involvement in
citizenship, leadership and life-skills
activities. Quantitative and qualita-
tive data from the reports was coded
according to the Targeting Life Skills
model developed at Iowa State
University. Most club participants re-
ported life-skill activities in each com-
ponent of the model (Head, Hands,
Heart and Health), in accordance with
established 4-H goals. This method is
applicable to other counties wishing
to perform 4-H program evaluations
using club participants’ record books. Participants in the Shasta County 4-H program exhibit their project lambs at the Shasta District Fair.
4-H is a youth organization in
which young people are given
opportunities to build confidence, learn
documenting participation in a variety
of 4-H projects and activities, such as
the number of meetings and events
son that they have rarely been used in
evaluation.
Little has been published regarding
responsibility and develop life skills. attended, projects completed, presenta- the use of record books in 4-H program
Youth make friends and share interests, tions given and awards received. The evaluation. Diem and Devitt (2003) ex-
ranging from building robots to raising record books include project pages and amined 89 record books of New Jersey
rabbits, from designing Web pages to a personal narrative written by the 4-H members and reported on what
landscape design; and they undertake member describing his or her 4-H par- youth recorded learning (such as goal-
volunteer projects in their communities. ticipation. This open-ended narrative is setting, subject matter knowledge, orga-
The 4-H programs in many states intended to describe the totality of one’s nization and public speaking). Previous
use record books as a tool to teach history and experiences in the program 4-H evaluations have primarily used
youth about record keeping, improve in a sequential fashion. survey data for program evaluation,
their projects and reflect on their These record books represent a large rather than record books (Astroth and
achievements. The record book format commitment of time by youth and adult Haynes 2002; Howard et al. 2001; Lerner
varies by state, and in most cases it is volunteers. As a collection of personal et al. 2008; Seevers and Dormody 1995).
a personal description and reflection information describing program expe- The goal of this project was to use
of their own experiences rather than a riences, the record books also provide available record book data to evaluate
public record. a potential data source for program how well the 4-H programs in Shasta
Along with personal data such as evaluation that has been heretofore and Trinity counties were achieving the
age, grade level and years in the 4-H untapped in California and elsewhere. statewide 4-H goals and mission, “to
program, the California record book Since the record books contain personal engage youth in reaching their fullest
includes a personal development re- reflections, most are not shared beyond potential while advancing the field of
port (PDR) with a quantitative page the club level, which may be the rea- youth development” (www.ca4h.org).
http://californiaagriculture.ucanr.org • OctOBER–DEcEMBER 2009 215
TABLE 1. Personal development report (PDR) categories coded to Targeting Life Skills model items,
2005–2006 (n = 341) This mission is lofty but challenging as
the basis for evaluation at the county
PDR category Youth reporting Targeting Life Skills model items level. It is difficult to determine whether
% youth have reached their “fullest” poten-
Local club meetings 71.8 (No skills coded) tial, and advancing the field of youth de-
attended
Project meetings attended 71.2 (No skills coded)
velopment is an important mission but
4-H camp (planned) 5.6 Planning/organizing, teamwork unrelated to members’ personal reflec-
Field days: club, county, 54.6 Marketable skills, self-motivation, self-responsibility, tions. To facilitate the evaluation, a more
region, state keeping records, planning/organizing, goal-setting concrete set of goals was needed.
State leadership conference 0.8 Communication, leadership, sharing, teamwork, self-
For many years, the 4-H program
esteem, planning/organizing
Committee chair 27.6 Communication, cooperation, social skills, nurturing has focused on the development of citi-
relationships, leadership, responsible citizenship, zenship, leadership and life skills. The
contribution to group effort, marketable skills, teamwork, specific life skills cultivated were iden-
self-motivation, planning/organizing, problem solving,
tified in the Targeting Life Skills model
decision making, self-esteem, self-responsibility
Judging contest 54.8 Communication, keeping records, critical thinking, developed by Patricia Hendricks of
decision making Iowa State University (Hendricks 1996).
Wrote and submitted 9.1 Communication, leadership, marketable skills, self- The model, also known as the Iowa
news-club paper motivation, planning/organizing, critical thinking,
wheel, includes 35 skills related to
learning to learn, self-discipline
Represented 4-H in 60.7 Contribution to group effort, responsible citizenship
“Heart” (relating to and caring about
other way others), “Head” (managing and think-
Committee member 49.6 Communication, cooperation, social skills, sharing, ing), “Hands” (giving and working)
nurturing relationships, contributions to group effort, and “Health” (living and being) (fig.
marketable skills, teamwork, planning/organizing, goal-
setting, problem solving 1). Life skills included in this model
Junior or teen leadership 16.7 Communication, cooperation, sharing, leadership, represent a diverse range of charac-
contributions to group effort, self-motivation, planning/ teristics, from personal qualities such
organizing, goal-setting, problem solving, decision as empathy, self-motivation and resil-
making, self-esteem, social skills
Prepared and gave talk 32.3 Communication, marketable skills, keeping records,
iency, to specific skills such as plan-
planning/organizing, critical thinking, learning to learn, ning and organizing, problem solving
self-discipline, sharing and keeping records.
Held an office 27.9 Communication, social skills, nurturing relationships, These 35 life skills were used as the
leadership, responsible citizenship, contributions to group
effort, marketable skills, teamwork, keeping records,
basis for our evaluation of quantitative
planning/organizing, goal-setting, decision making, self- components in record book data from
esteem, self-motivation Shasta and Trinity counties describing
Radio or tV appearance 2.9 Communication, leadership, marketable skills the types of activities in which young
Medalist 27.0 Self-motivation, self-responsibility, planning/organizing,
goal-setting, sharing, marketable skills
people participated. Qualitative narra-
Project exhibit 73.9 Sharing, marketable skills, self-motivation, planning/ tive data describing personal program
organizing, self-esteem experiences was coded and analyzed
Participation other than 0.0 (No skills coded) according to a statement from the 4-H
4-H
Web site summarizing the 4-H experi-
Demonstration 55.1 Communication, sharing, self-motivation, planning/
organizing, critical thinking, self-esteem, self-discipline, ence: “4-H enables youth to have fun,
social skills meet new people, learn new life skills,
Number of projects 87.4* Self-motivation, keeping records, goal-setting, self- build self-confidence, learn responsibil-
completed responsibility, self-discipline
ity and set and achieve goals!”
Planned group activity 38.7 Communication, cooperation, social skills, concern for
others, sharing, nurturing relationships, leadership, Questions we sought to answer with
contributions to group effort, marketable skills, this project included: (1) are record
teamwork, self-motivation, planning/organizing, goal- books usable as a data source for pro-
setting, problem solving, decision making, self-discipline
gram evaluation? and (2) how well is
Attended event 0.0 (No skills coded)
County winner 0.0 Self-motivation, self-responsibility, planning/organizing,
the 4-H program helping young people
goal-setting, sharing, marketable skills to develop citizenship, leadership and
Other: individual 48.7 Self-motivation life skills? To answer the first question,
achievement we had to examine the record books
Other: group achievement 31.7 (No skills coded)
available to see whether they contained
Community pride service 70.4 Cooperation, concern for others, empathy, sharing,
nurturing relationships, community-service volunteering, the kind of information that could be
responsible citizenship, contributions to group effort, used to provide data on the success of
teamwork, character the program. Once the first question
Project showing contest 68.6 Communication, sharing, planning/organizing, self-
was answered affirmatively, we could
responsibility
analyze the data to learn about young
* Percentage who completed at least one project.
people’s experiences in 4-H.
216 CALIFORNIA AGRICULTURE • VOLUME 63, NUMBER 4
Coding record book data
Record books were submitted to
the 4-H youth development program
representatives in Shasta and Trinity
counties on a volunteer basis, with the
knowledge that they would be used
for this project. We used data from the
2005–2006 4-H year (4-H runs from July
1 to June 30, with most youth enrolling
at the beginning of the school year).
From 919 members who participated,
341 record books were collected, 330 of
which included narratives. The sample
represented 37.1% of all youth in the
4-H programs in Shasta and Trinity
counties, and the narrative data rep-
resented 35.9% of all participants. We
used two components of the record
books for evaluation, the personal
development report (PDR) and the per-
sonal narrative, “My 4-H Story.”
Personal development report. The
PDR includes tables that are filled in Fig. 1. Targeting Life Skills model. Source: Hendricks 1996.
to quantify participation in 26 types of
activities or honors, for the current year conflict resolution, resiliency, wise use their animals, meeting people at places
and previous years. For this analysis, of resources, service learning, manag- where youth volunteer, and making new
we used the data simply to indicate ing feelings, healthy lifestyle choices, friends through program participation.
whether the youth had participated stress management, disease prevention “Learning new life skills” included learn-
in the activity or not, rather than to and personal safety) may be developed ing any of the 35 life skills listed in the
determine how many times they had through a variety of 4-H activities, but Targeting Life Skills model. Youth who
participated (i.e., as a binary rather than were not coded because the PDR catego- reported increased self-confidence as a re-
a continuous model). ries were not adequate to definitively say sult of their participation in the program
We compared the activities listed in that the young person had developed were coded as having developed “self-
the PDR to the life skills described in the these skills. For example, a young person confidence.” “Responsibility” included
Targeting Life Skills model and identi- participating in a rocketry project may learning responsibility or becoming more
fied which life skills each PDR item develop personal safety skills. However, responsible through 4-H participation.
would help develop (table 1). The indi- no PDR item is exclusive to such a proj- “Set/achieve goals” included youth who
vidual PDR items were coded with more ect, so it was not possible to say using reported that they had set one or more
than one of the Targeting Life Skills the PDR data how many young people goals during that 4-H year or reported
model items in most cases. For example, developed personal safety. achieving one or more goals.
having a radio or television appearance Personal narrative. To examine Statistical analysis. The narratives
was coded as developing communica- qualitative experiences in the program, were hand-entered into Microsoft
tion, leadership and marketable skills. the personal narratives were coded ac- Word and then coded using Excel. Data
Not all of the Targeting Life Skills cording to the Web site statement of were analyzed using SAS Institute
items relate to the categories defined in 4-H goals. After the research group software for quantitative and NVivo
the PDR. Therefore, not all of the skills reviewed and agreed upon the codes, software (QSR International) for quali-
could be identified and coded; of the three researchers individually coded tative data analysis.
35, 25 were coded to one PDR category the youth narratives. To ensure consis- Demographics. Participants ranged
or more. Conversely, 21 of the 26 PDR tency, the group discussed any items from 9 to 18 years old, with a median
items received Targeting Life Skills about which a team member was uncer- age of 13 (table 2). Most record books
model codes, while five items received tain. Data from coding the narratives came from youth who had participated
no code (participation in activities other was entered into a spreadsheet. in 4-H for fewer than 4 years, although
than 4-H, attended event, other group For example, if a narrative reported 40% had been in the program 4 years
achievement, and attending local club that the youth had fun, had a good time or longer. The majority of participants
and project meetings). or enjoyed the program, he or she was were female (57%). Eighty percent lived
Ten of the Targeting Life Skills coded as “having fun.” “Meeting new in Shasta County, while the remainder
model items (accepting differences, people” included meeting buyers for lived in Trinity County.
http://californiaagriculture.ucanr.org • OctOBER–DEcEMBER 2009 217
TABLE 2. Demographics of 4-H study participants,
Shasta and Trinity counties, 2005–2006
Characteristic Responents
number (%)
Age
9–10 68 (20.1)
11–12 78 (23.0)
13–14 97 (28.6)
15–16 65 (19.2)
17 or older 31 (9.1)
Years in program
1 86 (25.9)
2–3 112 (33.7)
4–5 54 (16.3)
6 or more 80 (24.1)
Gender
Male 146 (42.9)
Female 194 (57.1)
County
Shasta 273 (80.1)
Trinity 68 (19.9)
TABLE 3. Youth who developed various citizenship,
leadership and life skills, according to personal
development report (PDR) data (n = 341)
Principle Members
number (%)
Information from PDRs
Developing life skills 312 (91.5) The personal development record (PDR) is filled out by 4-H members to quantify their participation
Developing citizenship 285 (83.6) in activities and honors. This data and the accompanying personal narratives were used to
Developing leadership 214 (62.8) evaluate whether the 4-H programs in Shasta and Trinity counties were meeting their goals.
Life skills developed*
HEAD agement or managing feelings, but it is
Keeping records† 341 (100.0) Can books be used for evaluation? not possible to ascertain from the PDR.
Planning/organizing 310 (90.9) The study’s first component exam- The qualitative data drawn from
Goal-setting 308 (90.3) ined whether record book data was narratives is also subject to limitations.
Decision making 281 (82.4)
Problem solving 260 (76.3)
adequate to evaluate the 4-H program. These narratives were open-ended and
Critical thinking 255 (74.8) Examination of the categories reported did not necessarily represent the totality
Learning to learn 31 (9.1) in the PDR and information reported of the young person’s experiences in the
HEART in the narratives led us to conclude that program. Since youth were not asked
Sharing 312 (91.5) the record books have data that can specific questions about their develop-
Communication 310 (90.9)
evaluate how well the program is meet- ment, or about any of the individual item
Nurturing relationships 295 (86.5)
Cooperation 294 (86.2)
ing its primary missions of engaging codes, the narrative can provide only
Social skills 287 (84.2) youth in citizenship, leadership and a baseline for members who mention
Concern for others 271 (79.5) life-skills activities, including many of particular characteristics, rather than an
Empathy 240 (70.4) the life skills identified in the Targeting accurate quantification of rates for vari-
HANDS Life Skills model. The PDR data pro- ables examined in the narratives.
Self-motivation 308 (90.3)
vides a structured look at participation
Marketable skills 305 (89.4) Evaluating 4-H program goals
Contributions to group effort 303 (88.9) in specific activities, while the narra-
Teamwork 295 (86.5) tives filled in additional qualitative PDR analysis. Youth in the 4-H
Responsible citizenship 285 (83.6) information about personal perceptions club programs in Shasta and Trinity
Community-service volunteering 240 (70.4) of program experiences. counties typically report a variety of
Leadership 214 (62.8)
These findings are not without limi- activities, most commonly related to
HEALTH
Self-responsibility 308 (90.3)
tations, however. The PDR categories community service, individual proj-
Self-discipline 304 (89.2) described specific activities rather ects and local fair participation (table
Self-esteem 292 (85.6) than the skills gained from those ac- 1). Completing at least one project was
Character 240 (70.4) tivities. Since the categories did not the most common activity noted on
* Ten life skills from the Targeting Life Skills model were not match items in the Targeting Life Skills the PDR (87%). Additional commonly
coded because PDR items did not relate to them.
model, the coding provides incomplete reported PDR items included “com-
† Since this data was drawn from record books completed by
members, all members in the analysis were considered to coverage of possible life skills gained. munity pride service” (community-
have developed the life skill of record keeping. Using only It is possible that specific projects may service projects) and “project showing
PDR codes from table 1, 90.3% of youth developed this skill.
develop skills, for example stress man- contest” and “project exhibit” (show-
218 CALIFORNIA AGRICULTURE • VOLUME 63, NUMBER 4
ing projects at fairs, judging contests they provide substantial detail about The response rate of approximately
and demonstrations). the 4-H participant’s experiences, but 37% in this case the proportion of
Analysis of the PDRs indicated that until now the record books have re- record books that were collected at
a majority of young people in the club mained an untapped data source. We the county level is an additional
program participated in leadership found that this currently existing data limitation. It is possible that the record
(63%), citizenship (84%) and life-skills source can be used for program evalu- books gathered did not constitute a
activities (92%) (table 3). Most members ation at the county level. Other coun- representative sample of all youth in
(59%) reported PDR items indicating that ties could use this project as a model the program, which could skew the
they were engaged in all three types — for their own evaluations. Coding the results. However, the record books
citizenship, leadership and life-skills ac-record books was time-consuming, collected represented members from
tivities — while 28% reported two of the particularly for the longer record books 22 different clubs, in the full range of
three and 4% reported just one. typically completed by older members, ages represented in the program as
Younger participants (below grade but once entered into the spreadsheet well as members who participated in a
6) are not expected to participate in and proofed, the PDR data was easily wide variety of activities, so it appears
leadership positions in their clubs accessible and quantified. to be a broad sample. The 4-H club
such as holding an office, so it is not Results showed that the 4-H pro- programs in Shasta and Trinity coun-
surprising that such engagement was grams in Shasta and Trinity counties ties are not representative of all youth
reported less frequently than citizen- were meeting the goals of cultivating participating in 4-H around the state,
ship or life skills. The proportion of citizenship, leadership and life skills so these results cannot be extended
members who reported leadership among most youth. One significant statewide. However, other counties
activities was higher when the analy- limitation of this study is that the data may use this method to evaluate their
ses were restricted to older members: available in this version of the PDR own programs.
does not spe-
More than 90% of members developed the cifically describe
life skills of sharing, communication, planning the skills young L. Forero is Livestock Farm Advisor, Shasta and
people gain, so
and organizing, goal-setting, self-motivation, the results are Trinity counties; K.E. Heck is Specialist, Agricul-
tural Experiment Station, UC Davis; P. Weliver
record keeping and self-responsibility. limited by the is Youth Development Program Representative,
accuracy of the Shasta and Trinity counties; R.M. Carlos is Aca-
among members in grades 7 to 12, 70% assumptions in coding the PDR to demic Coordinator, 4-H Center for Youth Devel-
reported at least one such activity. the Targeting Life Skills model. Also, opment, UC Davis; T. Nguyen is Analyst, California
Based on matching PDR data to life no data was available in the PDR on Department of Social Services, and formerly
skills delineated in the Targeting Life the amount of time youth spent on Junior Specialist, 4-H Center for Youth Develop-
ment, UC Davis; and A. Lane is Youth Develop-
Skills model, more than 90% of mem- particular activities, so duration and ment Program Representative, Trinity County.
bers developed the life skills of sharing, intensity could not be quantified.
communication, planning and organiz- While this analysis attempted to
ing, goal-setting, self-motivation, record identify the likely skills that youth
keeping and self-responsibility (table 3). were gaining from their participa- References
Narratives. In the 330 qualitative nar- tion, the life skills enumerated may Astroth KA, Haynes GW. 2002. More than cows &
cooking: Newest research shows the impact of 4-H.
ratives evaluated, most youth reported not precisely match the skills youth J Extension 40(4). www.joe.org/joe/2002august/
having fun (65%) and learning new life gained. Our coding scheme may have a6.shtml.
skills (68%). About 44% reported set- overreported or underreported the Diem KG, Devitt A. 2003. Shifting the focus of 4-H
ting or achieving goals, and 39% said resulting life skills, and the imprecise record-keeping from competition and subject mat-
ter to youth development and life skills. J Extension
they learned responsibility. About one coding of life skills is a limitation of 41(6). www.joe.org/joe/2003december/iw1.php.
in five reported meeting new people our study. The PDR items included
and 16% reported that they gained self- some specifics allowing for the coding Hendricks PA. 1996. Targeting Life Skills Model. Iowa
State University Extension. Ames, IA. www.extension.
confidence through participating in of 25 of the 35 life skills identified in iastate.edu/4H/lifeskills/homepage.html.
the program. These qualitative results the Targeting Life Skills model. The
Howard JW, Chilek KD, Boleman CT, et al. 2001. De-
represent only the characteristics or ex- record book format has changed since veloping a program evaluation instrument for Texas
periences that the members chose to de- this data was collected, improving the 4-H: A work in progress. J Extension 39(4). www.joe.
org/joe/2001august/iw4.php.
scribe, and as such represent a minimal ability to evaluate members’ life-skills
baseline. For example, it is possible that development in the future. Rather Lerner RM, Lerner JV, Phelps E, et al. 2008. The posi-
tive development of youth. The 4-H Study of Positive
more than 16% gained self-confidence, than quantifying a particular array of Youth Development: Report of the findings from the
but only 16% mentioned it in their activities, the new record book format first four waves of data collection, 2002–2006. Tufts
open-ended narratives. allows youth to identify their develop- University, Medford, MA. http://ase.tufts.edu/iaryd/
documents/4HStudyFindings2008.pdf.
ment of leadership, citizenship,
Pros and cons of coding methods community-service and communica- Seevers BS, Dormody TJ. 1995. Leadership life skills
development: Perceptions of senior 4-H youth. J Ex-
Record books are completed by most tion skills, as well as individual proj- tension 33(4). www.joe.org/joe/1995august/rb1.php.
youth in many county 4-H clubs, and ect skills gained.
http://californiaagriculture.ucanr.org • OctOBER–DEcEMBER 2009 219
Related docs
