A Return on Investment Study for an Engineering Company in Huntsville, Alabama using a Community College Web-based Training Program
February, 2007
Presented by: Dr. Denny Smith Presented to: Workforce Development Institute (WDI)
Purpose and Need
American College Testing (ACT) first established the ACT Center network concept in 2000 The Calhoun ACT Center established the first national contract in 2001 with Teledyne Brown Engineering (TBE) ACT’s Evaluation Services division conducted an ROI study in 2001 to evaluate this training for TBE. Present study to compare 2001 versus 2005
Purpose
Teledyne Brown Engineering (TBE) and the Calhoun ACT Center have undergone various changes since 2001. With these changes and the fact that all TBE employees have access to web-based training, TBE and the Calhoun ACT Center would like to reevaluate the ROI and determine the impact of offering web-based training to all employees.
Study Overview
Identify employees who had participated in an online course since 2001 pilot study Send cover letter and survey to employees Complete surveys (by participants) Collect survey responses and analyze data Calculate Return on Investment (ROI) as a percentage using net benefits divided by program costs
Demographics
Majority of participants
– – – –
Caucasian Male 30 to 59 years old Highest degree completed was Bachelor or Masters
Length of employment with TBE ranged from 20 months to 26 years
6% 1% 6% Under 20 20-29 36% 30-39 40-49 50-59 Over 60 27%
24%
Ages of Participants
1%
4% 4%
High School Diploma Associate Degree Bachelor of Science or Arts Master of Science or Arts 57% Doctor of Philosophy
34%
Highest Degrees Completed by Participants
Research Questions
Is there a return on investment for dollars spent for web-based training for TBE employees who take web-based courses through the Calhoun ACT Center? Is there a difference in ROI in the 2001 results versus the 2005 results, using identical survey questions?
Limitations and Delimitations
Limitations Applicable to TBE Applicable to time and money spent on training Self-reported data Delimitations Limited to web-based training accessed by TBE employees through Calhoun ACT Center from 2001-2005 Data is subjective and limited to participant’s ability to recall information Study length extended to four weeks due to technical difficulties
Participants
183 employees had enrolled in a web-based course through the Calhoun ACT Center since 2001 pilot study
–
–
50 individual no longer employed with TBE or contracted to TBE 50 employees never accessed a course 70 employees completed a survey 3 employees’ answers were excluded from data calculations
83 employees met inclusion criteria
– –
67 participants included in data calculations
ROI
Benefits Money saved due to the application of training
–
Average reported $1,920.73 per person Average reported 65.22 hours per person Average salary $42 per hour Average time savings reported (in dollars) $2,739.24 per person
Time saved due to the application of training
– – –
ROI
Costs Lost time
– – –
Average reported 0.96 hours per person Average salary $42 per hour Average time cost $40.32 per person Average number of courses accessed 2.13 per person Average cost per course $99 Average cost of courses $210.87 per person
Cost of the Course
–
– –
Web-based Course Usage
Average number of courses accessed = 2.13 Average number of courses completed = 1.81 Course completion rate = 85% Average time spent training = 9.96 hours Average amount of time spent training outside of regular work hours = 9 hours
Comments about Web-based Training
Likes
Accessibility Flexibility Convenience
Dislikes
Technical difficulties Course selection Course catalog Certificate Award Process
ROI
Benefits – Costs ROI = ------------------------- X 100% Costs Average net return on investment per person was 1,755.16% (due to the application of training) For every $1 TBE invested, TBE realized a net benefit of $17.55
Research Questions
Is there a difference in ROI in the 2001 results versus the 2005 results, using identical survey questions?
– –
2001 ROI was 121% 2005 ROI was 1,755.16%
Study Results
Number of participants Course completion rate (Completed / Accessed) Total hours of training taken (Web-based) Average hourly salary 67 85% (1.81 / 2.13) 9.96 hours $42
Benefits
Average amount of money saved due to application of training Average amount of time saved due to application of training Average amount of time saved due to application of training (dollars) $1,920.73 65.22 hours $2,739.24
Costs
Average number of hours of training completed during work hours Average lost time cost Average course cost per respondent 0.96 hours $40.32 $210.87
Study Results Number of participants
2001 Pilot Study 24
2005 Study 67*
Course completion rate (Accessed/Completed) Total hours of training taken (Web-based) Total hours of web-based training completed outside of regular work hours
58% (1.0)1.7 7.6 hours 6.7 hours
85%(1.81)2.13 9.96 hours 9 hours
Amount of money saved due to application of training Amount of time saved due to application of training Average hourly salary Time saved due to application of training (in dollars) Hours of training completed during regular work hours Lost time cost Average course cost per respondent
$227.00 5.9 hours $37 $218.30 0.9 hours $33.30 $168.30
$1,920.73 65.22hours $42 $2,739.24 0.96 hours $40.32 $210.87
Recommendations
Expand ROI research Develop follow-up procedures for those who sign up for training but fail to access online courses Improve process for receiving certificates of completion Improve technical issues
– –
Computer systems and accessing courseware Courseware catalog
Recommendations
Improve process for receiving certificates of completion
–
–
–
Require courseware providers to add this feature to courseware Allow employees to print certificate upon completion of course Allows the Calhoun ACT Center to verify who has completed courses
Conclusions
Employees enroll in courses but fail to complete the courses Employees favor convenience, flexibility, and self-paced nature of web-based training Technical issues need to be addressed and improved
Questions?