SUPPORT FOR SUCCESS ACTION RESEARCH PROJECTS
Organisation Name
Trinity Training Services Limited
Title of Project Measuring the impact of target setting on the progress of work-based learners Key Words
MIS, Student Tracking, Learner Review, Work Based Learning.
Summary of Project
The aim of the project was to measure the impact of target setting on the progress of work-based learners. The focus of the project was based around the learner review process and the use of management information systems to increase awareness of targets amongst staff. A sample of learners’ reviews was looked at to check for SMART, challenging targets and this information was compared to that held on the Management Information System (MIS). Learners were then interviewed and staff answered questionnaires as to their views on the effectiveness of the review process. Improvements were recommended and taken on board.
The main outcomes of the project were
Improved MIS, containing full and accurate system records of NVQ unit targets and achievements for each learner. This enabled production of accurate reports for use at staff and team meetings ensuring all staff were aware of learner targets;
Improved learner review process, including issue of review documentation to the 1
Lesley Bridgewood
learner and employer prior to the review date. This would enable a fuller contribution at learner reviews and, with the learner’s agreement, SMART objectives would also be recorded. Clear actions and responsibilities will be recorded on reviews when learner targets are not met.
Staff fully trained in SMART target setting, thus enabling learners to move swiftly towards unit/qualification achievement.
The research was shown to ALI inspectors during a recent monitoring visit who considered it an aid to address two areas of weakness found at inspection 1. Insufficient use of MIS
2. Ineffective target setting
Context
Trinity Training Services Limited is a charity formed in 1986 by the amalgamation of three voluntary organisations whose prime aim was to support youth training in disadvantaged wards of Wolverhampton. In 2001, the organisation moved to city centre premises to ensure more accessibility for learners.
The organisation offers both pre-vocational training and work-based learning to young people throughout the Black Country and surrounding areas. The organisation currently has a high percentage of learners progressing to employment or further work-based learning.
Trinity Training currently offer work-based learning programmes in the following occupational areas: Care of the Elderly; Early Years & Education; Catering & Hospitality; Business Administration; Distribution & Warehousing; and Customer Service.
The table below shows the current numbers of learners on each programme:
Lesley Bridgewood
2
Programme Name Care of the Elderly Early Years & Education Catering & Hospitality Business Administration Distribution & Warehousing Customer Service Entry to Employment
No. Of Learners 32 9 3 6 2 2 57
Aims and Objectives
Issues: Some learners were not achieving their qualifications within the given timescale. There was insufficient use of MIS. Learners were not being set challenging targets Not all staff had been trained in setting SMART targets and completion of effective reviews.
Aims: To measure outcomes of staff development on target setting, learner review and learner progress. To ensure staff were fully trained in setting SMART, challenging targets during learners’ reviews. To improve the effectiveness of the learner review process. To use MIS data to ensure staff awareness of targets set for learners.
Objectives: By the end of the project the company wished to have achieved the following:
Reviewed staff development and implemented training on target setting where 3
Lesley Bridgewood
necessary. Identified a selection of learners and checked their files to identify weaknesses in target setting and review. Reviewed staff practices to identify any areas for concern. Standardised review practices amongst staff. Identified any barriers learners experienced which prevented target achievement and then work to overcome them Used MIS data to track impact following intervention.
Strategies
The Project Leader started the research project by forming a project team and then clarified its terms of reference, roles and responsibilities.
Initial research
Initial research included a review of all staff development logs to ascertain which staff had received training in the use of SMART objectives to ensure completion of effective learner reviews.
Staff development
From the staff development logs it was clear that not all staff had received training in setting SMART targets. Whilst training had initially been mandatory, there were new members of staff who had joined since the training so had not been included. Staff training for these was arranged and took place in October 2004. The project team noted that the quality of the reviews improved immediately following target setting training received by staff.
Sampling learner reviews
The project team then sampled 20 learner reviews for evidence of quality, SMART target setting and achievement. The sample included learners who were recent starts, midpoint
Lesley Bridgewood
4
learners and some who had been on programme for some time and were out of funding. The data collected from this sample is recorded in the table below.
Evidence of SMART Targets recorded Yes No No Targets Recorded Number of Learners 9 10 1
The data held on the reviews was then compared to that held on MIS. The data collected from this sample of 20 learners is provided in the table below:
MIS Record Full Number of Learners 8 Partial 6 None 6
Analysis of existing data on MIS
The project leader analysed the data recorded on MIS and found that most of the unit targets/unit achievements recorded had been put in since June 2004, coinciding with the appointment of a new Provider Integrated Computer System (PICS) administrator. Following this activity the organisation spent time to make sure that every learner had all unit targets and all unit achievement recorded on MIS.
The PICS administrator was now able to access a diary report monthly, which showed units achieved and units outstanding for each learner during the period. A
recommendation was made to management to issue the report at the monthly staff meeting for managers so they could take these to team meetings to address non-achievement of learner targets. It was felt this would develop a constant awareness among co-ordinators and keep learners on track.
Review process
Lesley Bridgewood
5
The next step of the research was to ascertain staff and learners’ views on the review process. Questionnaires were devised to find out how effective staff and learners felt the review process was in aiding learners’ achievement.
Staff questionnaire
Questionnaires were issued to seven co-ordinators. The following questions are a sample of those asked –
When setting targets with learners, what information do you take into consideration? Are learners given a copy of a review form prior to the set time to enable them to reflect and contribute fully? Who do you involve in the target setting/review process? Is unit target/achievement recorded on MIS (PICS)?
Surprisingly, answers were varied and some co-ordinators took more time and consideration when answering the questions. Most felt that learners/employers should be given a copy of the review documentation prior to interview and this has resulted in a recommendation to management for a change of procedure. Also, not all staff questioned realised that NVQ unit target/achievement was recorded on MIS; this has now been reinforced at a recent staff meeting.
There were no further concerns with responses from co-ordinators.
Learner questionnaire
Questionnaires were issued to 20 learners. The following questions are a sample of those asked –
1. How much notice do you have to prepare and organise yourself for your review? 2. Do you feel confident to contribute at your review?
Lesley Bridgewood
6
3. Do you know what information to prepare ready for your review? 4. Do you always agree to the targets set for you? 5. If targets are set for you are you given the opportunity to discuss them? 6. If you are unable to meet the targets set in your review what action is taken? 7. Do you think the review process helps you in any way to work towards your qualification? Members of the project team were issued with the questionnaires and learners’ names and contact details and given a timescale to return the completed documents.
Very quickly several of the team raised concerns about the quality of the answers on the questionnaires. Learners were giving one word ‘yes/no’ answers and some seemed to misinterpret the questions, while others recorded ‘not sure’. The project leader looked through the questionnaires and agreed that the answers given were not sufficient to gather any conclusions. A decision was made to abandon the idea of issuing questionnaires. It was agreed that the project leader would speak to each selected learner on an informal one-to-one basis when they were in-house for training, asking the same questions but in a less formal manner, giving further explanation when necessary. This exercise proved far more beneficial in gaining learners’ views. The team realised from their responses that research needed to go much deeper into the quality and use of Individual Learning Plans. They also felt that induction procedures should include the role and benefit of the review. General comments learners made in response to the above questions –
1. Learners are notified at an assessment visit that a review will be carried out at the next visit, if employer available. 2. Learners understood that they could contribute but did not always feel confident enough to do so, particularly if employer is present. 3. Most learners answered ‘no’ to this question, even though they all retain copies of all reviews. 4. Most learners interviewed agreed with targets, but one learner wasn’t sure! 5. All learners commented that they discussed their targets but some seemed unsure
Lesley Bridgewood
7
as to how the targets clearly related to the NVQ . Two learners said this had improved now that clearer targets were being set. 6. Some learners were not sure what would happen if they did not meet set targets. One learner asked if she would be ‘thrown off the course’. Others said this was discussed with their assessor and then target dates amended. 7. Only two of the learners interviewed used the review form to check on targets, while others said they used the ongoing assessment/action plan to make sure they met deadlines for work set.
The research into the effectiveness of reviews and the use of MIS for target setting has led to recommendations for change being made to management. The process will be ongoing with further evaluation carried out as part of our quarterly ‘continuous improvement evaluation’ to test the impact of the changes on learner achievement.
Recommendations
From the information gathered during the research project, the following
recommendations have been made to the management team for inclusion in organisational procedures –
Learners to be issued with blank review forms at the time of booking the review interview to enable them to consider each section and contribute fully. Learners and employers to be encouraged to prepare for the review by making notes on their review forms prior to interview. Reinforcement needed of ‘why learners are reviewed’ to all learners and employers. More detailed discussion on reviews needed during learner induction, so that all learners recognise the importance and consequences. Ensure challenging targets are set at review, detailing actions clearly on action plans so that learners can see the relation and relevance to the NVQ. Staff training on setting SMART targets to be held quarterly, again to stress the importance of effective reviews. Learners’/employers’ attention to be brought to the planned review dates recorded in the Individual Learning Plan at the start of the qualification.
Lesley Bridgewood
8
Co-ordinators to ensure that the most appropriate person from the placement participates in the review, not necessarily the employer. This could be the shift supervisor, someone who can identify progress/problems.
Outcomes
Improved MIS, with full and accurate system records of NVQ unit targets and achievements for each learner. This has resulted in accurate reports for use at staff and team meetings ensuring all staff are aware of personal and learner targets;
Improved learner review process, including issue of review documentation to learner and employer prior to review date for consideration. This has enabled both to make a fuller contribution towards SMART objectives. Clear actions and responsibilities are recorded on reviews when learner targets are not met;
Staff fully trained in SMART target setting, enabling learners to move swiftly towards unit/qualification achievement.
The research was shown to ALI inspectors during a recent monitoring visit who considered it an aid to address two areas of weakness found in the last inspection1. Insufficient use of MIS 2. Ineffective target setting
Learning Points
The research has been interesting and beneficial to carry out. The team now recognise that the quality cycle is ongoing and are prepared to test, challenge and make recommendations for improvement.
The project team realised that they wasted some time by issuing questionnaires to learners and that informal interviews in a non-threatening situation worked best for gaining their views.
Lesley Bridgewood
9
Issuing regular reports to staff on expected unit achievement has proven to be a benefit in ensuring accurate data on MIS. necessary. Staff now check, query and amend dates where
Next Steps
The research project has shown how closely the review process links to the Individual Learning Plan and the organisation should now consider updating its ILP to make it more user friendly for learners and implement staff training on effective completion.
The research project has also given an insight to the extensive management information system we have and the organisation is now storing data on all of its other learners in the database (private clients, train2gain etc.).
Contact Details Lesley Bridgewood Quality Manager Trinity Training Service 2nd Floor Bankfield House Waterloo Road Wolverhampton West Midlands WV1 E-mail: lesley.bridgewood@trinity-training.com
.
Lesley Bridgewood
10