Certificate Mechanical Technology
Educational Effectiveness Assessment Plan AY06 &AY07
Adopted by The Mechanical Technology faculty: March 31, 2007 Submitted to the Director: June 10, 2007 The Office of Academic Affairs: June 15, 2007
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 1 of 32
TABLE OF CONTENTS
Introduction To Assessment Of Student Outcomes __________________________________ 3 Points To Ponder… ___________________________________________________________ 4 Introduction to the Mechanical Technology Certificate ______________________________ 5 Program Outcomes ___________________________________________________________ 7 Assessment Tools_____________________________________________________________ 8 Assessment Implementation & Analysis for Program Improvement ____________________ 11
General Implementation Strategy _________________________________________________________ 11 Method of Data Analysis and Formulation of Recommendations for Program Improvement ___________ 11 Modification of the Assessment Plan ______________________________________________________ 11
Appendix A: Skills Check-Off list and Activity Sheets ______________________________ 12 Appendix B: Course-level Assessment ___________________________________________ 19 Appendix C: Grade Data _____________________________________________________ 20 Appendix D: Welding Certification Test Results ___________________________________ 21 Appendix E: Exit/Graduate (Alumni) Survey _____________________________________ 22 Appendix F: Employer Survey _________________________________________________ 25 Appendix G: Instructor Impression Survey _______________________________________ 29 Appendix H: Course Groupings Associated With Each Outcome _____________________ 32
Table 1: Program Outcomes Assessment Tools and Administration ................................................ 9 Table 2: Association of Assessment Tools to Program Outcomes .................................................. 10
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 2 of 32
INTRODUCTION TO ASSESSMENT OF STUDENT LEARNING
Student Outcomes Assessment is an evaluation process where the program faculty review the published program outcomes and determine the success with which students have achieved those outcomes. Student outcomes assessments normally result in recommendations by the faculty for improvements related to program content or delivery. Plans for assessing student learning outcomes are prepared by program faculty and approved by their college. Active plans and outcomes assessment reports are filed yearly with the Office of Academic Affairs. Assessment of educational effectiveness is increasingly recognized nationally as an essential component for all of higher education. In addition to its centrality to our goal of becoming a true learning organization, assessment of outcomes is also required by our accrediting commission, Northwest Association of Schools and Colleges, as well as the University of Alaska Board of Regents. The assessment of educational effectiveness and outcomes must be a part of every course [program] of study if we are to meet the expectations of our stakeholders, satisfy the requirements of our institutional and program accreditation, and be accountable to ourselves that we are providing the highest quality educational programs possible. Each academic program at UAA is to engage in a continuous improvement process. This process is to be formalized and repeated on a regular cycle and must include the essential elements of: 1. 2. 3. 4. Setting program outcomes Selecting indicators that can be used to measure progress toward achieving those outcomes Gathering and analyzing data to determine achievement Recommending and implementing program changes that facilitate further progress in achieving the desired outcomes.
A framework that reflects well established and validated practices has been adopted for the continuous improvement of each academic and administrative unit of the University of Alaska Anchorage. It has been assigned the acronym PAAR; which stands for Plan, Act, Access, and Revise.
This assessment plan document reflects the PAAR outline for the Certificate in Mechanical Technology program at the University of Alaska – Kenai Peninsula College.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 3 of 32
POINTS TO PONDER…
Everything should be made as simple as possible, but not simpler. ALBERT EINSTEIN (quoted in ‘Reader’s Digest’, Oct. 1977) Not everything that can be counted counts; and not everything that counts can be counted. ALBERT EINSTEIN (sign hanging in his office at Princeton) Measure what is measurable, and make measurable what is not so. GALILEO GALILEI (quoted by H. Weyl ‘Mathematics and the Laws of Nature’ in I. Gordon and S. Sorkin (eds.) The Armchair Science Reader (1959) The things taught in college and schools are not an education, but the means of education. RALPH WALDO EMERSON
I remember a farmer friend of mine saying that weighing a hog doesn’t fatten it up. To fatten up a hog, we need to do more than just weigh it. To increase student achievement we need to do more than just measure it. We need to nourish it, invest in it. TERRY CRANEY (Presidential Address to 2000 WEAC Convention) Start by doing what’s necessary; then do what’s possible, and suddenly you are doing the impossible. ST. FRANCIS OF ASSISI
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 4 of 32
INTRODUCTION TO THE M ECHANICAL TECHNOLOGY CERTIFICATE
This document defines the educational outcomes for the Certificate in Mechanical Technology program and outlines a plan for assessing the achievement of the stated outcomes. The mission of the Certificate in Mechanical Technology program is to prepare students for employment as entry-level mechanics or millwrights in all types of industrial plants and provide skill upgrading for existing employees, and to provide responsiveness, engagement, and leadership in vocational and technical education in Alaska. The Mechanical Technology program is committed to excellence in real-world experiences, shop and machine tool knowledge, skills, safety habits, life-long learning, and the academic growth of each student.
Mechanical technology faculty frequently meets with industry employers to discuss industry needs and trends in the area of mechanical technology. Besides the KPC certificate program courses, KPC’s mechanical technology program and faculty have provided training for Sakhalin Energy (Russia), Alpine (North Kenai and North Slope), Unocal (North Kenai), Phillips Petroleum (North Kenai), Tesoro (North Kenai), Cook Inlet Platform, Veco (Alaska), Quick Construction (North Kenai, Peak (Alaska), local machine shops and local welding contractors. Each training session was preceded by meetings to discuss current needs and KPC’s ability to meet those needs. Curriculum and equipment have been continually adjusted as needed to meet those needs. The Mechanical Technology program has provided our community and region with fundamental mechanical skills and specialized machine skills. Besides machinists, the program has helped people gain employment as millwrights, mechanics, gunsmiths, pump repair persons, instrument technicians, process operators, heavy equipment mechanics, refrigeration technicians, aircraft mechanic/machinists, diesel mechanics, watchmakers, etc.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 5 of 32
Former program students, many now in supervisory positions, often look to KPC to hire mechanical technology students. The Mechanical Technology program shares courses with the Process Technology, Industrial Process Instrumentation, and Welding Technology programs. The KPC Mechanical Technology program is unique in the State of Alaska and in the UAA system.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 6 of 32
PROGRAM OUTCOMES
The specific educational outcomes that support the program objectives are to produce graduates who are able to: Operate basic machine tools at an entry level Lathe Mill Grinder Saws Drill Press Sanders Arbor Press Radial Drill Ovens Precision Measuring Tools measure, identify and apply with real world parts and pieces, pipe, pipe schedules, fittings and related steel structural materials and produce appropriate blue prints. show proficiency in the use, calibration, repair, maintenance, and care of all precision measuring tools. complete advanced machine shop projects in a variety of materials using standard machine tools and student-created blue prints plan and complete machining jobs on the CNC (Computer Numerical Control) equipment in a variety of materials including steel, aluminum, brass, cast iron, stainless, nylon, plastics and hardwood. (optional focus) pass entry-level welding certification test. (optional focus)
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 7 of 32
ASSESSMENT TOOLS
A description of the tools used in the assessment of the program outcomes and their implementation are summarized in Table 1. The tools and their relationships to the program outcomes are listed in Table 2. There is a separate appendix for each tool that includes a more detailed description than is provided here and also describes the factors that affect the results and give examples of the tools and how they will be implemented. The assessment tools may be used in total, or a sufficient number may be selected to accurately assess any given objective or outcome. The assessment tools may be used in total, or a sufficient number may be selected to accurately assess any given objective or outcome.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 8 of 32
Table 1: Program Outcomes Assessment Tools and Administration Collection Method
Faculty
Tool
Skills Checkoff lists
Description
Machine Shop Skills check-off list for demonstrated safety and proper use of Skills
Frequency/ Start Date
Administered by
Faculty
Mech 101 each time offered
Each semester taught starting Spring 2004) Assessment of courses relative to their own outcomes. Course results are mapped to program outcomes. 3 course-level assessments will be prepared and administered in the fall of 2003. 3 courses will be added to this list each semester until all courses have this assessment tool Annually. Starting Fall 2003 (may collect data from previous years) Reports submitted by instructors or pulled from Banner reports Assessment Coordinator in conjunction with Advisors, Faculty & staff
Course-level Assessment
Grades
GPA in grouped course subject categories. Welding Certification Test Perception survey of attainment of stated program outcomes, as seen by the graduating students. Perception survey of graduates attainment of stated program outcomes, and assessment of outcomes to current industry needs, as seen by employers Instructors will evaluate outcomes for each course and interpret data from other course-level assessments
Grade reports from instructors or from Banner reports Faculty Hand delivered at KPC to current graduates -mailed for follow up
Assessment Coordinator in conjunction with Advisors, Faculty & staff Faculty Certified Tester Assessment Coordinator in conjunction with Advisors/Staff
Welding Certification Test Exit/Graduate (Alumni) Survey
Annually, on-going
Annually starting Spring 2003 for new graduates and in 2 and 5 years as a graduate follow up Every three years starting with 2002 – may survey some employers each year with the intent of not resurveying any given employer more frequently than every three years.
Employer Survey
mailed survey or personal interview
Assessment Coordinator in conjunction with Advisors, Faculty & staff
Instructor Impression
Each semester starting Spring 2004
Assessment Coordinator prepares and gives to Faculty
Faculty returns to assessment coordinator for tabulation & reporting
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 9 of 32
Table 2: Association of Assessment Tools to Program Outcomes Welding Certification Test
operate basic machine tools at an entry level show proficiency in the use, calibration, and care of all precision measuring tools complete advanced machine shop projects in a variety of materials using standard machine tools and student-created blue prints measure, identify and apply with real world parts and pieces…and produce appropriate blue prints plan and complete machining jobs on the CNC (optional focus) pass entry-level welding certification test (optional focus)
1
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
1
1
0
0 = Tool is not used to measure the associated objective. 1 = Tool is used to measure the associated objective.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 10 of 32
Instructor Impression 1 1 1 1 1 0
Skills Check-off List
Employer Survey
Exit/Graduate (Alumni) Survey
Course-Level Assessment
Grades
ASSESSMENT IMPLEMENTATION & ANALYSIS FOR PROGRAM IMPROVEMENT
General Implementation Strategy The original Educational Effectiveness Assessment Plan was implemented in 2002 by the Business and Industry Division of Kenai Peninsula College. At that time, a graduate survey and an employer survey were conducted, collected, and reported. In 2003, Kenai Peninsula College has appointed an Assessment Coordinator who will work with the staff and faculty to collect the data indicated in Tables 1 and 2. This coordinator will also provide support for course-level assessment and other assessment activities as needed. The assessment coordinator will work with staff and faculty to assemble the data and forward it to the director of KPC for final report preparation. The final report will be reviewed by the program faculty before submitting it to the UAA Academic Affairs assessment coordinator in June of 2004. Method of Data Analysis and Formulation of Recommendations for Program Improvement The faculty of the Mechanical Technology program is to meet at least once a year with the division chair and the assessment coordinator to review the data collected using the assessment tools. This meeting should result in recommendations for program changes that are designed to enhance performance relative to the program’s objectives and outcomes. The results of the data collection, an interpretation of the results, and the recommended programmatic changes are to be forwarded to the office of Academic Affairs (in the required format) by the end of May each year. A plan for implementing the recommended changes, including of advertising the changes to all the program’s stakeholders, is also to be completed at this meeting. The proposed programmatic changes may be any action or change in policy that the faculty deems as being necessary to improve performance relative to programs objectives and outcomes. Recommended changes should also consider workload (faculty, staff, and students), budgetary, facilities, and other relevant constraints. A few examples of changes made by programs at UAA include: o changes in course content, scheduling, sequencing, prerequisites, delivery methods, etc. o changes in faculty/staff assignments o changes in advising methods and requirements o addition and/or replacement of equipment o changes to facilities Modification of the Assessment Plan The faculty, after reviewing the collected data and the processes used to collect it, may decide to alter the assessment plan. Changes may be made to any component of the plan, including the objectives, outcomes, assessment tools, or any other aspect of the plan. The changes are to be approved by the faculty of the program. The modified assessment plan is to be forwarded to the dean/director’s office and the Office of Academic Affairs.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 11 of 32
APPENDIX A: SKILLS CHECK -OFF LIST AND ACTIVITY SHEETS
Tool Description: Check-list completed by the faculty as a student demonstrates proper safety and operation of basic machine tools at an entry level. (Tool added as a result of a recommendation in the 2000 program review and discussions following the 2002 Education Effectiveness Assessment Cycle.) Factors that affect the collected data: Data is collected by observation of the faculty. How to interpret the data: The data collected clearly shows demonstrated ability to safely operate specific Skills. Sample Check-list follows (faculty may revise or develop their own).
Skills Check-off list Student demonstrates proper safety and operation of the following tools. Student Name______________________________ Semester _____________ Tool □ Lathe □ Mill □ Saws (vert & horiz) □ Drill Press □ Arbor Press □ Radial Drill □ Ovens □ Precision Measuring Tools Date of observation ________________ ________________ ________________ ________________ ________________ ________________ ________________ ________________ Faculty initials ____________ ____________ ____________ ____________ ____________ ____________ ____________ ____________
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 12 of 32
Skill Sheet – SAWS (sample questions) 1. What do we mix 20:1 and use as a coolant in the horizontal band saw? _________________________________________________________ 2. What is the sliding weight on the top of the horizontal band saw for? _________________________________________________________ 3. Why do we often use the step blocks from the mills when cutting short work in the cutoff saw? _________________________________________________________ _________________________________________________________ 4. How many speeds does the cut-off saw have? _________________________________________________________ a. How many machinists does it take to change a light bulb?_____ 5. Using the CORRECT units, what is the speed range of the vertical saw? _________________________________________________________ 6. What is a reasonable allowance for clean-up on the cut-off saw for 3” diameter material? _________________________________________________________ 7. The bull horns are used to hold small or irregular shaped pieces for sawing on the table of the vertical band saw. Why? _________________________________________________________ 8. SPECIFICALLY, what would happen if you used a 3/8” wide blade with ¾” guides in the band saw? _________________________________________________________ 9. Briefly, how is the saw blade tension adjusted? a. Horizontal saw…______________________________________ ____________________________________________________ b. Vertical saw…________________________________________ ____________________________________________________ 10. What procedure (correction) is necessary for a successful cut if the teeth are facing the wrong way in the band saw?___________________________________ __________________________________________________________ (many more questions)
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 13 of 32
Skill Sheet - VERNIER CALIPER MEASUREMENT (uses the same basket of parts as the micrometer measures activity) 1. ________ Length, small roll pin 2. ________ Maximum O.D., small roll pin 3. ________ Major diameter, small washer 4. ________ Small washer, I.D. 5. ________ Thickness, small washer 6. ________ Thread maximum diameter, small screw 7. ________ Screw slot width, small screw with nibs 8. ________ Normal length, small screw 9. ________ Length, as the thread goes, small nut 10. ________ Measurement across flats, small nut 11. ________ Screw slot width, large machine screw with nibs 12. ________ Head diameter, large machine screw 13. ________ Maximum body size, large roll pin 14. ________ Nominal size, large roll pin 15. ________ L.O.A. (length over all), large roll pin 16. ________ Largest, maximum bore size, large roll pin 17. ________ Wall thickness, large roll pin 18. ________ Rivet, length of shank 19. ________ Shank, body size 20. ________ Rivet length over all 21. ________ L.O.A., cotter pin 22. ________ Nominal diameter size, cotter pin 23. ________ Diameter across peaks, large nut 24. ________ Dimension across flats, large nut 25. ________ Inside diameter, large washer 26. ________ Largest diameter, big washer
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 14 of 32
Skills sheet – Micrometer Measurement (uses the same basket of parts as the micrometer measures activity) 1. ______________Thickness, large washer 2. ______________Thickness, length, large nut 3. ______________Nominal size, cotter pin diameter 4. ______________Rivet shank, body size 5. ______________Large roll pin, maximum diameter 6. ______________Large roll pin, nominal size 7. ______________Large machine screw, head diameter 8. ______________Major diameter, thread, large machine screw 9. ______________Thread outside diameter, small machine screw 10. ______________Measurement across the flats, small nut 11. ______________Thickness, small washer 12. ______________Maximum O.D., small washer 13. ______________Largest outside diameter, small roll pin 14. ______________Nominal size, small roll pin 15-16. ___________ How many thousandths are there in one-hundredth of an inch? 17-18. ___________ How many divisions are there on the thimble of the micrometer? 19-20. ___________ How many threads-per-inch does a mic have? 21-22. ___________ How many turns would you advance the thimble of a micrometer to travel .175? 23-27. ___________ Please list the five main parts of the micrometer… ___________ ___________ ___________ ___________
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 15 of 32
Skills Sheet – MORSE GUIDE EXPERIENCE 1. ________ You have only a #8 screw extractor. Can you remove a 5/8-11 bolt with it? 2. ________ What is the major diameter of a #10 machine screw? 3. ________ If you were going to cut an angle of 9 degrees, 28 minutes with the center line, on the lathe, how much taper per foot would you be cutting? 4. ________ What would be a common cause of broken drill bit tangs? 5. ________ How many millimeters are there in 1/2"? 6. ________ In relation to threads…what does “three-wire” mean? (keep it shore and simple) 7. ________ Would a 1/2" drill bit require a greater lip relief angle than a 3/4" drill bit? 8. ________ What is the thread form angle of an acme thread? 9. ________ What is the single thread depth of a 3”-4 thread? 10. ________ How much does a square foot of 1/4" steel plate weigh? 11. ________ What is the pitch diameter of an 8-32 thread? 12. ________ What is the root diameter of this 8-32 thread? 13. ________ What is the major diameter of this 8-32 thread? 14. ________ How many thousandths in a diameter is a 19mm piece of round bar? 15. ________ How much does two feet of 6” diameter carbon steel weigh? 16. ________ Charts pages 52-67… a. What does R.P.M. mean? ________________________________ b. What does S.F.M. mean? ________________________________ 17. ________ What is the maximum diameter of a 1"-8 thread 18. ________ What is the root diameter of a 1"-8 thread? 19. ________ What is the pitch diameter of a 1"-8 thread? 20. True / False - If we had some ham, we’d have some ham ’n eggs, if we had some eggs.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 16 of 32
Skills Sheet – Turn Shafts 1, 2, & 3 (same diagram used for many configurations, including Turn Shafts 1, 2, & 3)
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 17 of 32
Skills Sheet – Sharpening a Drill
Drill Sharpening Sequence 1. Practice continuous motion of grind with follow through. 2. Keep cutting lip parallel to centerline of grinder. 3. Lip clearance, don’t let the shank end get too high. 4. Grind the lip lengths the same, using a scale. 5. Check drill angle and lip lengths with the drill gage. 6. Check for worn margins. 7. Watch for too much lip clearance. 8. Hone the cutting lit for long life. 9. Provide web thinning where necessary.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 18 of 32
APPENDIX B: COURSE-LEVEL ASSESSMENT
Tool Description: Course-level assessment is essentially the same as program level assessment in that it uses a variety of assessment tools to determine the effectiveness of a course relative to the course’s published outcomes. The results of the course-level outcomes assessment are translated/mapped to the program’s outcomes for use in program level assessment. The resulting course contributions to program outcomes for all courses in a program (or some subset of the courses in a program) are combined to get an indication of how the program outcomes are being met from a course perspective. One side benefit from this process is that a “gap analysis” results from the table that combines course contributions to program outcomes. A view of the table will show where the curriculum is the strongest and where there may be “gaps” in content. Factors that affect the collected data: The course-level assessment is strongly influenced by the instructor’s perceptions since this is the individual supplying the data. This is tempered somewhat if one of the course-level assessment tools is a student perception survey. The factors that affect the results depend, in large measure, on the assessment tools used by the instructor. Some factors that influence the collected data include: o The standard set by the instructor. A low standard tends to result in fairly high values in course-level assessments. Students will not know if the standard was appropriately set until they leave the course and have to apply the knowledge and skills learned, so both the student perceptions and the instructor-introduced measures may tend to be higher than is reasonable if the standards are set too low. The opposite effect is seen when the standards are set too high. The level of the standard should be determined by the program faculty so that comparison between instructors is valid. Variability of standards makes data combination and comparison questionable. o The number and detail of assessments used in the course. For example, a single exam is often not a good indicator of performance for a variety of reasons (must be comprehensive, does not account for students that “have a bad test”, etc.). The fewer the assessments and the less detailed the assessments, the less reliable the results. How to interpret the data: Care should be taken to investigate and discuss the factors influencing the results before interpreting the outcome. The results of course-level assessments should also be compared against other measures to get good picture of program performance. See excel spreadsheet for sample course-level assessment tools.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 19 of 32
APPENDIX C: GRADE DATA
Tool Description: While many factors may affect a student’s grade in a course, a student’s proficiency in the subject matter is generally indicated by the grade that the student earns in the course. A student’s proficiency in a subject area can be determined by looking at the student’s grades in all relevant courses. When looking at the proficiency of the student body as a whole, the course GPA may give an indication of the proficiency of the students and the quality of the course. To evaluate many of its educational goals, the Mechanical Technology program will maintain a database of course grade point averages (GPA’s) each semester and a historical graph of these GPA’s. Courses to be included in the database are those courses that apply to the Mechanical Technology program and are taught by the UAA Community and Technical College. The course instructor’s name and the number of students in the course are to be clearly associated with each course GPA record and on the graph. Factors that affect the collected data: In evaluating the resulting data it must be noted that the factors influencing course GPAs are numerous and difficult to separate. Some of the factors include the preparation and basic academic abilities of the student, the quality of the presentation of the material by the instructor, the quality of reference material, student participation, and the grading philosophy of the instructor. How to interpret the data: The data will not be a very precise indicator of student performance until the faculty come to a consensuses regarding grading philosophy and performance standards. Spikes and dips in the data should be viewed in relation to the instructors who where assigned to the courses. As with other data, this should be compared with other data used to assess the same outcomes. Sample data: (this is only sample data – not actual data for any particular course)
MECH 101 – Intro to Machine Shop
Average Grade # Passed # Audits # Failed/Withdrawn # Incompletes
Student Statistics 3.36 18 2 2 1
Tabulating and Reporting Results: The staff prepares a summary of grade data on all Mech-Tech (and related welding) courses taught each year, by section. The staff also computes the GPA, for the year, over the course groupings identified below.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 20 of 32
APPENDIX D: WELDING CERTIFICATION TEST RESULTS
Tool Description: Welding Certification Test Results (only for those who choose a welding option) Factors that affect the collected data: The Welding Certification test is conducted by a nationally certified welding tester and should be an accurate reflection of skill level for this outcome. How to interpret the data: A passing result on a certification test indicates completion and mastery of this outcome.
This is a state certification and therefore no sample is included in this packet.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 21 of 32
APPENDIX E: EXIT/GRADUATE (ALUMNI) SURVEY
Tool Description: The exit survey asks graduates of the program to rate their performance relative to the program’s outcomes. Additionally, graduates are asked to rate the program’s delivery of the material related to the objectives from their viewpoint. A sample of the survey instrument is included on the following pages. Surveys may be distributed to students when they apply for graduation, or later by mail. The students return the surveys by mail or to the staff. The results are not examined until after graduation. Factors that affect the collected data: A number of factors need to be taken into consideration when analyzing the data. The following factors are those that we have identified. Low return rates. It has proven difficult get a good return rate from the graduates. This reduces the accuracy of the results. Timing may be an issue. They do not know for a couple of years, or more, how their skills apply to the real world.
How to interpret the data: Care should be taken to investigate and discuss the factors influencing the results before interpreting the results. The results of the surveys should also be compared against the other indicators to get good picture of program performance relative the expected outcomes. Every group of student going through the program is different and results may vary accordingly. Sample Survey: A sample survey is provided on the next page. An electronic version may be developed to replace this version. The questions and data collected will be the same. Tabulating and Reporting Results: The survey is prepared by the faculty. The survey is administered by the staff. Staff receives the results and tabulates them for use in faculty outcomes review.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 22 of 32
Mechanical Technology Certificate Program Spring 2008 Exit/Graduate (Alumni) Survey
The Mechanical Technology program has received mandates from the university to implement an outcomes-based assessment program. As a part of the program, we are surveying graduating students to find ways of improving our program. Your feedback will go a long way in helping us determine how well we are doing and what we can do to better serve our students, alumni, and the engineering community. Please return in the provided envelope. These surveys will be opened after our Spring final exam week. Please enter the year that you received your certificate _________ Have you accepted a permanent Mechanical Technology position? Primary Mechanical Technology Field that you hope to work in: □ Yes □ No
Machine Shop Welding Shop Construction Oil Field Water & Wastewater No Mechanical Technology function Other Mechanical Technology: ___________________________
The KPC Mechanical Technology Certificate program has adopted six expected outcomes, please rate your knowledge/skills and the program’s effectiveness in teaching you knowledge/skills relative each objective. The graduates of the KPC Mechanical Technology program will have the ability to: Importance of this objective to your employment Somewhat Important Extremely Important
How well are you able to function in this objective
Not Important
Very Important
Unacceptable
4.
6.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 23 of 32
5.
3.
2.
measure, identify and apply with real world parts and pieces…and produce appropriate blue prints show proficiency in the use, calibration, and care of all precision measuring tools complete advanced machine shop projects in a variety of materials using standard machine tools and studentcreated blue prints plan and complete machining jobs on the CNC (if applicable) pass entry-level welding certification test (if applicable)
1
operate basic machine tools at an entry level
Outstanding
Very Good
Important
Good
Poor
Optional comments:
Somewhat Satisfied Maybe Extremely Satisfied
Very Satisfied
How would you rate the overall preparation of KPC Mechanical Technology graduates to: be a machinist, mill write or mechanic? compete professionally as a machinist, mill write or mechanic?
No
Yes
Would you recommend KPC mechanical technology education to a friend or relative?
Please indicate your satisfaction with each of the following aspects of your experience at UAA/KPC. Please feel free to use the space after the list to briefly explain any of your responses, especially if you feel less than satisfied. Quality of the Advising: Quality of Instruction: Quality of Physical Facilities: □ poor, □ fair, □ good, □ excellent, □ outstanding, □ No opinion □ poor, □ fair, □ good, □ excellent, □ outstanding, □ No opinion □ poor, □ fair, □ good, □ excellent, □ outstanding, □ No opinion
Quality of Computer Laboratories: □ poor, □ fair, □ good, □ excellent, □ outstanding, □ No opinion Quality of Physical Laboratories: Optional Explanations: Please list up to three major strengths of your mechanical technology education or other UAA/KPC experiences. □ poor, □ fair, □ good, □ excellent, □ outstanding, □ No opinion
Please list up to three areas for improvement in KPC’s Mechanical Technology program or other aspects of KPC.
With respect to the previous question, do you have any suggestions on how KPC could address these improvements?
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 24 of 32
Not Satisfied
No Opinion
Satisfied
APPENDIX F: EMPLOYER SURVEY
Tool Description: The employer survey asks employers to rate the performance of their employees who have graduated from UAA relative to the program’s objectives. Additionally, employers are asked to rate the importance of the program objectives from their viewpoint. A sample of the survey instrument is included on the following pages. Employers of our graduates may be obtained from Alaska Department of Labor reports or through other means. Employers are surveyed approximately every three years The results are gathered by the school staff. Factors that affect the collected data: A number of factors need to be taken into consideration when analyzing the data. The following factors are those that we have identified. Low return rates. It has proven difficult get a good return rate from the employers, even with follow up phone calls. This reduces the accuracy of the results. Some employers resent or are bombarded by surveys and will not fill them out. Some classification groups may not respond to the survey. The result is that the data is skewed to the viewpoint of a subset of the employers of our graduates. It may be possible that the employer is not aware of all the UAA graduates in their employ. This leads to an assessment that is not representative across the group.
How to interpret the data: Care should be taken to investigate and discuss the factors influencing the results before interpreting the outcome. The results of the surveys should also be compared against alumni surveys to get good picture of program performance. Be aware that there is not a direct connection in the two surveys between the employers and the alumni that work for them. Sample Survey: A sample survey is provided on the next page. An electronic version may be developed to replace this version. The questions and data collected will be the same. Tabulating and Reporting Results: The survey is prepared by the faculty. The survey is administered by the staff. Staff receives the results and tabulates them for use in faculty outcomes review.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 25 of 32
UAA /Kenai Peninsula College
Certificate of Mechanical Technology 2005 Employer Survey
The UAA/Kenai Peninsula College Certificate of Mechanical Technology Program has received mandates from the University and to implement an outcomes-based assessment program. As a part of the program, we are surveying alumni and employers of alumni to find ways of improving our program. Your feedback will go a long way in helping us determine how well we are doing and what we can do to better serve our students, alumni, and the Process Industries. Please contact Drew O’Brien at Kenai Peninsula College Mechanical Technology Department (907 262-0344) if you have any questions regarding this survey.
The KPC Mechanical Technology Certificate program has four educational objectives. In this survey, we ask for your opinion relative to each of these four objectives. First, rate how important each is relative to your organization. Please note that importance relates to the needs of your organization and not to the importance of the item as it may apply to other organizations. Second, rate each item relative to how well you think our graduates are able to function relative to each objective. Please feel free to use the space after the list to briefly explain any of your responses, especially if you feel the preparation of the graduates was less than adequate. The objectives of the KPC Mechanical Technology Certificate Program are to produce graduates who: How well is the KPC graduate able to function in this objective
Importance of this objective to your firm
Somewhat Important
Not Important
Extremely Important
Very Important
Unacceptable
2.
measure, identify and apply with real world parts and pieces…and produce appropriate blue prints
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 26 of 32
3.
show proficiency in the use, calibration, and care of all precision measuring tools 4. complete advanced machine shop projects in a variety of materials using standard machine tools and studentcreated blue prints 5. plan and complete machining jobs on the CNC (if applicable) 6. pass entry-level welding certification test (if applicable)
1.
operate basic machine tools at an entry level
Outstanding
Very Good
Important
Good
Poor
Somewhat Satisfied Maybe
Extremely Satisfied
Very Satisfied
How would you rate the overall preparation of KPC Mechanical Technology graduates to: be a machinist, mill write or mechanic? compete professionally as a machinist, mill write or mechanic?
No
Yes
Would you recommend KPC mechanical technology education to a friend or relative?
Please list up to three major strengths of your KPC Mechanical Technology employees.
Please list up to three areas for improvement in our Mechanical Technology programs or other aspects of KPC.
With respect to the previous question, do you have any suggestions on how KPC could address these improvements?
What can the KPC Mechanical Technology program do to better serve the needs of your organization?
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 27 of 32
Not Satisfied
No Opinion
Satisfied
Demographics: Type of Organization: Federal Gov. State Gov. Design Consulting Construction Sales Other Consulting Educational Testing/Reliability Research & Development Other: ________________________________________
Mechanical Technology Field(s) that your firm works in (check all that apply): Machine Shop Welding Shop Construction Oil Field Water & Wastewater No Mechanical Technology function Other Mechanical Technology: ___________________________
Primary Job Function of Mechanical Technology graduates (check all that apply): Mechanic Welder Operator Other: __________________________
Number of total employees at your work site: < 10 10 – 25 26-50 51-100 101-500 501-1000 > 1000 Approximate number of employees with UAA Mechanical Technology Certificate at your work site: < 10 10 – 25 > 25 Number of total employees at worldwide sites: no other sites < 10 10 – 25 26-50 51-100 101-500 501-1000 > 1000
Thank you for taking the time to complete this survey. Please return it in the enclosed self addressed, stamped, envelope to:
Kenai Peninsula College 34820 College Drive Soldotna, AK 99669
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 28 of 32
APPENDIX G: INSTRUCTOR IMPRESSION SURVEY
Tool Description: Instructors will receive the tabulated results of the student impression data (student course-level assessments). The instructor will evaluate the course based on the same course-level outcomes that the students rated. Instructors will evaluate outcomes for each course he or she teaches and express his or her opinion of how well that outcome was taught and/or received by the students. Instructors will also indicate the value of each outcome to the overall objectives of the program. Factors that affect the collected data: In evaluating the resulting data it must be noted that the factors influencing an instructor’s impression or opinion will be subjective. Mechanical Tech faculty should meet to discuss the significance of any impressions, opinions, and recommendations, and include any actions or recommendations in the assessment plan report. Tabulating and Reporting Results: The staff prepares a survey for each course to be completed by the instructor of each course. The staff (and/or assessment coordinator) will tabulate results if appropriate (for courses with more than one section taught in a given semester), or provide the assessment to the Mechanical Tech faculty to review.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 29 of 32
Instructor Impression Survey (Sample)
Introduction to Machine Shop Course: MECH A101 Instructor(s): Xxxx X’Xxxxx Attached are the results of the course-level assessments that were completed by the students in this course. Please look over the results, and in the spaces below, write your own impressions and recommendations, as the instructor of this course, for each of the objectives. (Any blank responses will be interpreted as satisfactory in classroom effectiveness and no current recommendations.)
Students participate in the safe shop attitude Poor / Fair / Good / Very Good / Excellent Poor / Fair / Good / Very Good / Excellent Poor / Fair / Good / Very Good / Excellent Poor / Fair / Good / Very Good / Excellent Poor / Fair / Good / Very Good / Excellent
Students practice safe fundamental machine tool operations in the shop. Hands-on practice using vernier calipers, micrometers, and other precision measuring tools Complete entry level machine shop projects using the correct sequence of events/processes experiencing successes and failure, both learning tools Students practice the rebuilding and fabrication of tired and worn mechanical components from the real world
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 30 of 32
Please comment on the effectiveness of any of the facility, lab, or other support for this course.
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 31 of 32
APPENDIX H: COURSE GROUPINGS ASSOCIATED WITH EACH OUTCOME
The following is a listing of the course groupings associated with each outcome. Outcome #1: operate basic machine tools at an entry level MECH 101, 102, 201, 202 Outcome #2: measure, identify and apply with real world parts and pieces… and produce appropriate blue prints
PETR 155, MECH 201, 202
Outcome #3: show proficiency in the use, calibration, repair, maintenance, and care of all precision measuring tools PETR 155, MECH 101, 102, 201, 202 Outcome #4: complete advanced machine shop projects in a variety of materials using standard machine tools and student-created blue prints MECH 105, 102, 201, 202 Outcome #5: plan and complete machining jobs on the CNC equipment MECH 220, 220B Outcome #6: pass entry-level welding certification test (optional focus) WELD 101 (and other optional welding courses)
KPC Cert Mechanical Technology Assessment Plan
Submitted 6-14-07 Page 32 of 32