February 2, 2010
Michigan Career Education Conference
Heidi Speese, Project Coordinator
Test Development Process
Michigan, NOCTI and the Accounting Assessment
Four decades of experience as a not-for-profit
Governed by a Consortium of 50 states and territories
Expertise in providing coordinated assessment
services for states and educational entities.
Gather a team of Subject Matter Experts
Secondary, Post Secondary and Industry
Incorporate Edits Pilot Assessments
Establish Cutscores Analyze Pilot
(Benchmarks) Test Results
Students will demonstrate an understanding of the professional practices and
standards related to working in the field of education. Rating Items
4.A Determine knowledge and skills needed by teaching professionals. 3.0 1
4.B Demonstrate personal characteristics needed to work in the teaching profession. 3.0 1
4.C Identify qualities of effective schools. 5.0 2
4.D Define a personal philosophy of education. 1.0 0
4.E Prepare a personal career plan in preparation for a career in the field of education. 1.0 0
Career Awareness 9% 3
Students will analyze career paths in the field of education.
Society and Culture 11% 4
Students will understand the historical perspective of U.S. public
Professionalism 11% 4
Students will demonstrate an understanding of the professional
practices and standards related to working in the field of education.
Integration of Technology and Instruction 6% 2
Students will demonstrate an understanding of the use of current
technologies that are directly related to effective teaching methods.
Classroom Climate 11% 4
Students will create an effective learning environment.
Quality Test Items
Developing multiple-choice assessment items
Test only one piece of information per question
Use only plausible distractors
Avoid using answers that can easily be
Review questions for inadvertent clues
Avoid “all of the above” and “none of the
above” as options
As part of meeting the reporting requirements of
the Carl D. Perkins Act of 2006, Michigan pilot-
tested NOCTI’s Advanced Accounting assessment in
2008-2009 for possible use as the secondary
technical skill assessment f or the Finance cluster.
The assessment was administered to all secondary
CTE completers and concentrators upon leaving or
completing a Finance and Financial Management
Services program (CIP Code 52.0800).
• Work with instructors to set test dates
• Place orders
Before • Identify and train proctors
Testing • Assign user codes (using student UIC)
• Maintain test security
During • Oversee the testing process
• Release online tests for scoring
After • Distribute score reports to instructors
How long will the test take?
What will the test cover?
What are the test dates?
What are the computer requirements?
Are other materials needed?
What if I need to change test date?
Are there accommodations for special needs?
Which are covered?
Which are not?
Does emphasis of a major competency area
or task in the classroom match the emphasis
on NOCTI’s assessment?
Help Students Prepare
Study Guide Package
One Teacher Companion Guide
Ten Student Study Guide Booklets
Ten Online Practice Tests
Teacher Companion Guide
Teacher Companion Guide
Group Score Report
Analysis of Scores
Task Link Report
Systems No. of Your
Demonstrate an understanding of how business systems function within the 5 2
Demonstrate an understanding of the functions of systems in an organization 1 1
Demonstrate principles of customer service 5 5
Apply industry standards and practices to ensure quality work 7 6
Mean, Standard Deviation, Sem
Compare Individual, Group, Site, State
Trend Analysis and more…
Refine Curricula Based on NOCTI Results
A. What competencies do you teach?
We had some we were not teaching
B. When do you teach the competency?
Several moved to the senior year
C. Time spent on the competency?
One content area removed, another expanded
D. How do you teach the Competency?
1. Theory only
2. Laboratory only
3. Theory and Laboratory
4. Reinforcing Activities
5. Use of Related Mathematics