Career Aptitude Test Samples - Excel by zsr14341

VIEWS: 284 PAGES: 152

Career Aptitude Test Samples document sample

More Info
									 Type of                                         Program Recruitment Data
Response

   Plan     R--N (by cohort for each program) and total N




   Plan     *Track number of scholarships and stipends actually awarded (as a partial reflection of the quality of
            recruitment efforts) *Semi-annual interviews with both students and recruiters focused on identifying and
            measuring the level of efforts associated with the recruitment and the response of those recruited (including
            students who decline the scholarship offer); R--chart detailing student recruitment efforts that lists the
            UTeach student, high school attended, and the city; R--number of students enrolled in the introductory
            courses (and percentage increase this represents);




   Plan     R--enrollment history of the UTeach program (Fall 1997 to Fall 2002) including: “new recruits,”
            “enrollment,” and “graduated.” R--Seat availability in introductory field courses




   Plan     Student N applying for vs. awarded Noyce Scholarships




 Overview

   Plan     Adequacy of Pima CC and U of AZ partnership for producing appropriate numbers of applicants from both
            institutions.

   Plan
Plan




Plan   *Documentation of how recruited and potential Noyce students learned of the program and examination of
       recruitment processes to determine strategies with greatest success


Plan   *Internal tracking of specific outcomes such as the recruitment of Scholars




Plan   *Review of recruitment strategies


Plan   *Evaluate success in attracting, preparing, and retaining Noyce Scholars in teaching careers via program
       manager, PI, and co-PI annual extraction from DB and education plans to determine retention, attrition, and
       timely completion of a degree or credential




Plan   *Number of students recruited/selected to participate in program *Rigorous single subject selection
       process (initial basic requirements screening, group interview sessions, and a writing activity (choose
       strength and weakness attributes) involving formative, self-reflective assessment using Attributes of Highly
       Effective Teachers (bulleted in Eval Plan Doc)


Plan   *Number counts (numbers recruited, number of meetings, etc.) *Recruitment of math/science interns (see
       "summative program effectiveness data" column).


Plan
Plan   *Gathering systematic data: to clarify best practices in recruiting students with different beginning levels
       (i.e., undergraduate or graduate) and starting majors and to answer questions such as: how career changers
       learn about education programs, the best point in a career to enter teaching cert programs, the best point to
       begin to work with STEM undergrads to interest them in teaching in high needs districts/schools, whether
       particular kinds of science or math majors better prep students/professionals for teaching, and whether
       students interested in rural teaching vs. urban high needs differ.

Plan   *Eval plan will provide effectiveness data of the program in attracting individuals to the program. *Each
       semester: number of applicants eligible and/or receiving NSF scholarship. *Percentage of students
       accepted compared to total completing applications.




Plan   *Effectiveness of recruitment strategies




Plan   *Applicant evaluation (prior to admission)




Plan   *Effectiveness data on the UMEP program in attracting individuals for teaching careers. *Indicators of
       high quality teachers in urban settings (instrument) used to screen TEEMS applicants.




Plan   *The number of teacher candidates admitted in the Master of Arts in Teaching (M.A.T.) degree and
       endorsement programs
  Plan     *Data on: requests, how requesters heard about the program, and scholarship application submissions (to
           determine: 1) where the most interest lies and 2) where recruitment efforts should be stepped up




  Plan




Overview

  Plan




  Plan     *Effectiveness of the Noyce funding as a recruitment tool in relation to recruitment strategies developed
           within the SMAR2T, (CM)2, and Missouri Center programs. *Evaluation-documented selection process.
           *Evaluation data collected via application and acceptance paperwork from the applicant and the university.
           *Ongoing, formative oversight of data collection addressing the recruitment and retention of alternative
           certification candidates (i.e., number of students recruited into the program, annual comparison of number
           of Noyce-recruited versus number into other alt cert programs looking for Noyce stipend related trends).
           *Noyce recipient interview data on impact of stipends on decisions to stay in the alt cert program.


  Plan     *Advertising effectiveness: (a) Some effectiveness data regarding the ad campaign (designed by a private
           firm) by comparing hits on the web site to the advertising cycle (note: these data are not able to be
           disaggregated by Noyce scholar and other scholarship recipient website visits) (b) Continued collection of
           survey responses and anecdotal communications to determine the impact of marketing and advertising
           efforts on recruitment. *NSF and pre-service Noyce scholar responses on the influence the award had on
           their decision to enter into a teacher certification program

  Plan     *Based upon the demographic info on recruitment contacts and applicants to program




  Plan




  Plan
       Plan          *Review of program literature and activities related to recruiting. *Use of interviews, surveys, and
                     observations to evaluate Noyce scholar selection activities.


       Plan          *Data that identify how the program has attracted superior students into teacher preparation




       Plan




       Plan




       Plan          *Survey Scholars and quantitatively analyze data to determine whether the Noyce level of financial support
                     influenced their decision to enter teaching through this program.




       Plan




       Plan




       Plan




Overview--Timeline
Plan   *The number of students recruited to participate in the program




Plan
                           Demographic Data


*A summary of demographic data and statistics on stipend recipients. R--
Collective profile information disaggregated for “applicants” and “teacher
interns”: Age, Gender, Ethnicity, Degree Level, major, discipline related
career experience




R--Percentage of students from historically underrepresented groups (i.e.,
African American, Asian/Pacific Islander, Hispanic, and Native American)
enrolled in UTeach introductory courses; R--chart detailing student
recruitment efforts that lists the UTeach student, high school attended, and
the city




*Demographic data on all program graduates; R--%age of minority
enrollment for UTeach vs. College of Natural Sciences (CNS) and University
(U), R--%age of students by gender, R--classification of students by year in
school, college, and major, R--distribution of UTeach students' GPA
compared with CNS and U student distributions; R--UTeach graduates’
demographics (number and percentage by gender, ethnicity, year of entry into
UTeach, and overall GPA); R--SAT II Math scores of undergraduate UTeach
Scholars
e.g., test scores, grades, gender, race/ethnicity, disabilities




Evaluation of numbers on graduate recipient and preliminary examination on
the undergraduates
Noyce applicant characteristics
*Regularly updated database of demographic data and statistics for all Noyce
scholarship/stipend recipients *Continued documenation of the demograhics
(including ethnicity) of recruited and potential Noyce students




*Program client information *(Salish I instrument) to gather high school and
college coursework, GPA, type of curriculum used, field experiences
*Project Director (PD) maintains current database with pertinent
demographic information
*Percentage of scholars from underrepresented groups.




*Compiled demographic data from applications. *Obtain demographic data
on scholarship recipients




*KSDs (see"research questions/evaluation goals" column) measured: 1-3, 8-
10, 13-16. *Pre-interview qualitative content analysis of the written
application; *Qualitative evaluation of formal interview responses; *Use of
transcript to assess coursework and performance (esp. in subject-area
courses); *Applicant GPA; Applicant GRE scores; Qualitative evaluation of
recommendation(s) and applicant personal statement to determine specific
KSD mastery




*Additional demographic data and tracking of scholarship/stipend recipients
as they fulfill their teaching obligation (i.e., at least 5-10 years to learn how
many candidates became teachers in local public or private schools and how
many committed to science or mathematics instruction.
*Data on scholarship application submissions to determine the demographics
of individual interested in STEM teaching




*Examine database data for impact of Noyce stipends on recruiting diverse
students.




*Based upon the demographic info on recruitment contacts and applicants to
program; *Based upon the demographic, retention, and progress information
of scholarship recipients.
*Data that identify how the program has helped students overcome financial
barriers and supported and increased the number of students from
underrepresented minority groups *Student demographic data at program
entry




Examples of demographic data collected include: *Age, ethnicity, gender,
disability, place of birth, marital status. *Number of dependents, spouses'
education and occupation, household income. *Employment (full¬time or
part-time), parents' education and occupation. *Educational levels of
siblings, GPA in courses on the degree plan. *GPA in math or science major,
Texas Higher Education Assessment (THEA) scores and other standardized
test scores. *Previous attendance at other colleges and GPA from those
colleges.

*Records of demographic information for programs designed to recruit
STEM majors. *FOCUS Project evaluator-designed survey to collect
information on the quantity, diversity, and quality of UCI students
participating in the Community Teaching Fellowships In Math & Science
(CTFMS) program and and document teacher prep paths chosen by UCI
students in the CTFMS program.




*Document review of application packages and administrator records using
quantitative and qualitative methods to determine answers to questions such
as: a) what is the racial/ethnic makeup of the Scholars? b) what is the gender
makeup of the Scholars?

*Pictorial files emailed to depict the diversity achieved within cohorts.




*Data tracking (through the end of the obligation period) of demographic
information.




*Existing baseline data of course and major enrollments by race and
ethnicity, and Noyce Scholars program enrollments
*Demographic characteristics of the recruited (e.g., ethnicity, race, gender,
SES)




*Applicant form data: applicant experiences with children, reasons for
wanting to teach, hobbies, major, minor, GPA, ethnicity, and citizenship
status.
                             Noyce Student Performance in Program




*GPA data on Noyce students; R--Time-to-graduate for UTeach graduates who entered as freshman




*Scholar performance in science, math, and pedagogy courses & field experiences


*Review of Records: and individual progress by outside consultants and faculty personnel to determine if
progress toward graduation is adequate. *Longitudinal study on a chosen sample of Scholars of their academic
progress throughout their program
*Lack capacity to run separate evaluation of Noyce students in credential program




*Internal tracking of the progression of Scholars to careers as middle or high school science or math teachers




*Qualitative assessment of scholar academic progress via Noyce education plans with single-subject advisors
responsible for monitoring progress of all Noyce scholars *Project manager-created DB containing links to
Scholars’ Noyce education plans, demographic info from application forms, GPA, financial circumstances,
…and be updated via any new contact data collected at monthly seminars *Evaluate success in attracting,
preparing, and retaining Noyce Scholars in teaching careers via program manager, PI, and co-PI annual
extraction from DB and education plans to determine retention, attrition, and timely completion of a degree or
credential

*Number of students completing endorsement/credential requirements *Project director maintained database
containing info on completion of credential program




*Graduation and certification of scholars (see "summative program effectiveness data" column).




*Tracking of performance of Noyce fellows in courses within major
*Oral defense of extensive portfolios based on rubrics rooted in principles of reflection and constructivist
learning.




*Advisor continued documentation of scholar progress toward degree completion and teacher certification
requirements. *Number of Noyce scholars making satisfactory progress during academic year. *Percentage of
students with 3.0 at end of each semester. *Number of Noyce Scholars obtaining the required degree/teaching
certificate.




*Data on student performance in courswork.




*KSDs (see"research questions/evaluation goals" column) measured: 1-5, 7, 12-14, 16. *Course instructor
content analyses of: MAT 303 (Unit & Lesson Outlines), MAT 302 & 303 (Candidate Self-reflection), MAT
302 & 303 (Initial Teaching Philosophy Statement), MAT 303 (State/National Content Area Standards
Analysis). *Initial Technology Assessment (survey and demonstration) assessed by Program Director.
*Program director use of transcript analysis (mid-internship) to review Noyce student progress in graduate
content courses.


*Monitoring of UMEP students on ongoing basis to document progress toward degree completion and teacher
certification requirements.
*Monitoring of Scholar academic process using: GPA, the summative decision points within the ISU teacher
education curriculum, the senior portfolio, and ISU’s Unit Assessment System (using portfolio Live Text
Software) *Conducted by (Indiana State University) ISU mentors to Scholars




*Continuous data collection on candidate qualifications and progress (including Praxis scores)




*Evaluation-documented timely progression of students through the certification program. *To examine
candidate success through alternative certification programs such as SMAR2T: examine entering-to-graduation
ratio of Noyce stipend recipients and compare success rate of teacher placement between Noyce students to
others in SMAR2T programs.




*Verification of course enrollment and verification of academic progress (1x per semester; 1x during final
coursework semester)
*Scholars’ portfolios charting their ability to develop quality mathematics and science learning experiences
*The academic advising program will also provide important data.




*Progress of the cohort through: *completion of their course work, *field-based semesters, *full-time
employment as a teacher until completion of service. *GPA




*Records of attendance and participation for programs designed to recruit STEM majors. *Records of
attendance and participation for all required and optional activities for scholarship recipients in pre-credential,
credential, and new teacher induction years. *University academic transcripts of scholarship recipients.
*Formative assessment performance information generated by instructors, advisors, and the candidates
themselves in in the credential program teaching events; *Dept of Ed Noyce Scholarship Program co-
coordinator will: *oversee data collection related to scholarship recipients and the program including data from
the teacher credential program as well as *administer surveys to scholarship recipients.




*Data tracking (through the end of the obligation period) of teacher test scores and progress through the
program. *Performance assessment data collected at: a) entry into the program, b) during the prepracticum, c)
during student teaching, and d) at the end of student teaching.


*Data to be collected: Noyce Scholars’ GPA
*Level of education and program activity using: a) academic progress, b) student performance, c) course
evaluation data. *Interim achievement determined by progress through each level of the educational pathway.
*Progression (operationalized as the percent of students completing the program within the established
timeframe). *Performance (including GPA in all teacher education related courses). *Student Information
Management Systems (SIMS) and Testing Offices will enable Co-PIs to ID and track all levels of student
progress




*Survey instruments (see "self-report data" column) are intended to identify changes in participants' beliefs as
they progress through the program.
                                        Program Retention Data


R--Examination of: 1) attriters as %age of total recruited; 2) %age of recruits of each gender and then attriters
by %age of gender; 3) %age of recruits of each ethnicity and then attriters by %age of ethnicity;
4) %age of recruits at each age range and then %age of attriters at each age range; 5) timing of attrition.




R--%age retained early in program, R--retention ranges (conservative to liberal) by year in school, R--number
of students leaving after completing specific numbers of education courses, R--attrition rates by semester, R--
profiles of UTeach leavers (# and % by gender, ethnicity, year of UTeach program entry, overall departure
GPA, and destination of leavers);




*Noyce teacher likelihood of departure (vs. peers in CoS TPP and SEPM)


Annual interviews with scholars during the commitment period to ID and act upon retention-related problems
Internal tracking of the retention of Schlolars
*Enrollment services tracking of scholars each semester to determine continued enrollment. *Eval plan will
provide effectiveness data of the program in retaining individuals in the program.




*The number of teacher candidates retained in the Master of Arts in Teaching (M.A.T.) degree and
endorsement programs. *The number of teacher candidates awarded graduate degrees or endorsements.
*Ongoing formative oversight of data collection addressing the effectivenes of the Noyce monetary support in
retaining students in alternative certification programs. *Noyce recipient interview data on impact of stipends
on decisions to stay in the alt cert program; *Compare program retention ratios: # of Noyce stipend vs. non-
Noyce SMAR2T students receiving certification
Data to be collected: UMB transfer rates for scholars vs. comparable students.
*Ultimate achievement determined by the number of students admitted to and graduating from UNCP with
math and science education degrees (i.e., the number of students continuing in the program over the
established period of time and percent completing all requirements for certification/licensure).
                         Transition Experiences/Support for Teachers




*Quality of recruitment efforts as well as quality of collaboration with public school districts and individual
public school mentors measured via: a) survey responses of enrolled UTSA-UTeach students regarding the
quality of the mentoring program; b) survey responses of participating mentor teachers regarding their
commitment to the effort and quality of their mentorship. *Closely follow the kind and degree of support
provided by the University to its graduates once they are placed in participating high need districts by:
assessing the diversity of teachers produced by the program (as a reflection of the quality of support provided
by Noyce Scholars Promoters and the advisors). *Assessment of the processes for interaction and support
(provided to new teachers after leaving the project and beginning teaching in high-need districts) will involve
close tracking, through interviews and observation, a random sample of 20% of each year's graduates and: a)
a focus less on the quality of the teacher’s work and more on the degree to which the new teacher is
successfully supported in his or her new classroom environment; b) evaluation of the University’s ongoing
support efforts as well as
the efforts of the participating districts to nurture and support these new teachers.




*Formative and/or summative question of: What are scholar challenges and successes in transitioning to
teaching in high need schools? *Continued communication with scholars after program when beginning
teaching; *Evaluator observation of Beginning Teacher Support meetings
*Evaluation activities focused on scholar attitudes and beliefs about their science and math teaching
experiences




Twice-yearly surveys of scholars, advisors, and districts to asses: 1) the obstacles or difficulties encountered,
2) the value of the supports provided
*No details are offered on the data to be collected, but a belief is presented that Noyce money will enable
students to work in high needs (often less-paying schools). Also Noyce money will fund a teacher support
network for teachers in rural high needs schools.




 *External evaluations: use qualitative and quantitative means to review effectiveness of mentorship
component. *Enrollment in seminar to provide professional development and address new teacher needs.
*Number of Noyce Teachers enrolled in Follow-up Seminar each Fall. *Number of Noyce Teachers who are
observed in their classrooms and who participate in follow-up consultation. *Number of Noyce Teachers
who participate in the Dowling Annual Science Symposium and Workshops. *Partner schools will provide:
mentor teacher observations, daily evaluations of student teacher, key stakeholder involvement in training
workshops and evaluation of student teachers

*Quarterly interviews and surveys (in years 2-4) with Noyce Scholars and Program Mentors to determine the
effectiveness of the mentoring. *Quarterly interviews and surveys (in years 3-4) with Noyce Scholars to
determine their evaluation of the Induction program.




*Used mixed methods to determine the extent to which the project is successful in identifying the variables
that contribute to the disconnection between the preparation of mathematics teachers and their induction
experiences in urban schools




*The number of teacher candidates mentored by teacher mentors
*External evaluator plan to examine teacher candidate perceptions (surveyed once during teaching year 1
and including Likert Scale and anecdotal information) regarding the efficacy of: multilayered mentoring
support system to bridge transition to teaching, support during the induction year of teaching, and ongoing
issues related to teacher support, resources, professional growth, etc.




*Evaluation to assess mentor to induction year teacher relationships.




*Recipient self-report on challenges/frustrations encountered or expected to encounter in early years of
teaching.
*Phone interviews, observations, and documentation to evaluate the extent to which field experiences and
student teaching are enabling students to acquire the skills needed for teaching. *Surveys, interview, and
observations to evaluate the extent to which Stony Brook and partner site mentors are initiating Noyce
students into the demands and culture of teaching and school work.
*Cross-university study of the effectiveness of induction year experiences to generate within- and between-
campus data to serve as a baseline for continuing study and long-term tracking of Noyce students in the
workforce *Quantitative (i.e., number and diversity of teachers retained, how long they remain in the
classroom) and qualitative data will be gathered *These data will involve surveys, focus groups, and selected
follow up interviews of Noyce Scholars *Induction Year program development is an institutional focus for
all TxCETP partner universities, and this area of research and evaluation will extend beyond the four years of
the TxCETP Noyce Scholars program.

*Induction needs assessment, information on new hires, and entering addresses for teachers changing
positions (gathered by induction coordinator). *Email listserv to provide current info on activities in the
schools, with the teachers, and with the math and science departments at A&M-Texarkana




*Use quantitative and qualitative methodologies in conjunction with surveys, focus groups, interviews,
mentor log doc reviews, and supervisor doc reviews with/from Scholars, school mentors, and university
supervisors to answers questions such as: a) to what extent are the Scholars mentored in the on-the-job paid
internship? and b) with what kinds of competencies did the mentors have to assist (classroom management,
curriculum, cultural)?
                                                   Monitoring of Noyce Teachers




*Personal visits or phone interviews with the UTeach induction coordinator or UTeach advisors to keep track of where UTeach
graduates are teaching (and if left teaching to acquire detailed information on the reasons for that decision). R--Certification profiles
(i.e., mathematics, science, computer science); R--Graduates’ teaching statuses (i.e., currently teaching, entered but left teaching, never
entered teaching); R--UTeach current teachers’ profiles (i.e., teaching level, geographical location and school district);




*Location of school districts employing scholars




*Evaluator observation of Beginning Teacher Support meetings


*Noyce grads included in yearly data collection by TEAM-math (Transforming East Alabama Mathematics) evaluation process
Track Noyce Scholars through their teaching careers




*CSULA and LAUSD agreement and development of a system to jointly track new science teachers from CSULA




*Project director maintained database with information on scholar successful teaching for two years in an underserved secondary school




*Interview, pre- and post questionnaires, focus groups, and field observations to determine student success and growth in the teaching
profession


*Tracking of Noyce fellows (first three years of teaching) *Development of web-based tracking and reporting mechanism
*All CTE grads currently tracked for at least 5 years to see if they remain in the profession.




*Annual updating of contact information and work status. *Partner schools will provide: mentor teacher observations, daily evaluations
of student teacher, key stakeholder involvement in training workshops and evaluation of student teachers




*Compare Noyce and non-Noyce assisted teachers on measures such as:
entry and long-term retention (4-5 years)into teaching




*Tracking of students 5 years following graduation




*In the process of collecting data on the retention rate of TEEMS students across the metro-Atlanta area.




*Additional demographic data and tracking of scholarship/stipend recipients as they fulfill their teaching obligation (i.e., at least 5-10
years to learn how many candidates became teachers in local public or private schools and how many committed to science or
mathematics instruction.
*Monitoring of service obligation. (Phase I): Mentor assessments of scholars, Scholar annual survey responses about the impact on
their professional teaching of targeted teacher preparation in the urban environment and with Problem-Based Learning (PBL)
strategies. (Phase II): Data on retention of Noyce Scholars during the two-year service obligation period, Request for Scholars to keep
current contact information with ISU for 5 years, and Annual survey of scholars to determine: retention in the teaching profession
within a STEM discipline, the importance of the Noyce Scholars Program to Scholar current teaching, and with additional information
gathered when a teacher leaves the profession. *Management Team reports on retention (annually, year 4 final, and 5 years post-
program).
*End of the school year interview conducted with successful candidates from the first cohort during year 1 of teaching.




*Success of students becoming certified as determined via: (a) The number of teachers remaining in teaching through Noyce funding
term (Teachers tracked through alt cert program with comparisons between % of Noyce students staying in teaching and % of either alt
cert, but no Noyce funds or traditionally certified)




*Dept chair at Recipient's K-12 school reports on the maturity level and rate of progression of the Recipient
*Online survey evaluation of Noyce student teaching.




*Existing tracking procedures for other scholarships will be used to monitor Noyce Scholars




*Tracking system to maintain contact with scholarship students following graduation. *Alumni Association info on current addresses of
graduates. *Progress of the cohort through full-time employment as a teacher until completion of service.




*University academic transcripts of scholarship recipients. *Formative assessment performance information generated by instructors,
advisors, supervising teachers, mentors, and the candidates themselves in the future teacher inquiry workshops, in the credential
program teaching events, in student teaching mentor-novice coaching sessions, and in new teacher induction program activities.
*Descriptions of professional development activities in scholarship recipients’ mini-grant proposals. *FOCUS Project evaluator-
designed survey to track these students' progress in securing teaching positions in or out of the region. *Dept of Ed Noyce Scholarship
Program co-coordinator will: *oversee data collection related to scholarship recipients and the program including data from the teacher
credential program, partner school districts, and induction programs as well as *administer surveys to scholarship recipients.




*Data tracking (through the end of the obligation period) of teaching positions during the post-licensure induction phase
*Student Information Management Systems (SIMS) and Testing Offices will enable Co-PIs to ID and track all levels of student progress




*Survey instruments (see "self-report data" column) are intended to identify changes in participants' beliefs after being in the schools.
            School/District Characteristics                            Teaching
                                                                      Assignment
                                                                     Characteristics
*For the schools in which Noyce students are teaching:
summary involving academic and demographic data on the
school community. *Map depicting CPS school demographics.
*R--Geographical location, school data: racial/ethnic
composition, low-income rates of schools, student mobility rate,
reading, math, science standardized test scores (at schools
where MGM/MGS teachers taught)




R--school names and regions where UTeach graduates were
teaching; R--range of starting salaries




*Nature and location of school districts employing scholars        *info on science teaching
                                                                   assignments (e.g., grade
                                                                   level(s) and subjects(s))
*Internal tracking of the progression of Scholars to careers as
middle or high school science or math teachers
*Pilot GIS project (linking annual report to governor and
legislature to student placement database at Dowling to
understand and improve placement of graduates within NY
school districts).




*Additional demographic data and tracking of
scholarship/stipend recipients as they fulfill their teaching
obligation (i.e., at least 5-10 years) to learn how many
candidates became teachers in local public or private schools
and how many committed to science or mathematics instruction.
                        Coordination between Programs or Institutions




*Quality of collaboration between UTSA and the public school districts and individual public school mentors;
R--school districts, names, and campuses of the school teachers serving as mentors; R--Location of
coordinated efforts with public school campuses as detailed by a chart depicting the campus, school district,
and mentor teacher involved with UTeach; R--Program coordination with Industry via a chart detailing the
driver industry, representative, and affiliation information for partnerships with the UTeach program.




*Examination of how the Pima CC and U of Arizona partnerships works
*Evaluation of undergrad scholars linked to any activities/experience or course/institute evaluations done on a
regular basis




*CSULA and LAUSD agreement to jointly monitor CSULA new science teacher effectiveness




*Tracking Noyce fellows for first three years of teaching using Teacher Education Program mechanisms
already in place
*Partner schools will provide: mentor teacher observations, daily evaluations of student teacher, key
stakeholder involvement in training workshops and evaluation of student teachers
*Unit performance (NCATE reviews, university review of strategic plans and progress).




*Evaluation to assess the organizational structure (collaborative support system) as monitored by attendance
at orientation workshop and other indications of on-going collaboration between schools and the university.
*Interviews and documentation of process to determine the strength and productivity of partnerships and their
role in facilitating student entry into the teaching profession with confidence.


*Compare our (the 9 Texas A&M system partners) data with data collected by the Texas Higher Education
Coordinating Board’s “Closing the Gap” effort to increase graduates in critical fields such as mathematics and
science




*FOCUS Project evaluator will collaborate to create a regional teaching database (that will include all
scholarship recipients) to track the educational progress and institutional choices of prospective teachers in
pre-credential, credential, and induction teaching phases of their careers. *Collaboration between FOCUS,
CTFMS, and Noyce programs.




*Eval plan carried out under direction of the UMass Center for Learning, Teaching, and School Change with
"data tracking" conducted in conjunction with the office of the Director of Teaching.




*Evaluation will: a) be managed and conducted (as well as having surveys and evaluation criteria developed)
by the University’s Office of Institutional Research (UOIR), b) utilize guidance and feedback from the
Advisory Committee and faculty advisors, and c)involve collaboration between the office of enrollment
management and UOIR to develop a database of baseline indicators and to track project impacts.
                                               Noyce Teacher Effectiveness Data


*Ongoing focus on quality of teacher performance in the schools throughout service obligation period and collected via: a) surveys
(completed online by teacher interns and school administrators); b) letters of recommendation from school and university faculty who
observed the teacher interns in the classroom; c) standards-based observation protocols (e.g., Arizona Observation Protocol) to evaluate
classroom performance.




*A selected series of follow-up interviews and observations of Noyce Scholars in public school classrooms; R--Mentor teacher reviews
of UTeach students (e.g., enthusiasm and level of preparation);




*Noyce student performance data on state certification examinations *Noyce teacher effectiveness via visits to each during the first year
of teaching utilizing the Fitness to Teach evaluation instrument (developed for the UTeach program and available for trial studies in Fall
2003 and available for program-wide use in Spring 2004). R--UTeach student performance on the ExCET Exam (subdivided by content
and pedagogy tests and by “passed initially,” “failed,” and “passed with retest.” R--Average total score performance on content and
pedagogy exams by content specialization (i.e., mathematics, science, computer science).




*Co-PIs eval of Noyce graduates' success in implementing learned teaching strategies *Math achievement of Scholar-taught students
(state-mandated ARMT and tests to measure students' achievement of big ideas taught to scholars) *Examination of the impact of
increased mathematical content knowledge on teacher pedagogical content knowledge, teacher practice, and student learning
*Stallings Observational System (SOS) data on OPTIONS (of which Noyce are subset) vs. HISD (standard certification) teachers to
compare timepoints when they begin implementing classroom behaviors typical of more expert science teachers and which behaviors
emerge. *Portfolios: containing goal statement, annotated evidence, and discussion of progress toward a self-specified science teaching
goal; these periodically-reviewed portfolios provide a formative assessment as well as summative evaluation of participant development
of teaching skills. *Observations: 1) within the classroom via the district mentoring program; 2) also 4 additional classroom obs.
occasions using the Stallings Observation System (SOS) that uses Microsoft Access to build teacher profiles based upon: a) "classroom
snapshot” data: the types of activities the teacher initiates with students and b) “five-minute interaction” data: the types of interactions the
teacher has with students (asking Qs, promoting discussion, lecturing, praising, inquiry, guiding exploration, etc.).




*4 annual observations at Noyce teachers sites using university-developed measures of teacher effectiveness *Discussions with school
administrators for perceptions on teacher performance.




*CSULA and LAUSD agreement to jointly monitor CSULA new science teacher effectiveness


*For Noyce scholars in the classroom: quarterly mentor reports assessing scholar performance and attitude *For district interns:
university field supervisor report of effectiveness *For scholars fulfilling 2-year commitment: districts will provide data on student
performance on standardized test scores for scholars’ students versus similar students *Effectiveness data will be assembled and analyzed
annually by the program manager, PI, and co-PI to determine the effectiveness of the Noyce Scholars in the classroom.




*Formal and informal data gathering *California State University Systemwide Evaluation of Professional Teacher Preparation to evaluate
the effectiveness of CSU grads who held K-12 teaching positions for > 1 year and gather administrator opinions (results are reported as
composites with longitudinal data system-wide or disaggregated by individual campus).
*Supervisor performance evaluations during first 2 years of teaching. * Pupil performance on federally required standardized science and
math tests (and comparisons with non-Noyce pupils) *Examination of conduciveness of classroom arrangement to constructivist
learning. *Samples of Noyce teacher assignments and student performance (and comparsions with non-Noyce). *Effectiveness of
preparation gauged by Noyce teacher integration of content and pedagogical knowledge to develop learning activities that enable pupils
to: learn how to learn and monitor understanding of STEM conceptions, carry out meaningful investigations of key topics, work
productively with others on meaningful learning and inquiry tasks, find STEM content relevant, worthwhile, and interesting.
*Performance on NY State required prospective teacher tests assessing content and teaching knowledge.
*While enrolled at Dowling: field-based experiences and pedagogical and content knowledge training measured via field supervisor
implementation of the Framework for Teaching classroom observation rubric. *Noyce teacher ability to improve student learning (using
portfolio and student work). *Internal evaluations (review scores from observation rubric and Noyce Teacher Assessment Portfolio).
From academic year 2001-2002 on: a comprehensive database with enrollment trends; indicators of student achievement in reading,
writing, mathematics, science, and vocational courses; graduation, college attendance and employment rates; *Partner schools will
provide: mentor teacher observations, daily evaluations of student teacher, and evaluation of student teachers


*Quarterly evaluation of quality of teaching (observations, evaluation of classroom performance). *Compare Scholars and Professionals
as teachers with respect to learning outcomes. *Compare Noyce and non-Noyce assisted teachers on performance on the NTE exam
required for licensure. *Semi-annual student achievement scores and quarterly student classroom performance data.




*Note: some data described in the "summative program effectiveness" column may also be applicable here. *KSDs (see"research
questions/evaluation goals" column) measured: 1-4, 7-15. *State licensure application results; *Praxis II results; *Licensure coordinator's
rubric-based examination of the 1st year teacher survey; *Program director and faculty advisory committee rubric-based examination of
the Hiring Principal Survey; 4) subject-area methods instructor and mentor teacher rubric-based evaluations of Noyce teacher unit plans
using SET; 5) course instructor evaluation of Noyce teacher IEP Project paper and presentation;




*Measurable objectives of the projects goals are: A list of valid and reliable performance measures that indicate a highly qualified
mathematics teacher in urban schools; • A list of attitudes, values, and beliefs that correlate with characteristics of a highly qualified
mathematics teacher in urban schools; *Effectiveness data on TEEMS teachers across the metro-Atlanta area; *Effectiveness data of
UMEP teachers in the classroom.
*Develop a set of indicators (instrument) of high quality teachers in urban settings using several phases and urban teaching TEEMS
graduates with > 3 years of teaching experience: *Phase 1: Conduct individual interviews using a narrative storytelling research method
in order to develop preliminary indicators; *Phase 2: Focus group with TEEMS grads to refine and add depth to indicators; *Phase 3:
Survey of secondary mathematics teachers in urban settings for wider range of feedback (results in semi-final set of indicators); *Phase 4:
Field test with high quality urban secondary mathematics teachers identified by another means (results in final version of instrument)




*The number of teacher candidates (M.A.T. or endorsement) that improve scores on the standardized tests in Maryland.
*Scholar impact on students as teachers: Baseline student demographic data, Academic information using Indiana's ISTEP+ exam results
in Math and Science and the NWEA Measures of Academic Progress (MAP) test scores (math only) for participating IPS sites (ISTEP
scores can be used for state-wide comparison, MAP scores can provide national comparisons as well). *ISU will fund a graduate student
researcher responsible for collecting baseline and impact data, *Abilities of Noyce Scholars-in-service will be monitored by the PDS
institution-wide annual assessment system offering multiple checkpoints throughout the program for remediation and refinement of noted
deficiencies, as well as the impact of the performance of the Scholars-in-service on the students they teach. *Annual summaries of the
program impact of NSP Scholars will be included in the annual assessment report prepared by the Assessment Coordinator and will be
*Mentor by the external and summative appraisals of pre-service teacher performance) during the academic year and end of each
reviewed input (formativeassessment consultant.
semester. *Professional portfolios (during 2nd student teaching experience) including: lesson plans for 5 consecutive lessons, video tape
of lesson taught, copies of student work from that lesson, analysis of impact of the lesson on student learning, analysis of their teaching.
*Assessed in content knowledge, task selection, classroom environment, analysis of student learning, and analysis of teaching (results
aggregated to determine if program-completing scholars possess the skills, knowledge and dispositions expected of beginning teachers as
established by Interstate New Teacher Assessment and Support Consortium (INTASC)). *2 observations (protocol designed by Horizon
Research, Inc. for NSF) with pre- and post-observation interviews during 1st year of teaching. *Follow-up on scholar effectiveness with




*Assessment of quality of Noyce teachers (Assessed through: interviews with Noyce recipients’ university instructors, university GPAs,
and interviews with administrators at the school/district where recipients teach)




*Classroom observations of scholar graduates in their 2nd year of teaching (using the NSF-CETP Core Observation Protocol)




*Student Teacher Appraisal Reports (STAR) (10x in year 1 and 2x in year 2) to evaluate recipients' teaching and completed by school-
based and field-experience mentor with observations by the building principal. *Texas EXamination of Educator Standards (TEXES)
content and pedagogy exams during final coursework semester. *Classroom Environment Scale (CES, Moos & Tricket, 2002) (pre/post
measurements in teaching years both 1 and 2) measures student and teacher perceptions of the classroom and provides a way to examine
the effects of course content, teaching methods, teacher personality, and classroom composition. *Texas Assessment of Knowledge and
Skills (TAKS) (1x during both teaching years 1 and 2) to measure the achievement of students in the scholarship recipients’ classrooms.
*Professional Development and Appraisal System (PDAS, Texas Education Agency, 2003) (4x during teaching year 1 and 2x during
teaching year 2) to evaluate teachers each academic year according to 7 criteria representing categories of teacher responsibilities and
levels of instructional effectiveness. *Teaching portfolio (2x during teaching year 1; 1x during teaching year 2) to determine if recipient
is applying the assess professionaland dispositions of effective teaching practice. *External evaluator designed practicum logs; c)
*Evaluation to knowledge, skills, development. *Evaluations conducted through: a) in-service workshops; b) plan to:
mentor/Induction year teacher meetings; d) follow-up survey sent each year beginning in fall 2004. *Success of classroom activities
measured by an increase in quality content labs that focus on laboratory experiences and manipulatives designed to improve outcomes for
students in high-need schools.


*Recipient self-report on plans for monitoring teaching success and 3 or more measures used to determine effectiveness
*Mentor teacher and other school personnel evaluations. *Annual evaluations by principals. *Other pertinent data.




*Performance assessment data will be used to conduct a longitudinal study of the early stages of teachers’ careers using students’
standardized test scores as a measure of teacher effectiveness.
*Teaching effectiveness (comparison of teachers’ N.C. end-of-year and end-of-course passing rates with program teachers). *Detailed
records maintained on teachers participating in professional development activities




*Survey data (of Noyce students) on what constitutes good teaching and the amount of time teachers should spend in various types of
pedagogical activities (e.g., lecture, discussion, real world problems).
                               Fulfillment of Scholarship Requirements


*Summary of data and statistics on stipend recipients including employment tracking during the service obligation to
the Chicago Public Schools (CPS) involving academic and demographic data on the school community. The
tracking system will retain recruitment, employment, and retention data across the years of the service obligation.




*Info on recipients not meeting requirements
*Database on all Noyce teachers while fulfilling obligation in high-need schools




*Education plans and monthly seminars to reinforce and track scholars’ compliance with program requirements
during their studies and their subsequent teaching commitment *DB containing info on attendance at Noyce
orientation and monthly seminars, filing of reports from school districts where Scholars teach *Program manager
and advisory committee (includes district representatives) created reporting form to track scholar compliance as
interns or teachers *Quarterly school principal indication of scholar job performance and whether it justifies
continued employment *HR dept confirmation of scholar employment *Mentor report of regular scholar attendance
at meetings with mentor
*Regular surveys into their beginning years of teaching and an online portfolio system monitored by the College of
Education Career and Program Support (CAPS) Office. *An electronic tracking system developed by the Indiana
Dept of Ed that identifies individuals currently teaching in the State of Indiana and lists the school that employs
them. *Required (years 1-6) written documents submitted to the Student Services Office indicating their current
teaching status. *MU Alumni Office’s maintained database of contact information on all MU graduates.




*Signed recipient acceptance agreement outlining terms of award *Annual mail correspondence to gather current
contact information and employment data *Fall 2005 (first year with Noyce graduates meeting teaching service
obligations) employment information to certifying teaching placements for service fulfillment




*Verification of placement in a qualifying school (1x during teaching years 1 and 2 with monitoring continuing until
service requirement is fulfilled)
*Tracking system to monitor service requirement until completed or reimbursed. *Student completed inquiry form
(web page to collect demographic data) and principal-signed certification form attesting to current employment each
semester)




*Review documents in the program files using quantitative methodologies to determine answers to questions such as:
do scholars sign necessary paperwork? *Doc review and subsequent quantitative analyses of the district employment
records database will assist in answering questions such as: a) how long do Scholars remain teaching in high need
districts? and b) what kind of schools do Scholars choose after graduation (diversity of students, faculty,
administration; SES; school grades)?




*Scholarship recipients required to participate in performance assessment process through end of obligation period
to evaluate teacher effectiveness




*See "summative program effectiveness data" column *To assess achievement of program objectives
*Tracking of scholarship/stipend recipients (while fulfilling teaching obligation) will follow the NC Teaching
Fellows model: a) recipients sign a promissory note; b) PI and Co-PIs verify academic success and post-graduation
employment compliance (with data collected during pre- and in-service development activities through agreements
with regional school systems).
                                            Formative Program Effectiveness Data




R--Details on program financial stability (current and projected expenditures)




*Survey of graduates about portions of UTeach that were most and least useful and institute program modifications in response




*Yearly evaluation to see if project goals are being met
*Evaluate short-term impacts of project activities




*Formative implementation evaluation: follow project evolution to see how well it becomes an established functioning entity and to
document changes in project plans *Formative progress evaluation: baseline for components; assess progress toward project goals (i.e.,
how are activities and strategies aiding participants to move toward goals)




*Review of program materials and other project activities and records *Meetings/interviews with the PI and other key staff to ID
strengths and weaknesses *Evaluator-led focus groups with students to get feedback on levels of support received during their
participation in the program *Questionnaires mailed or emailed to students periodically
*Twice-yearly surveys of scholars, advisors, and districts to assess satisfaction with the program and to determine any adjustments or
improvements that need to be made with scholars asked to asses: 1) the obstacles or difficulties encountered, 2) the value of the supports
provided




*Program description both as proposed and as implemented (including program objectives and enabling activities as well as resources
allocated to accomplishing program services). *Science Teacher Efficacy Belief Instrument (STEBI) to determine teacher candidates'
perceptions on the effectiveness of the science and math methods courses. *Salish I Research Instruments to holistically use separately
gathered data on: demographics, pedagogical philosophy, teaching (observation), and personal attitudes of the teacher towards the nature
of science, math, and technology to determine preservice math and science program effectiveness.

*Qualitative data to alert project management to strengths and weaknesses in order to adjust components and report the effectiveness and
impact of key project elements. *Benchmarking data on: key student expectations, perceived needs, assumptions, program satisfaction,
and preparedness and success. *Data to formatively evaluate program effectiveness and make mid-course corrections will be gathered
via: interviews, pre- and post- questionnaires, focus groups, and field observations.
*Rubric-based analysis of Cornell Teacher Education (CTE) student projects to determine if: some students take longer to learn to reflect,
if students beginning program at later points in life progress differently than college juniors, and if some assignments better foster the
thinking/acting desired in future teachers.




*Eval plan will provide effectiveness data of the program in preparing program participants.




*Quarterly focus groups and interviews with STEM scholars, STEM professionals, and Drexel Faculty; analysis of student teaching
journals, electronic porfolios, and performance-based course assessments for STEM scholars. *Evaluate practices used to prepare Noyce
scholars and professionals.




*KSDs (see"research questions/evaluation goals" column) measured: 1-16. *Mid-Internship: 1) evaluation of Noyce teacher internship
by mentor teacher using the Summative Evaluation Tool (SET); 2) conference with discipline area specialists to evaluate Noyce teacher
internship; 3) program director evaluation of Noyce teacher internship using SET; 6) 2nd Technology Assessment by program director
(using survey & demonstration methods).




*External evaluator, PI, Co-PIs will collect year 1 baseline data to evaluate the project against in years 2-4; *Report formative findings
on a continuous basis;
*Scholar perceptions (after years 1, 2, and 4) on how well the program prepared them to teach in relationship to content knowledge,
teaching for understanding, evaluating student learning, reflective practice, and professionalism.




*Formative Self-evaluation of Faculty Teams (considered important to monitor and modify program elements)




*Evaluation including formative feedback on the process and progress of project implementation. *Evaluation-documented effectiveness
of web site information to the Noyce recipients.




*Whether NSF and pre-service Noyce scholars participated in grant-related activities available to them, and how helpful these activities
were in their development as a mathematics / science teacher




*Formative evaluation to assess: organizational structure (collaborative support system), professional development, mentor to induction
year teacher relationships, and expectations (sustainability of project activities after the grant, information dissemination, weaknesses).
*Attitudinal instruments will measure satisfaction with project activities by participants. *Additional data will be collected through
mentor notes and practicum logs. *Evidence of technology and best practices being integrated into coursework assignments will be
present in course syllabi. *Course evaluations by project participants will provide further evidence of project success.
*Online survey evaluation of particular courses. *Annual reports on all aspects of the program including appropriate recommendations.
*Program documentation, interviews, observations, and discussions to evaluate how well courses meet science/math content and
pedagogical needs and that students are satisfied, challenged, and successful in course and lab work. *Observation, interviews, and
documentation to evaluate how well program administration establishes structures and roles and enables students to move smoothly
through the program with problems solved as needed.




*Utilize doc reviews and surveys as well as both quantitative and qualitative methodologies with course enrollment documents, course
syllabi to determine whether Scholars have been initiated into the field of education and becoming part of a professional collaborative
community (Scholar opportunities to develop a cohort group are relevant for this outcome).


*See "summative program effectiveness data" column as these data may be intended for formative evaluation use.




*Formative evaluation using: a) analysis of scholarship recipient surveys, b) questionnaires, c) focus groups, and d) data from other
stakeholders (e.g., cooperating teachers, school administrators).




*Evaluation of the Noyce Scholars program will use ongoing formative assessment tools as well as existing instruments for assessments
(course grades, cumulative GPAs) whenever possible: a) to assess programmatic strengths and weaknesses (including quality with which
the proposed benchmarks within the activities are obtained); b) to assess the extent to and quality with which proposed benchmarks
within activities are attained, and c) to assess the systems for documenting service delivery.
*Outcome data including: assessments of the # of program participants receiving services and the # of contact hours (with efforts made to
make services equally accessible to diverse groups). *Formative (to determine if program is implemented as intended and improve the
project) activities utilizing ongoing analyses of quantitative and qualitative data will be conducted. *Questions to guide the evaluation of
the integrity of program delivery will be developed according to the NSF’s Division of Research, Evaluation and Communication “User-
Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering and Technology Education.”




*Possibly survey response change over time.
                                           Summative Program Effectiveness Data


*Summation of the formative data on the quality of teacher performance




*Degree to which the interaction of the Noyce project and the UTSA-UTeach project actually produces a significant increase in the
number of science and mathematics teachers certified (measured as the number of students enrolled in the program and the number of
appropriately certified teachers produced by the program against the baseline data embodied in the statistics of the past five years and
conducted at 6-month intervals throughout the project with info on both enrollees and graduates of the program collected)




*Survey of graduates about portions of UTeach that were most and least useful and institute program modifications in response; R--graph
of enrollment trends in student teaching (by major) from 1996 to 2004.
*Evaluate long-term impacts of program activities. *Standard evaluation forms: (created for BCM’s GK-12 program) to assess
participants’ satisfaction with the quality of the “Introduction to Science Teaching” experience (including documentation of the actual
activities undertaken by each participant (number of hours spent co-teaching, planning/teaching a lesson, conducting observations)).




*Eval to determine extent to which FCEPT/Noyce activities led to major and long-lasting reform in: teacher preparation, support of new
teachers, teacher retention rates, and student classroom achievement




*Exit or end-of-program interview survey *Annual follow-up questionnaires mailed to students starting one year after program
completion

*Effectiveness data will be assembled and analyzed annually by the program manager, PI, and co-PI to determine the effectiveness of the
program as a whole.




*Information on the impact of program services and activities on participants and other service recipients (including number of students
employed in positions directly related to the program).




*Initially counts and graphs of numbers representing each of the following outcomes (recruitment of math/science interns, graduation and
certification, employment by many in low-income urban districts, and continued employment of teachers especially those of color) in
pipeline format for a given cohort with subsequent time series data for key outcome measures and comparing cohorts to previous cohorts.
*Scholar exit interview and survey. *From academic year 2001-2002 on: a maintained comprehensive database with information on
teacher and administrator preparation, turnover, in-service education and performance. *Eval plan will provide effectiveness data of the
program in preparing individuals for teaching careers. *Number of Noyce Scholars who obtain math/science teaching positions in high
need districts. *Number of Scholars who maintain these positions beyond required time (records maintained until 2014).




*Annual interviews and focus group meetings with key program staff representing partner institutions to evaluate the overall program and
to obtain suggestions for improvement. *Determine long-term performance of scholars and professionals in terms of retention as a
teacher, job satisfaction and identification with the profession




*KSDs (see"research questions/evaluation goals" column) measured: 1-16. *End-of-Internship: 1) Mentor teacher evaluation of Noyce
teacher internship using SET; 2) Thesis committee rubric-based examination of Noyce teacher's Teaching Portfolio (i.e., final philosophy
statement, technology portfolio, meta-reflection, student growth and development demonstration, and content knowledge assessment).




*Effectiveness data on the UMEP program in preparing individuals for teaching careers; *Report summative findings upon completion of
the project, comparing project outcomes against baseline data;
*For TEEMS program, the frequency of graduates: *with a pass-rate of the PRAXIS II (teacher certification) exam at 100%; with a 3.0
GPA or above; with grades of “C” or better in mathematics courses; with grades of “B” or better in all field work; meeting or exceeding
the technology standards and skills in appropriate and meaningful uses of technologies in teaching mathematics; demonstrating and
documenting evidence of knowledge and skills across the ten Principles of Interstate New Teacher Assessment and Support Consortium
(INTASC).
*Annual data reported on the overall effectiveness of the NSP program to to attract, prepare, and retain highly qualified pre-service
teachers; *Collection of data on the impact of the Noyce Scholars Program (NSP) teacher preparation on student achievement




*Scholar and immediate supervisor perception data analyzed to determine program effectiveness.




*Continuous data collection on performance of graduates (exiting student surveys, National Board certification data), *Program
performance (specialty program reviews, accreditation reviews, university program reviews, NCATE reviews, faculty evaluations),


*The quality of teacher produced with support of Noyce funding and her or his longevity in teaching within the timeline of Noyce
funding. *Successful implementation of project as outlined in proposal. *Success of students becoming certified as determined via: (a)
The number of teachers remaining in teaching through Noyce funding term (Teachers tracked through alt cert program with comparisons
between % of Noyce students staying in teaching and % of either alt cert, but no Noyce funds or traditionally certified) and (b)
Assessment of quality of Noyce teachers (Assessed through: interviews with Noyce recipients’ university instructors, university GPAs,
and interviews with administrators at the school/district were recipients teach)




*Survey responses will also be gathered from in-service teachers on unmet professional needs and preferences (see "Specific Analyses or
Methodologies" column for greater detail); *How these scholars rated the quality of their teacher preparation programs




*Teacher technology survey (pre-test in semester 1; post-test prior to teaching year 1) (South Central Regional Technology in Education
Consortium, 2004) to establish baseline data about the technology skills of beginning teachers; *External evaluator plan to examine the
relationship of content and pedagogy training to student achievement on state exams.




*Summative evaluation to assess: mentor to induction year teacher relationships, expectations (sustainability of project activities after the
grant, information dissemination, weaknesses), and outcomes (benchmarks of the program, final results of activities)
*Qualitative student survey data on: their experience in their degree program, *satisfaction with the support services, *perceived value of
program participation, *impact of family and financial assistance on their college experience.




*Gerdeman’s dissertation (a 2004 resource is cited in the received email) is being used as a baseline for change in the perception and
interest of UCLA undergrads in science and math towards teaching careers.




*Annual summative evaluation (to determine program impact) will involve interviews with involved faculty and BPS head administrators,
yearly assessments and interviews with each of the participants (including those in new teaching positions for less than two years) to: a)
assess achievement of the program objectives, b) to measure the impact of the activities on Noyce Scholars around retention, academic
performance in STEM courses, quality of teaching preparation experiences, and graduation with a baccalaureate degree, and c) to
determine whether the UMB Noyce Scholars Program proved the intellectual merit of its teacher preparation model for UMB, BPS, and
the public record for effective practices
*Summative (to determine the extent to which goals were achieved) activities utilizing ongoing analyses of quantitative and qualitative
data will be conducted.




*Possibly survey response change over time.
              Document Analysis




*Program and student records
*Review of project documents and records;
                                       Observations




*Of Advisory panel meetings


*By co-PIs of scholar graduate's implementation of learned teaching strategies.
*Initial or final student teaching scholars in credential program are evaluated by university
supervisors (i.e., by a program already in place) using performance guidelines by SOE *Intend
to evaluate fully credentialed Noyce teachers next fall (PI applied for sabbatical to do so)




*(Salish I) controlled observational instruments to supplement data from the Teachers'
Pedagogical Philosophy Interview (TPPI).
*Classroom observations and evaluations with program status and evaluation data published
annually on the Dowling website. *Spring classroom observations in first critical years after
graduation with follow-up consultation with Noyce teachers.




*Years 1-5: on-site visits
*3 classroom observations conducted in the first year teachers’ classrooms




*Structured observations of graduates: an induction facilitator will conduct these and provide
data
                               Self-Report Data




*Recipient opinions on teacher preparation program(s) and first teaching
assignment


*1st year exit interviews with recipients.

*Annual interviews with undergrad scholars *Annual surveys with scholars once
teaching

*Interviews with scholar graduates on experiences in implementing learned
teaching strategies
*Surveys: To assess participants’ science teaching efficacy beliefs (At three time
points: before any project activities, after preliminary teaching experience, and
after completion of all requirements) using the SETAKIST (Roberts, 2002)




*Attitudinal surveys given yearly to Noyce graduates while teaching




*Evaluation activities focused on Scholar attitudes and beliefs concerning the
program's value; *Scholar "self-evaluative" journaling regarding experiences
tutoring and student teaching; *Evaluation activities focused on Scholar attitudes
and beliefs about high needs schools' attributes
*Spot phone calls to alumni to get feedback on successes and challenges they
face as science teachers.

Scholar survey (administered at beginning of program and annually) to assess
scholar opinion of their ability to teach science or math, scholar level of
commitment to teaching, scholar opinion of the teaching profession, the impact of
Noyce funds on their decision to become a teacher * …with questions adjusted
once teaching to capture satisfaction or frustration with teaching, plans to
continue teaching, and intention to continue teaching in their current school or
another high-need school

*(Salish I) focus group exit interview of graduates' pedagogical philosophy to
elicit teacher perspective on effectiveness of science teacher strategies and
evidence of student success on achieving science and math concepts. *(Salish I)
nature and implications of science and technology survey that explores the
teacher candidate consistency or change in perceptions of the nature of science.

*Interviews with participants each semester in the program and each year
thereafter
*Qualitative data on scholar impressions of and overall satisfaction with the
services of the project




*Annual survey of Noyce Scholars to determine whether they've developed an
identity with the teaching profession. *Compare Noyce and non-Noyce assisted
teachers on surveyed teacher attitudes towards teaching during the first three
years and surveyed job satisfaction and teacher motivation




*Years 1-5: focus groups, and interviews with UMEP Students and Leadership
Team; *Survey of UMEP participants each year of participation in the project;
*Efficacy scale completed both upon program admission and at end of program
with aggregated data used to determine if program impacted the scholars’
perceptions of their skills, knowledge and dispositions toward teaching. *3
interviews and a final questionnaire administered to the teachers




*Evaluation data in the form of: *interviews with selected stipend recipients and
members of the review committee as well as *interviews with the PI and
developers of the database.




*Both NSF scholars and pre-service Noyce scholars' opinions of various
“reformed” teaching techniques, and their philosophical leanings on key
MMSTEC values (many of these survey items adopted from CETP Core surveys)
*Survey responses from in-service teachers on unmet professional needs and
preferences *By end of 3rd year of teaching survey all teaching scholars to
determine if (and to what extent) the scholarship influenced their teaching.

*Intern survey (1x during teaching year 1 including Likert Scale and anecdotal
information) on teacher efficacy, knowledge/skills, and commitment to teaching.




*Recipient self-report of 3 or more measures s/he will use to evaluate satisfaction
with teaching.
*Annual interviews of all Noyce scholars and a selection of Stony Brook faculty
and partner participants. *On-line surveys to evaluate other components of the
program.




*Survey data from: *A&M Texarkana faculty, *mentor teachers, *principals, *the
induction coordinator, *other school personnel, and *parents.




*FOCUS Project evaluator designed a survey for undergraduate CTFMS
participants that senior-year scholarship recipients will complete. *Contracted
independent evaluator (within the university) will design new surveys for
scholarship recipients: 1) in their credential year and 2) in their new teacher-
induction year.




*Survey data has recently been collected from the second cohort of Noyce
Fellows who are earning their credential this year.




*Email interview with cohort 1; develop questions on first semester of teaching;
personal interview with cohort 2, detailed questionnaire for cohort 1; data on
satisfaction with whole program; email interviews with cohort 2; detailed
questionnaire to cohort 2; emailed exit interview
*Survey instruments (with different math and science forms) are completed: upon
entering the program, after completing certification program, and at the end of the
first induction year as a teacher. These instruments examine: locus of control,
general teaching beliefs, specific math or science teaching beliefs, perceptions of
self-efficacy in managing student behavior, commitment to teaching in urban
settings, perceptions of barriers that might prevent them from reaching urban
students
                                             Specific Analyses or Methodologies




*Qualitative semantic domain analysis over time *Compare the experience of these Scholars with teachers coming from more traditional
backgrounds. *Using Data gathered from two groups of non-Noyce mathematics teachers (in- and pre-service) in addition to Noyce
Scholars: (*Instruments to test mathematical knowledge of fundamental ideas including ability to use that knowledge to solve math
problems, *Instruments to test ability to apply math knowledge in teaching situations and to analyze student responses to math situations
and suggest appropriate course of action, and Observations of Scholar classroom instruction especially the view of math they portray to
students).
*Evaluate each project objective both quantitatively and qualitatively with the evaluation embedded within ongoing evals of the
OPTIONS program. *NOYCE objectives include: a) ID and recruit 10 graduate science students and/or post-graduate scientists each
year; b) Provide support, instruction, and experiences to retain all 40 Noyce Fellows; c) Create reporting and accountability system to
assure full participant compliance and recover stipends from defaulted participants; d) Evaluate short- and long-term impacts of project
activities, and track Noyce Scholars through their teaching careers. OPTIONS objectives include: a) Promote innovative natural science
preparation through partnership; b) Complete and deploy modular online instructional program; c) Provide opps for ATC candidates to
observe classrooms and gain practical teaching experiences fulfilling Texas ATC requirements; d) Promote ongoing professional
development of all OPTIONS participants entering teaching via ATC; e) Utilize external evaluator to formatively and summatively assess
project activities attending to short-term outcome indicators and longer-term implications; and f) disseminate results. *Specific
methodologies include:
ANOVA and repeated measures to monitor diffs between and among observed groups; Multiple regression to
 ID diffs in teaching behaviors among groups of observed teachers; Explore relationship among variables related to
classroom observations, science teaching efficacy beliefs and the science background of each OPTIONS
participant (vague).




*Pre- and post tests centered around Scholars *Focus group activities centered around the Scholars




*Formative and Summative Evaluation to determine programmatic strengths and weaknesses using enrolled and completed student data
collected via evaluator observations, focus groups with participants mailed surveys, emailed websurveys, and phone interviews




*Pre- and post-program administrations of STEBI and Stalish I Instruments to measure growth/extent of change *After 1 year of actual
teaching: randomly sample CSUSM credential graduates employed at 50-50 schools (experimental) and non-CSUSM credential
candidates employed at similar schools (control) to examine teacher effectiveness (using Salish I Instruments).




*Two external evaluators *Internal evaluation with both formative and summative components *California Teaching Performance
Expectations (to measure teaching performance) for the 2 years they are observed. *These multiple methods will enable triangulation of
data to determine effective and less effective aspects of the program.

*Intend to conduct a comparative study of pre-service teachers, student teachers, and first/second year teachers who are Noyce Fellows
and those who were not Noyce Fellows (analysis of great interest because believe that the Undergraduate Learning Assistant (ULA)
experience (what Noyce-eligible students participated in) leads to better prep of K-12 teachers).
*PI (who is experienced in case study and interpretive research) will design interviews and questionnaires to obtain and analyze rich data.




*Project manual detailing every component of designing, implementing, carrying out and evaluating the program.




*Obtain qualitative information on case studies of scholars *Evaluation conducted to assess the project’s effectiveness in attracting,
preparing, and retaining teacher candidates. *Evaluation facilitates collecting demographic data on, and tracking recipients as they fulfill
their teaching obligation in the Philadelphia Public Schools.




*Longitudinal research component to investigate how the alternative certification program prepares high quality teachers for urban school
environments. *Used mixed methods to determine the extent to which the project is successful in: a) increasing the retention rates of
UMEP grads teaching in urban schools; b) developing a set of indicators of teaching commitment to urban education; c) developing a set
of indicators of high quality mathematics teachers in urban settings; d) identifying the variables that contribute to the disconnect the
preparation of mathematics teachers and their induction experiences in urban schools; e) identifying what new teachers need for the next
phase of career development (moving from induction to advanced professional training).




*External evaluation (by Dr. Dia Sekayi) including developing formative assessment instruments, analyzing data, communicating and
sharing data, determining whether goals have been achieved, identifying indicators for successes and shortcomings, evaluating
management roles and responsibilities, conducting longitudinal study of progress of teacher candidates
*Assess the value of problem-based learning (PBL) in urban settings. *Some discrete evaluation tools will be developed to assess
targeted teacher training in urban environments and in PBL teaching strategies. *ISTEP+ and MAP exam performance will be monitored
for the students taught by Noyce Scholars and compared with district-wide results.




*Will be employing an independent individual to conduct the evaluation of the program

*External Evaluation Consultant - an evaluation expert will advise the design of the permanent evaluation system and provide written
recommendations.


*Evaluation of the progress of the establishment of the database for tracking students.




*Spring 2005 data are tabulated, but not summarized (first year to include Noyce scholarship graduates now teaching) *(Fall 2004)
Classroom observations of scholar graduates in their 2nd year of teaching (using the NSF-CETP Core Observation Protocol) *Since
2005-2006 is the first year with Noyce graduates in their 2nd year of teaching (a limited # of these Noyce graduates may be added to the
observation pool)




*Texas Examination of Educator Standards (TExES): State test of content and pedagogy. Pre-test (sample) scores are used to determine
strengths and weaknesses and later compared with actual scores. *Using Texas Assessment of Knowledge and Skills (TAKS) in an
effectiveness comparison: a stratified random sample of scores from students whose teachers participated in the program will be
compared with students whose teachers did not participate in the program.




*Use of both qualitative and quantitative measures
*Compare our (the 9 Texas A&M system partners) data with data collected by the Texas Higher Education Coordinating Board’s
“Closing the Gap” effort to increase graduates in critical fields such as mathematics and science




*Focus groups: with cohort and non-cohort students as well as students with other teaching majors, to determine the reasons math or
science was or was not selected as their major (data will be entered in an Access database).




*Contracted independent evaluator (within the university) will analyze and synthesize all of the eval data and write annual reports with
formative results and a final report with summative conclusions.
*Identify changes in survey responses over time.
                                            Research Questions/Evaluation Goals


*Evaluated program in terms of its ability to attract, prepare and retain STEM career-change individuals to teaching careers in the
Chicago Public Schools. *Both formative and summative processes and products




*Process and product-oriented evaluation activities to determine how successfully the three objectives for the project have been attained:
Objective 1) Provide scholarships for STEM Juniors and Seniors and stipends for STEM post-baccalaureates who are willing to teach
science and/or mathematics in high-need school districts; Objective 2) Provide a focused teacher certification program through
collaboration among the College of Education and Human Development, the College of Sciences, the College of Engineering, and the
Honors College; Objective 3) Provide a support system for new teachers in the high-need school districts through a partnership between
UTSA and collaborating high-need districts.




*What are the effects of the increased mathematical content knowledge of pre-service teachers in terms of teacher pedagogical content
knowledge, teacher practice, and student learning? *How can the content courses be improved to inculcate pedagogical content
knowledge and produce an appreciation for it so that it is used in the classroom?
How can the content courses be improved to inculcate pedagogical content knowledge and produce an appreciation for it so that it is used in the classroom?
1. Are partners able to recruit 10 science graduate students/professionals to participate in OPTIONS as Noyce Fellows? In what ways do applicants vary in terms of science field,
previous teaching experience and current position? Do these differences influence their success as teachers?
2. What are levels of satisfaction with the online instructional modules regarding access, content appropriateness and difficulty, functionality, relevance of information, etc.? Do
participants demonstrate mastery of content in each module using online assessment tools incorporated into the modules? Are some modules more effective than others, and in what
ways?
3. Do the preliminary teaching experiences provide an appropriate introduction to secondary science teaching? Does the mentored teaching internship help participants develop
classroom teaching skills?
4. Do OPTIONS participants, in general, and Noyce Fellows, in particular, demonstrate improved teaching skills (demonstrate more “expert” behaviors) over the course of the one-
year internship, as evidenced by
classroom observations? In which teaching areas do participants demonstrate greatest
improvement? Least improvement? Which programmatic variables are most related to the
development of teaching expertise by participants? How many participants complete all
six modules, take the TExES, and complete teaching internships in satisfaction of the
ATC requirements? What factors contribute to failure or success of OPTIONS
participants in completing all ATC requirements? Is a clear strategy for addressing the
ongoing professional development needs of participants articulated, with sequential
activities and related resources targeted for development?
5. Do the Noyce Fellows persist as teachers in high need schools for the required two
years? What are the most challenging aspects of working in high need schools for Noyce
Fellows? How is the OPTIONS program adjusted in response to feedback from Noyce
Fellows? If any Fellows fail to fulfill their teaching obligations are appropriate steps
*Was preservice education enhanced? *Has the minority enrollment in the preservice program increased? *Is the innovative teacher
preparation program attracting a different type of student? *How do preservice students perceive the program, and does that perception
change? *FCEPT surveys will also be used and SMEC will survey principals of schools where students from the program are placed.




*Eval focus is to determine extent, nature, and adequacy of support received by program participants *Eval objective is to assess overall
efficacy of Noyce Scholarship Program in producing practicing secondary science teachers.
*All gathered data based on the Knowledge, Skills, and Dispositions (KSDs): 1) demonstrated knowledge of the subject area(s) in there
area of licensure; 2) effective application of a variety of content area relevant teaching strategies; 3) understanding of the needs of diverse
learners; 4) knowledge of the basic principles of child and adolescent psychology; 5) knowledge of the philosophical, historical, and
societal roots and foundations of education; 6) knowledge and understanding of the ethical, legal, and policy issues informing current
educational debates; 7) knowledge and understanding of curriculum issues informing current debates; 8) collaborative work with family
members, school colleagues, and community resources for benefit of all learners; 9) development and implementation of effective
methods for classroom management; 10) development and implementation of effective methods for planning lessons and units; 11)
development and implementation of fair an equitable assessment system; 12) knowledge and understanding of uses of technology to
facilitate teaching; 13) presentation of the characteristics of professional teachers and emerging leaders; 14) practice of regular reflection;
*Specific evaluation questions include:
• To what extent has the UMEP been successful in preparing high-quality mathematics teachers in urban schools? • To what extent has
UMEP been successful in preparing teachers who are committed to teaching in urban schools? • Which components of these of program
do graduates perceive as being most effective? • Which program components were perceived to be most effective for different types of
students (based on undergraduate major, age, gender, ethnicity, etc.)? • What do the UMEP graduates perceive to be the major barriers to
their development as high quality mathematics teachers in urban schools? • What do the UMEP graduates perceive to be the major
barriers to being committed to remaining in urban schools for three years or more. • What additional professional development activities
were identified and implemented during the project? • How actively did master teachers mentor new UMEP graduates in their schools?
*Summative evaluation questions:
• To what extent did UMEP meet its goal? Specifically, to what extent was the project able to:
-Increase the retention rates of UMEP graduates teaching in urban schools; -Develop a set of indicators of teacher “commitment” to
urban education; -Develop a set of
indicators of “high quality mathematics teachers in urban settings;” -Identify the variables that contribute to the “disconnect” between the
preparation of mathematics
teachers and their induction experiences in urban schools; -Identify what new teachers need for the next phase of career development, as
they move from induction to
advanced professional training;
*A measurable objective of the projects goals is: • feel that you have the project findings through papers atweeks of teaching? What are
*Mailed surveys with the following items: Do you Dissemination of been successful during your first six professional conferences,
some of the strategies and techniques you learned in the “Science and Mathematics for All” program that have helped you to be
successful? What have been your challenges during your first six weeks in your teaching position? How were you prepared for such
challenges in the “Science and Mathematics for All” program? What kind of support could you use going forward in your teaching
career? How can the “Science and Mathematics for All” program assist in the support you need?
*The key question addressed with continual assessment and improvement of the Noyce Scholars Program is: to what extent has the
TxCETP Noyce Scholars Program enhanced the number of outstanding undergraduate mathematics and science students committed to
and well prepared for a teaching career?




*Three preparation phase evaluation to examine: *recruitment strategies to attract a diverse and talented pool of STEM majors to early
teacher preparation activities, to a teacher credential program, and to a teaching position in a high-need district; *preparation of
scholarship recipients for success in teaching in terms of knowledge and use of subject matter expertise, effective teacher strategies for all
learners, and strategies for engaging in teacher inquiry; *preparation of scholarship recipients to take the initiative, and a leadership role,
in examining and sharing knowledge and strategies with peers; *success in sustaining scholarship recipients’ commitment to teaching and
to a district or school.




*Evaluation addresses: a) the importance of the financial assistance in the decision for students to come to UCLA; b) the environment
they experienced as undergrads here at UCLA.


1) Evaluate the effectiveness of STEM Bridge in attracting, preparing, and retaining STEM individuals in teaching.
2) Evaluate the effectiveness of Noyce scholarship recipients as teachers.
3) Provide formative information for the improvement of the STEM Bridge project.
4) Track scholarship recipients during the period in which they are fulfilling their service obligation.
*Attitudinal change over time.
             School/District Recruitment


*Recruitment data for Noyce recipients is tracked across the
two years of the service obligation.
Impact of Noyce Fellowships in recruiting qualified teachers




*Employment counts of the number of Noyce teachers in low-
income districts (see "summative program effectiveness data"
column)
*Number of Noyce Scholars who obtain math/science teaching
positions in high need districts.




*Semi-annual observations, interviews and surveys with
Noyce Scholars. *Semi-annual interviews and surveys with
school personnel and principals




*Increased recruitment of teachers from minority and
disadvantaged populations into partner schools.
*External evaluator designed plan to examine the recruitment
of high quality mathematics and science teachers for high need
districts.
*Data on entry into teaching (percent of students who obtain
teaching jobs in designated areas)




*Survey data on commitment to teaching in urban settings.
                             School/District Retention


*Retention data for Noyce recipients is tracked across the two years of the service
obligation.
*Impact of Noyce Fellowships in retaining qualified teachers




*Number of years (beyond obligation) Noyce teachers remain in low-performing schools




*Evaluate success in attracting, preparing, and retaining Noyce Scholars in teaching careers
via program manager, PI, and co-PI annual extraction from DB and education plans to
determine retention, attrition, and timely completion of a degree or credential




*Number of teachers retained in the teaching field for at least two years




*Continued employment counts of teachers, especially of color (see "summative program
effectiveness data" column)
*Data collected on personal and contextual variables contributing to retention. *Work with
Noyce teachers to ID support mechanisms that work and fail to retain teachers in high
needs districts/schools.




*Number of Scholars who maintain these positions beyond required time (records
maintained until 2014).




*Annual interviews, surveys, and examination of documented retention data (school
records) of Noyce scholars and professionals as well as Non-Noyce teachers in undergrad
and alt cert programs (to enable comparisons (i.e., Noyce vs. Non-Noyce assisted and
Noyce scholars vs. Noyce professionals).




*Retention data on TEEMS teachers across the metro-Atlanta area. *Effectiveness data on
the UMEP program in retaining individuals in teaching careers.

*Measurable objectives of the projects goals are: • An increase in the retention rate of
UMEP graduates in urban schools; • A list of valid and reliable performance measures that
indicate commitment to urban education; • A list of attitudes, values, and beliefs that
correlate with commitment to urban education; • A list of mutable variables that contribute
to the perception of a disconnect between teacher preparation and induction; • A decrease
in TEEMS graduates perception of a disconnect between their preparation and induction
experiences;




*Improved teacher retention within partner schools on an annual, five-and ten-year period.
*2002-2003 academic year information indicated that: 100% of 1st cohort graduates
continued into their third year of teaching;
100% of 2nd cohort graduates returned for their 2nd year of teaching.




*External evaluator designed plan to examine teacher retention.




*Recipient self-report on factors/events that could lead to leaving teaching; *Recipient self-
report on whether s/he anticipates remaining in teaching for at least the next 5 years.
*See "summative program effectiveness data" column
*Retention in teaching (percent of program completers still teaching with a three-year time
period and after a five-year time period)




*Survey data on perceived efficacy in managing student behavior, commitment to teaching
teaching in urban settings, and perceptions of barriers to reaching children in urban settings.
               Additional Notes




Realize that gains in student learning represent the
 most compelling evidence of the effectiveness of
 Noyce scholarship recipients and would welcome
future research partnerships to obtain this type of
                         data.




    Uncertainty about retention data gathering




 One first year graduate; planned longitudinal
                     analysis
Grad student data analyst; annual formative eval
             reports; evaluator funds
Noyce scholarships are described as just a means to
 bring in more competitive candidates for existing
    programs with funding already in place for
  delivering instruction and support as well as for
      evaluating the OPTIONS ATC program.
*How do they, and how will we, disentangle Noyce
           effects from OPTIONS effects?




           Evaluation timeline available


  A clearly specified agreement will ensure that
 Noyce Scholars understand their obligations and
                    commitment




External evaluator role and misnamed instrument




 Provided "basic" plan with further eval plan info
 available; Noyce here is an extension of another
                     program
 Part of the eval plan was described as no longer
 planned, but the exact section was not specified.




 By 2004 GIS project should be fully operational
 with "map" to guide stakeholder queries. Noyce
LISTSERV expected to help contribute information
                to the database.




 Still seeking an external evaluator to develop and
   implement a mixed methods evaluation of the
                       project
 Have not begun eval process as only in 2nd year
     with only 4 scholars in teaching positions
 An evaluation expert will advise the design of the
 permanent evaluation system and provide written
                 recommendations.




    Eval plan references a "strategic plan" that
   provides specific evaluation and benchmark
  indicators, but that portion of the plan was not
                      received.


Evaluation (as provided) is based entirely on survey
                    instruments
Noyce administrative funds will be used to hire a
master teacher to facilitate weekly seminars for
                novice teachers.




      More detailed description of CFTMS




UCLA has opened a Teaching Resource Center to
promote science (and math) teaching as a visible
       and accessible option for majors.
 Type of                                        Program Recruitment Data                                                                             Demographic Data                                                               Noyce Student Performance in Program                                                                                    Program Retention Data
Response

   Plan     R--N (by cohort for each program) and total N                                                                 *A summary of demographic data and statistics on stipend recipients. R--                                                                                                                  R--Examination of: 1) attriters as %age of total recruited; 2) %age of recruits of each gender and then
                                                                                                                          Collective profile information disaggregated for “applicants” and “teacher                                                                                                                attriters by %age of gender; 3) %age of recruits of each ethnicity and then attriters by %age of ethnicity;
                                                                                                                          interns”: Age, Gender, Ethnicity, Degree Level, major, discipline related                                                                                                                 4) %age of recruits at each age range and then %age of attriters at each age range; 5) timing of attrition.
                                                                                                                          career experience




   Plan     *Track number of scholarships and stipends actually awarded (as a partial reflection of the quality of        R--Percentage of students from historically underrepresented groups (i.e.,
            recruitment efforts) *Semi-annual interviews with both students and recruiters focused on identifying         African American, Asian/Pacific Islander, Hispanic, and Native American)
            and measuring the level of efforts associated with the recruitment and the response of those recruited        enrolled in UTeach introductory courses; R--chart detailing student
            (including students who decline the scholarship offer); R--chart detailing student recruitment efforts that   recruitment efforts that lists the UTeach student, high school attended, and
            lists the UTeach student, high school attended, and the city; R--number of students enrolled in the           the city
            introductory courses (and percentage increase this represents);




   Plan     R--enrollment history of the UTeach program (Fall 1997 to Fall 2002) including: “new recruits,”               *Demographic data on all program graduates; R--%age of minority           *GPA data on Noyce students; R--Time-to-graduate for UTeach graduates who entered as freshman                   R--%age retained early in program, R--retention ranges (conservative to liberal) by year in school, R--number
            “enrollment,” and “graduated.” R--Seat availability in introductory field courses                             enrollment for UTeach vs. College of Natural Sciences (CNS) and                                                                                                                           of students leaving after completing specific numbers of education courses, R--attrition rates by semester, R--
                                                                                                                          University (U), R--%age of students by gender, R--classification of                                                                                                                       profiles of UTeach leavers (# and % by gender, ethnicity, year of UTeach program entry, overall departure
                                                                                                                          students by year in school, college, and major, R--distribution of UTeach                                                                                                                 GPA, and destination of leavers);
                                                                                                                          students' GPA compared with CNS and U student distributions; R--UTeach
                                                                                                                          graduates’ demographics (number and percentage by gender, ethnicity, year
                                                                                                                          of entry into UTeach, and overall GPA); R--SAT II Math scores of
                                                                                                                          undergraduate UTeach Scholars
   Plan     Student N applying for vs. awarded Noyce Scholarships                                                         e.g., test scores, grades, gender, race/ethnicity, disabilities




 Overview                                                                                                                 Evaluation of numbers on graduate recipient and preliminary examination
                                                                                                                          on the undergraduates


   Plan     Adequacy of Pima CC and U of AZ partnership for producing appropriate numbers of applicants from              Noyce applicant characteristics                                                *Scholar performance in science, math, and pedagogy courses & field experiences                            *Noyce teacher likelihood of departure (vs. peers in CoS TPP and SEPM)
            both institutions.

   Plan                                                                                                                                                                                                  *Review of Records: and individual progress by outside consultants and faculty personnel to determine if   Annual interviews with scholars during the commitment period to ID and act upon retention-related problems
                                                                                                                                                                                                         progress toward graduation is adequate. *Longitudinal study on a chosen sample of Scholars of their
                                                                                                                                                                                                         academic progress throughout their program




   Plan




   Plan     *Documentation of how recruited and potential Noyce students learned of the program and examination           *Regularly updated database of demographic data and statistics for all      *Lack capacity to run separate evaluation of Noyce students in credential program
            of recruitment processes to determine strategies with greatest success                                        Noyce scholarship/stipend recipients *Continued documenation of the
                                                                                                                          demograhics (including ethnicity) of recruited and potential Noyce students


   Plan     *Internal tracking of specific outcomes such as the recruitment of Scholars                                                                                                                  *Internal tracking of the progression of Scholars to careers as middle or high school science or math      Internal tracking of the retention of Schlolars
                                                                                                                                                                                                         teachers




   Plan     *Review of recruitment strategies




   Plan     *Evaluate success in attracting, preparing, and retaining Noyce Scholars in teaching careers via program                                                                                     *Qualitative assessment of scholar academic progress via Noyce education plans with single-subject
            manager, PI, and co-PI annual extraction from DB and education plans to determine retention, attrition,                                                                                      advisors responsible for monitoring progress of all Noyce scholars *Project manager-created DB
            and timely completion of a degree or credential                                                                                                                                              containing links to Scholars’ Noyce education plans, demographic info from application forms, GPA,
                                                                                                                                                                                                         financial circumstances, …and be updated via any new contact data collected at monthly seminars
                                                                                                                                                                                                         *Evaluate success in attracting, preparing, and retaining Noyce Scholars in teaching careers via program
                                                                                                                                                                                                         manager, PI, and co-PI annual extraction from DB and education plans to determine retention, attrition,
                                                                                                                                                                                                         and timely completion of a degree or credential


   Plan     *Number of students recruited/selected to participate in program *Rigorous single subject selection           *Program client information *(Salish I instrument) to gather high school       *Number of students completing endorsement/credential requirements *Project director maintained
            process (initial basic requirements screening, group interview sessions, and a writing activity (choose       and college coursework, GPA, type of curriculum used, field experiences        database containing info on completion of credential program
            strength and weakness attributes) involving formative, self-reflective assessment using Attributes of         *Project Director (PD) maintains current database with pertinent
            Highly Effective Teachers (bulleted in Eval Plan Doc)                                                         demographic information




   Plan     *Number counts (numbers recruited, number of meetings, etc.) *Recruitment of math/science interns                                                                                            *Graduation and certification of scholars (see "summative program effectiveness data" column).
            (see "summative program effectiveness data" column).
  Plan                                                                                                                                                                                                         *Tracking of performance of Noyce fellows in courses within major




  Plan     *Gathering systematic data: to clarify best practices in recruiting students with different beginning levels                                                                                        *Oral defense of extensive portfolios based on rubrics rooted in principles of reflection and constructivist
           (i.e., undergraduate or graduate) and starting majors and to answer questions such as: how career                                                                                                   learning.
           changers learn about education programs, the best point in a career to enter teaching cert programs, the
           best point to begin to work with STEM undergrads to interest them in teaching in high needs
           districts/schools, whether particular kinds of science or math majors better prep students/professionals for
           teaching, and whether students interested in rural teaching vs. urban high needs differ.

  Plan     *Eval plan will provide effectiveness data of the program in attracting individuals to the program. *Each      *Percentage of scholars from underrepresented groups.                                *Advisor continued documentation of scholar progress toward degree completion and teacher certification *Enrollment services tracking of scholars each semester to determine continued enrollment. *Eval plan will
           semester: number of applicants eligible and/or receiving NSF scholarship. *Percentage of students                                                                                                   requirements. *Number of Noyce scholars making satisfactory progress during academic year.              provide effectiveness data of the program in retaining individuals in the program.
           accepted compared to total completing applications.                                                                                                                                                 *Percentage of students with 3.0 at end of each semester. *Number of Noyce Scholars obtaining the
                                                                                                                                                                                                               required degree/teaching certificate.




  Plan     *Effectiveness of recruitment strategies                                                                       *Compiled demographic data from applications. *Obtain demographic                    *Data on student performance in courswork.
                                                                                                                          data on scholarship recipients




  Plan     *Applicant evaluation (prior to admission)                                                                     *KSDs (see"research questions/evaluation goals" column) measured: 1-3, 8-            *KSDs (see"research questions/evaluation goals" column) measured: 1-5, 7, 12-14, 16. *Course
                                                                                                                          10, 13-16. *Pre-interview qualitative content analysis of the written                instructor content analyses of: MAT 303 (Unit & Lesson Outlines), MAT 302 & 303 (Candidate Self-
                                                                                                                          application; *Qualitative evaluation of formal interview responses; *Use of          reflection), MAT 302 & 303 (Initial Teaching Philosophy Statement), MAT 303 (State/National Content
                                                                                                                          transcript to assess coursework and performance (esp. in subject-area                Area Standards Analysis). *Initial Technology Assessment (survey and demonstration) assessed by
                                                                                                                          courses); *Applicant GPA; Applicant GRE scores; Qualitative evaluation of            Program Director. *Program director use of transcript analysis (mid-internship) to review Noyce student
                                                                                                                          recommendation(s) and applicant personal statement to determine specific             progress in graduate content courses.
                                                                                                                          KSD mastery


  Plan     *Effectiveness data on the UMEP program in attracting individuals for teaching careers. *Indicators of                                                                                              *Monitoring of UMEP students on ongoing basis to document progress toward degree completion and
           high quality teachers in urban settings (instrument) used to screen TEEMS applicants.                                                                                                               teacher certification requirements.




  Plan     *The number of teacher candidates admitted in the Master of Arts in Teaching (M.A.T.) degree and               *Additional demographic data and tracking of scholarship/stipend                                                                                                                                    *The number of teacher candidates retained in the Master of Arts in Teaching (M.A.T.) degree and
           endorsement programs                                                                                           recipients as they fulfill their teaching obligation (i.e., at least 5-10 years to                                                                                                                  endorsement programs. *The number of teacher candidates awarded graduate degrees or endorsements.
                                                                                                                          learn how many candidates became teachers in local public or private
                                                                                                                          schools and how many committed to science or mathematics instruction.




  Plan     *Data on: requests, how requesters heard about the program, and scholarship application submissions (to        *Data on scholarship application submissions to determine the                        *Monitoring of Scholar academic process using: GPA, the summative decision points within the ISU
           determine: 1) where the most interest lies and 2) where recruitment efforts should be stepped up               demographics of individual interested in STEM teaching                               teacher education curriculum, the senior portfolio, and ISU’s Unit Assessment System (using portfolio
                                                                                                                                                                                                               Live Text Software) *Conducted by (Indiana State University) ISU mentors to Scholars




  Plan




Overview



  Plan                                                                                                                                                                                                         *Continuous data collection on candidate qualifications and progress (including Praxis scores)




  Plan     *Effectiveness of the Noyce funding as a recruitment tool in relation to recruitment strategies developed  *Examine database data for impact of Noyce stipends on recruiting diverse                *Evaluation-documented timely progression of students through the certification program. *To examine           *Ongoing formative oversight of data collection addressing the effectivenes of the Noyce monetary support in
           within the SMAR2T, (CM)2, and Missouri Center programs. *Evaluation-documented selection process. students.                                                                                         candidate success through alternative certification programs such as SMAR2T: examine entering-to-              retaining students in alternative certification programs. *Noyce recipient interview data on impact of
            *Evaluation data collected via application and acceptance paperwork from the applicant and the                                                                                                     graduation ratio of Noyce stipend recipients and compare success rate of teacher placement between             stipends on decisions to stay in the alt cert program; *Compare program retention ratios: # of Noyce stipend
           university. *Ongoing, formative oversight of data collection addressing the recruitment and retention of                                                                                            Noyce students to others in SMAR2T programs.                                                                   vs. non-Noyce SMAR2T students receiving certification
           alternative certification candidates (i.e., number of students recruited into the program, annual
           comparison of number of Noyce-recruited versus number into other alt cert programs looking for Noyce
           stipend related trends). *Noyce recipient interview data on impact of stipends on decisions to stay in the
           alt cert program.
  Plan     *Advertising effectiveness: (a) Some effectiveness data regarding the ad campaign (designed by a private
           firm) by comparing hits on the web site to the advertising cycle (note: these data are not able to be
           disaggregated by Noyce scholar and other scholarship recipient website visits) (b) Continued collection of
           survey responses and anecdotal communications to determine the impact of marketing and advertising
           efforts on recruitment. *NSF and pre-service Noyce scholar responses on the influence the award had on
           their decision to enter into a teacher certification program
   Plan      *Based upon the demographic info on recruitment contacts and applicants to program                     *Based upon the demographic info on recruitment contacts and applicants         *Verification of course enrollment and verification of academic progress (1x per semester; 1x during
                                                                                                                    to program; *Based upon the demographic, retention, and progress                final coursework semester)
                                                                                                                    information of scholarship recipients.




   Plan




   Plan


   Plan      *Review of program literature and activities related to recruiting. *Use of interviews, surveys, and
             observations to evaluate Noyce scholar selection activities.



   Plan      *Data that identify how the program has attracted superior students into teacher preparation           *Data that identify how the program has helped students overcome           *Scholars’ portfolios charting their ability to develop quality mathematics and science learning
                                                                                                                    financial barriers and supported and increased the number of students from experiences *The academic advising program will also provide important data.
                                                                                                                    underrepresented minority groups *Student demographic data at program
                                                                                                                    entry




   Plan                                                                                                             Examples of demographic data collected include: *Age, ethnicity, gender,        *Progress of the cohort through: *completion of their course work, *field-based semesters, *full-time
                                                                                                                    disability, place of birth, marital status. *Number of dependents, spouses'     employment as a teacher until completion of service. *GPA
                                                                                                                    education and occupation, household income. *Employment (full¬time or
                                                                                                                    part-time), parents' education and occupation. *Educational levels of
                                                                                                                    siblings, GPA in courses on the degree plan. *GPA in math or science
                                                                                                                    major, Texas Higher Education Assessment (THEA) scores and other
                                                                                                                    standardized test scores. *Previous attendance at other colleges and GPA
                                                                                                                    from those colleges.
   Plan                                                                                                             *Records of demographic information for programs designed to recruit            *Records of attendance and participation for programs designed to recruit STEM majors. *Records of
                                                                                                                    STEM majors. *FOCUS Project evaluator-designed survey to collect                attendance and participation for all required and optional activities for scholarship recipients in pre-
                                                                                                                    information on the quantity, diversity, and quality of UCI students             credential, credential, and new teacher induction years. *University academic transcripts of scholarship
                                                                                                                    participating in the Community Teaching Fellowships In Math & Science           recipients. *Formative assessment performance information generated by instructors, advisors, and the
                                                                                                                    (CTFMS) program and and document teacher prep paths chosen by UCI               candidates themselves in in the credential program teaching events; *Dept of Ed Noyce Scholarship
                                                                                                                    students in the CTFMS program.                                                  Program co-coordinator will: *oversee data collection related to scholarship recipients and the program
                                                                                                                                                                                                    including data from the teacher credential program as well as *administer surveys to scholarship
                                                                                                                                                                                                    recipients.
   Plan      *Survey Scholars and quantitatively analyze data to determine whether the Noyce level of financial     *Document review of application packages and administrator records using
             support influenced their decision to enter teaching through this program.                              quantitative and qualitative methods to determine answers to questions
                                                                                                                    such as: a) what is the racial/ethnic makeup of the Scholars? b) what is the
                                                                                                                    gender makeup of the Scholars?

   Plan                                                                                                             *Pictorial files emailed to depict the diversity achieved within cohorts.




   Plan                                                                                                             *Data tracking (through the end of the obligation period) of demographic        *Data tracking (through the end of the obligation period) of teacher test scores and progress through the
                                                                                                                    information.                                                                    program. *Performance assessment data collected at: a) entry into the program, b) during the
                                                                                                                                                                                                    prepracticum, c) during student teaching, and d) at the end of student teaching.


   Plan                                                                                                             *Existing baseline data of course and major enrollments by race and             *Data to be collected: Noyce Scholars’ GPA                                                                  Data to be collected: UMB transfer rates for scholars vs. comparable students.
                                                                                                                    ethnicity, and Noyce Scholars program enrollments




Overview--
 Timeline




   Plan      *The number of students recruited to participate in the program                                        *Demographic characteristics of the recruited (e.g., ethnicity, race, gender,   *Level of education and program activity using: a) academic progress, b) student performance, c) course     *Ultimate achievement determined by the number of students admitted to and graduating from UNCP with
                                                                                                                    SES)                                                                            evaluation data. *Interim achievement determined by progress through each level of the educational          math and science education degrees (i.e., the number of students continuing in the program over the
                                                                                                                                                                                                    pathway. *Progression (operationalized as the percent of students completing the program within the         established period of time and percent completing all requirements for certification/licensure).
                                                                                                                                                                                                    established timeframe). *Performance (including GPA in all teacher education related courses).
                                                                                                                                                                                                    *Student Information Management Systems (SIMS) and Testing Offices will enable Co-PIs to ID and
                                                                                                                                                                                                    track all levels of student progress




   Plan                                                                                                             *Applicant form data: applicant experiences with children, reasons for          *Survey instruments (see "self-report data" column) are intended to identify changes in participants'
                                                                                                                    wanting to teach, hobbies, major, minor, GPA, ethnicity, and citizenship        beliefs as they progress through the program.
                                                                                                                    status.
                         Transition Experiences/Support for Teachers                                                                                              Monitoring of Noyce Teachers                                                                          School/District Characteristics                            Teaching
                                                                                                                                                                                                                                                                                                                                  Assignment
                                                                                                                                                                                                                                                                                                                                 Characteristics
                                                                                                                                                                                                                                                             *For the schools in which Noyce students are teaching:
                                                                                                                                                                                                                                                             summary involving academic and demographic data on the
                                                                                                                                                                                                                                                             school community. *Map depicting CPS school
                                                                                                                                                                                                                                                             demographics. *R--Geographical location, school data:
                                                                                                                                                                                                                                                             racial/ethnic composition, low-income rates of schools,
                                                                                                                                                                                                                                                             student mobility rate, reading, math, science standardized test
                                                                                                                                                                                                                                                             scores (at schools where MGM/MGS teachers taught)
*Quality of recruitment efforts as well as quality of collaboration with public school districts and
individual public school mentors measured via: a) survey responses of enrolled UTSA-UTeach students
regarding the quality of the mentoring program; b) survey responses of participating mentor teachers
regarding their commitment to the effort and quality of their mentorship. *Closely follow the kind and
degree of support provided by the University to its graduates once they are placed in participating high
need districts by: assessing the diversity of teachers produced by the program (as a reflection of the quality
of support provided by Noyce Scholars Promoters and the advisors). *Assessment of the processes for
interaction and support (provided to new teachers after leaving the project and beginning teaching in high-
need districts) will involve close tracking, through interviews and observation, a random sample of 20% of
each year's graduates and: a) a focus less on the quality of the teacher’s work and more on the degree to
which the new teacher is successfully supported in his or her new classroom environment; b) evaluation of
the University’s ongoing support efforts as well as
the efforts of the participating districts to nurture and support these new teachers.
*Have data on retention of new teachers in Austin Independent School District as a consequence of on-            *Personal visits or phone interviews with the UTeach induction coordinator or UTeach advisors to keep track of where UTeach                 R--school names and regions where UTeach graduates were           *Have conducted a
demand support program. Continue to offer support in first years to UTeach graduates, and measure                graduates are teaching (and if left teaching to acquire detailed information on the reasons for that decision). R--Certification profiles   teaching; R--range of starting salaries                           survey of UTeach
retention.                                                                                                       (i.e., mathematics, science, computer science); R--Graduates’ teaching statuses (i.e., currently teaching, entered but left teaching,                                                                         graduates (Noyce not
                                                                                                                 never entered teaching); R--UTeach current teachers’ profiles (i.e., teaching level, geographical location and school district);                                                                              broken out) in which
                                                                                                                                                                                                                                                                                                                               graduates reported on
                                                                                                                                                                                                                                                                                                                               teaching assignments.



                                                                                                                 *Location of school districts employing scholars                                                                                            *Nature and location of school districts employing scholars       *info on science teaching
                                                                                                                                                                                                                                                                                                                               assignments (e.g., grade
                                                                                                                                                                                                                                                                                                                               level(s) and subjects(s))




*Formative and/or summative question of: What are scholar challenges and successes in transitioning to           *Evaluator observation of Beginning Teacher Support meetings
teaching in high need schools? *Continued communication with scholars after program when beginning
teaching; *Evaluator observation of Beginning Teacher Support meetings
                                                                                                                 *Noyce grads included in yearly data collection by TEAM-math (Transforming East Alabama Mathematics) evaluation process




                                                                                                                 Track Noyce Scholars through their teaching careers




*Evaluation activities focused on scholar attitudes and beliefs about their science and math teaching                                                                                                                                                        *Internal tracking of the progression of Scholars to careers as
experiences                                                                                                                                                                                                                                                  middle or high school science or math teachers




                                                                                                                 *CSULA and LAUSD agreement and development of a system to jointly track new science teachers from CSULA




Twice-yearly surveys of scholars, advisors, and districts to asses: 1) the obstacles or difficulties
encountered, 2) the value of the supports provided




                                                                                                                 *Project director maintained database with information on scholar successful teaching for two years in an underserved secondary
                                                                                                                 school




                                                                                                                 *Interview, pre- and post questionnaires, focus groups, and field observations to determine student success and growth in the teaching
                                                                                                                 profession
                                                                                                              *Tracking of Noyce fellows (first three years of teaching) *Development of web-based tracking and reporting mechanism




*No details are offered on the data to be collected, but a belief is presented that Noyce money will enable   *All CTE grads currently tracked for at least 5 years to see if they remain in the profession.
students to work in high needs (often less-paying schools). Also Noyce money will fund a teacher support
network for teachers in rural high needs schools.




 *External evaluations: use qualitative and quantitative means to review effectiveness of mentorship          *Annual updating of contact information and work status. *Partner schools will provide: mentor teacher observations, daily                   *Pilot GIS project (linking annual report to governor and
component. *Enrollment in seminar to provide professional development and address new teacher needs.          evaluations of student teacher, key stakeholder involvement in training workshops and evaluation of student teachers                         legislature to student placement database at Dowling to
*Number of Noyce Teachers enrolled in Follow-up Seminar each Fall. *Number of Noyce Teachers who                                                                                                                                                           understand and improve placement of graduates within NY
are observed in their classrooms and who participate in follow-up consultation. *Number of Noyce                                                                                                                                                           school districts).
Teachers who participate in the Dowling Annual Science Symposium and Workshops. *Partner schools
will provide: mentor teacher observations, daily evaluations of student teacher, key stakeholder
involvement in training workshops and evaluation of student teachers
*Quarterly interviews and surveys (in years 2-4) with Noyce Scholars and Program Mentors to determine         *Compare Noyce and non-Noyce assisted teachers on measures such as:
the effectiveness of the mentoring. *Quarterly interviews and surveys (in years 3-4) with Noyce Scholars      entry and long-term retention (4-5 years)into teaching
to determine their evaluation of the Induction program.



                                                                                                              *Tracking of students 5 years following graduation




*Used mixed methods to determine the extent to which the project is successful in identifying the             *In the process of collecting data on the retention rate of TEEMS students across the metro-Atlanta area.
variables that contribute to the disconnection between the preparation of mathematics teachers and their
induction experiences in urban schools




*The number of teacher candidates mentored by teacher mentors                                                 *Additional demographic data and tracking of scholarship/stipend recipients as they fulfill their teaching obligation (i.e., at least 5-10   *Additional demographic data and tracking of
                                                                                                              years to learn how many candidates became teachers in local public or private schools and how many committed to science or                   scholarship/stipend recipients as they fulfill their teaching
                                                                                                              mathematics instruction.                                                                                                                     obligation (i.e., at least 5-10 years) to learn how many
                                                                                                                                                                                                                                                           candidates became teachers in local public or private schools
                                                                                                                                                                                                                                                           and how many committed to science or mathematics
                                                                                                                                                                                                                                                           instruction.


                                                                                                              *Monitoring of service obligation. (Phase I): Mentor assessments of scholars, Scholar annual survey responses about the impact on
                                                                                                              their professional teaching of targeted teacher preparation in the urban environment and with Problem-Based Learning (PBL)
                                                                                                              strategies. (Phase II): Data on retention of Noyce Scholars during the two-year service obligation period, Request for Scholars to keep
                                                                                                              current contact information with ISU for 5 years, and Annual survey of scholars to determine: retention in the teaching profession
                                                                                                              within a STEM discipline, the importance of the Noyce Scholars Program to Scholar current teaching, and with additional information
                                                                                                              gathered when a teacher leaves the profession. *Management Team reports on retention (annually, year 4 final, and 5 years post-
                                                                                                              program).




                                                                                                              *End of the school year interview conducted with successful candidates from the first cohort during year 1 of teaching.




                                                                                                              *Success of students becoming certified as determined via: (a) The number of teachers remaining in teaching through Noyce funding
                                                                                                              term (Teachers tracked through alt cert program with comparisons between % of Noyce students staying in teaching and % of either
                                                                                                              alt cert, but no Noyce funds or traditionally certified)
*External evaluator plan to examine teacher candidate perceptions (surveyed once during teaching year 1
and including Likert Scale and anecdotal information) regarding the efficacy of: multilayered mentoring
support system to bridge transition to teaching, support during the induction year of teaching, and ongoing
issues related to teacher support, resources, professional growth, etc.




*Evaluation to assess mentor to induction year teacher relationships.




*Recipient self-report on challenges/frustrations encountered or expected to encounter in early years of      *Dept chair at Recipient's K-12 school reports on the maturity level and rate of progression of the Recipient
teaching.

*Phone interviews, observations, and documentation to evaluate the extent to which field experiences and      *Online survey evaluation of Noyce student teaching.
student teaching are enabling students to acquire the skills needed for teaching. *Surveys, interview, and
observations to evaluate the extent to which Stony Brook and partner site mentors are initiating Noyce
students into the demands and culture of teaching and school work.
*Cross-university study of the effectiveness of induction year experiences to generate within- and between- *Existing tracking procedures for other scholarships will be used to monitor Noyce Scholars
campus data to serve as a baseline for continuing study and long-term tracking of Noyce students in the
workforce *Quantitative (i.e., number and diversity of teachers retained, how long they remain in the
classroom) and qualitative data will be gathered *These data will involve surveys, focus groups, and
selected follow up interviews of Noyce Scholars *Induction Year program development is an institutional
focus for all TxCETP partner universities, and this area of research and evaluation will extend beyond the
four years of the TxCETP Noyce Scholars program.

*Induction needs assessment, information on new hires, and entering addresses for teachers changing           *Tracking system to maintain contact with scholarship students following graduation. *Alumni Association info on current addresses
positions (gathered by induction coordinator). *Email listserv to provide current info on activities in the   of graduates. *Progress of the cohort through full-time employment as a teacher until completion of service.
schools, with the teachers, and with the math and science departments at A&M-Texarkana




                                                                                                              *University academic transcripts of scholarship recipients. *Formative assessment performance information generated by instructors,
                                                                                                              advisors, supervising teachers, mentors, and the candidates themselves in the future teacher inquiry workshops, in the credential
                                                                                                              program teaching events, in student teaching mentor-novice coaching sessions, and in new teacher induction program activities.
                                                                                                              *Descriptions of professional development activities in scholarship recipients’ mini-grant proposals. *FOCUS Project evaluator-
                                                                                                              designed survey to track these students' progress in securing teaching positions in or out of the region. *Dept of Ed Noyce Scholarship
                                                                                                              Program co-coordinator will: *oversee data collection related to scholarship recipients and the program including data from the teacher
                                                                                                              credential program, partner school districts, and induction programs as well as *administer surveys to scholarship recipients.

*Use quantitative and qualitative methodologies in conjunction with surveys, focus groups, interviews,
mentor log doc reviews, and supervisor doc reviews with/from Scholars, school mentors, and university
supervisors to answers questions such as: a) to what extent are the Scholars mentored in the on-the-job
paid internship? and b) with what kinds of competencies did the mentors have to assist (classroom
management, curriculum, cultural)?




                                                                                                              *Data tracking (through the end of the obligation period) of teaching positions during the post-licensure induction phase




                                                                                                              *Student Information Management Systems (SIMS) and Testing Offices will enable Co-PIs to ID and track all levels of student
                                                                                                              progress




                                                                                                              *Survey instruments (see "self-report data" column) are intended to identify changes in participants' beliefs after being in the schools.
                       Coordination between Programs or Institutions                                                                                                                         Noyce Teacher Effectiveness Data                                                                                                                             Fulfillment of Scholarship Requirements


                                                                                                              *Ongoing focus on quality of teacher performance in the schools throughout service obligation period and collected via: a) surveys (completed online by teacher interns and school administrators); b)       *Summary of data and statistics on stipend recipients including employment tracking during the service obligation
                                                                                                              letters of recommendation from school and university faculty who observed the teacher interns in the classroom; c) standards-based observation protocols (e.g., Arizona Observation Protocol) to             to the Chicago Public Schools (CPS) involving academic and demographic data on the school community. The
                                                                                                              evaluate classroom performance.                                                                                                                                                                              tracking system will retain recruitment, employment, and retention data across the years of the service obligation.




*Quality of collaboration between UTSA and the public school districts and individual public school           *A selected series of follow-up interviews and observations of Noyce Scholars in public school classrooms; R--Mentor teacher reviews of UTeach students (e.g., enthusiasm and level of preparation);
mentors; R--school districts, names, and campuses of the school teachers serving as mentors; R--Location
of coordinated efforts with public school campuses as detailed by a chart depicting the campus, school
district, and mentor teacher involved with UTeach; R--Program coordination with Industry via a chart
detailing the driver industry, representative, and affiliation information for partnerships with the UTeach
program.




                                                                                                              *Noyce student performance data on state certification examinations *Noyce teacher effectiveness via visits to each during the first year of teaching utilizing the Fitness to Teach evaluation              *Reports to NSF
                                                                                                              instrument (developed for the UTeach program and available for trial studies in Fall 2003 and available for program-wide use in Spring 2004). R--UTeach student performance on the ExCET and
                                                                                                              TeXeS Exams (subdivided by content and pedagogy tests and by “passed initially,” “failed,” and “passed with retest.” R--Average total score performance on content and pedagogy exams by content
                                                                                                              specialization (i.e., mathematics, science, computer science).




                                                                                                                                                                                                                                                                                                                           *Info on recipients not meeting requirements




*Examination of how the Pima CC and U of Arizona partnerships works


                                                                                                              *Co-PIs eval of Noyce graduates' success in implementing learned teaching strategies *Math achievement of Scholar-taught students (state-mandated ARMT and tests to measure students'
                                                                                                              achievement of big ideas taught to scholars) *Examination of the impact of increased mathematical content knowledge on teacher pedagogical content knowledge, teacher practice, and student
                                                                                                              learning




                                                                                                              *Stallings Observational System (SOS) data on OPTIONS (of which Noyce are subset) vs. HISD (standard certification) teachers to compare timepoints when they begin implementing classroom
                                                                                                              behaviors typical of more expert science teachers and which behaviors emerge. *Portfolios: containing goal statement, annotated evidence, and discussion of progress toward a self-specified science
                                                                                                              teaching goal; these periodically-reviewed portfolios provide a formative assessment as well as summative evaluation of participant development of teaching skills. *Observations: 1) within the
                                                                                                              classroom via the district mentoring program; 2) also 4 additional classroom obs. occasions using the Stallings Observation System (SOS) that uses Microsoft Access to build teacher profiles based
                                                                                                              upon: a) "classroom snapshot” data: the types of activities the teacher initiates with students and b) “five-minute interaction” data: the types of interactions the teacher has with students (asking Qs,
                                                                                                              promoting discussion, lecturing, praising, inquiry, guiding exploration, etc.).




*Evaluation of undergrad scholars linked to any activities/experience or course/institute evaluations done    *4 annual observations at Noyce teachers sites using university-developed measures of teacher effectiveness *Discussions with school administrators for perceptions on teacher performance.                  *Database on all Noyce teachers while fulfilling obligation in high-need schools
on a regular basis




*CSULA and LAUSD agreement to jointly monitor CSULA new science teacher effectiveness                         *CSULA and LAUSD agreement to jointly monitor CSULA new science teacher effectiveness




                                                                                                              *For Noyce scholars in the classroom: quarterly mentor reports assessing scholar performance and attitude *For district interns: university field supervisor report of effectiveness *For scholars           *Education plans and monthly seminars to reinforce and track scholars’ compliance with program requirements
                                                                                                              fulfilling 2-year commitment: districts will provide data on student performance on standardized test scores for scholars’ students versus similar students *Effectiveness data will be assembled and        during their studies and their subsequent teaching commitment *DB containing info on attendance at Noyce
                                                                                                              analyzed annually by the program manager, PI, and co-PI to determine the effectiveness of the Noyce Scholars in the classroom.                                                                               orientation and monthly seminars, filing of reports from school districts where Scholars teach *Program manager
                                                                                                                                                                                                                                                                                                                           and advisory committee (includes district representatives) created reporting form to track scholar compliance as
                                                                                                                                                                                                                                                                                                                           interns or teachers *Quarterly school principal indication of scholar job performance and whether it justifies
                                                                                                                                                                                                                                                                                                                           continued employment *HR dept confirmation of scholar employment *Mentor report of regular scholar
                                                                                                                                                                                                                                                                                                                           attendance at meetings with mentor


                                                                                                              *Formal and informal data gathering *California State University Systemwide Evaluation of Professional Teacher Preparation to evaluate the effectiveness of CSU grads who held K-12 teaching
                                                                                                              positions for > 1 year and gather administrator opinions (results are reported as composites with longitudinal data system-wide or disaggregated by individual campus).
*Tracking Noyce fellows for first three years of teaching using Teacher Education Program mechanisms
already in place




                                                                                                        *Supervisor performance evaluations during first 2 years of teaching. * Pupil performance on federally required standardized science and math tests (and comparisons with non-Noyce pupils)
                                                                                                        *Examination of conduciveness of classroom arrangement to constructivist learning. *Samples of Noyce teacher assignments and student performance (and comparsions with non-Noyce).
                                                                                                        *Effectiveness of preparation gauged by Noyce teacher integration of content and pedagogical knowledge to develop learning activities that enable pupils to: learn how to learn and monitor
                                                                                                        understanding of STEM conceptions, carry out meaningful investigations of key topics, work productively with others on meaningful learning and inquiry tasks, find STEM content relevant,
                                                                                                        worthwhile, and interesting. *Performance on NY State required prospective teacher tests assessing content and teaching knowledge.


*Partner schools will provide: mentor teacher observations, daily evaluations of student teacher, key   *While enrolled at Dowling: field-based experiences and pedagogical and content knowledge training measured via field supervisor implementation of the Framework for Teaching classroom
stakeholder involvement in training workshops and evaluation of student teachers                        observation rubric. *Noyce teacher ability to improve student learning (using portfolio and student work). *Internal evaluations (review scores from observation rubric and Noyce Teacher
                                                                                                        Assessment Portfolio). From academic year 2001-2002 on: a comprehensive database with enrollment trends; indicators of student achievement in reading, writing, mathematics, science, and
                                                                                                        vocational courses; graduation, college attendance and employment rates; *Partner schools will provide: mentor teacher observations, daily evaluations of student teacher, and evaluation of student
                                                                                                        teachers



                                                                                                        *Quarterly evaluation of quality of teaching (observations, evaluation of classroom performance). *Compare Scholars and Professionals as teachers with respect to learning outcomes. *Compare
                                                                                                        Noyce and non-Noyce assisted teachers on performance on the NTE exam required for licensure. *Semi-annual student achievement scores and quarterly student classroom performance data.




                                                                                                        *Note: some data described in the "summative program effectiveness" column may also be applicable here. *KSDs (see"research questions/evaluation goals" column) measured: 1-4, 7-15. *State
                                                                                                        licensure application results; *Praxis II results; *Licensure coordinator's rubric-based examination of the 1st year teacher survey; *Program director and faculty advisory committee rubric-based
                                                                                                        examination of the Hiring Principal Survey; 4) subject-area methods instructor and mentor teacher rubric-based evaluations of Noyce teacher unit plans using SET; 5) course instructor evaluation of
                                                                                                        Noyce teacher IEP Project paper and presentation;




                                                                                                        *Measurable objectives of the projects goals are: A list of valid and reliable performance measures that indicate a highly qualified mathematics teacher in urban schools; • A list of attitudes, values,
                                                                                                        and beliefs that correlate with characteristics of a highly qualified mathematics teacher in urban schools; *Effectiveness data on TEEMS teachers across the metro-Atlanta area; *Effectiveness data of
                                                                                                        UMEP teachers in the classroom.
                                                                                                        *Develop a set of indicators (instrument) of high quality teachers in urban settings using several phases and urban teaching TEEMS graduates with > 3 years of teaching experience: *Phase 1:
                                                                                                        Conduct individual interviews using a narrative storytelling research method in order to develop preliminary indicators; *Phase 2: Focus group with TEEMS grads to refine and add depth to
                                                                                                        indicators; *Phase 3: Survey of secondary mathematics teachers in urban settings for wider range of feedback (results in semi-final set of indicators); *Phase 4: Field test with high quality urban
                                                                                                        secondary mathematics teachers identified by another means (results in final version of instrument)




                                                                                                        *The number of teacher candidates (M.A.T. or endorsement) that improve scores on the standardized tests in Maryland.




                                                                                                        *Scholar impact on students as teachers: Baseline student demographic data, Academic information using Indiana's ISTEP+ exam results in Math and Science and the NWEA Measures of Academic
                                                                                                        Progress (MAP) test scores (math only) for participating IPS sites (ISTEP scores can be used for state-wide comparison, MAP scores can provide national comparisons as well). *ISU will fund a
                                                                                                        graduate student researcher responsible for collecting baseline and impact data, *Abilities of Noyce Scholars-in-service will be monitored by the PDS institution-wide annual assessment system
                                                                                                        offering multiple checkpoints throughout the program for remediation and refinement of noted deficiencies, as well as the impact of the performance of the Scholars-in-service on the students they
                                                                                                        teach. *Annual summaries of the program impact of NSP Scholars will be included in the annual assessment report prepared by the Assessment Coordinator and will be reviewed by the external
                                                                                                        assessment consultant.




                                                                                                        *Mentor input (formative and summative appraisals of pre-service teacher performance) during the academic year and end of each semester. *Professional portfolios (during 2nd student teaching
                                                                                                        experience) including: lesson plans for 5 consecutive lessons, video tape of lesson taught, copies of student work from that lesson, analysis of impact of the lesson on student learning, analysis of
                                                                                                        their teaching. *Assessed in content knowledge, task selection, classroom environment, analysis of student learning, and analysis of teaching (results aggregated to determine if program-completing
                                                                                                        scholars possess the skills, knowledge and dispositions expected of beginning teachers as established by Interstate New Teacher Assessment and Support Consortium (INTASC)). *2 observations
                                                                                                        (protocol designed by Horizon Research, Inc. for NSF) with pre- and post-observation interviews during 1st year of teaching. *Follow-up on scholar effectiveness with principals or department chairs
                                                                                                        after years 1, 2, and 4.




*Unit performance (NCATE reviews, university review of strategic plans and progress).




                                                                                                        *Assessment of quality of Noyce teachers (Assessed through: interviews with Noyce recipients’ university instructors, university GPAs, and interviews with administrators at the school/district where      *Regular surveys into their beginning years of teaching and an online portfolio system monitored by the College of
                                                                                                        recipients teach)                                                                                                                                                                                           Education Career and Program Support (CAPS) Office. *An electronic tracking system developed by the Indiana
                                                                                                                                                                                                                                                                                                                    Dept of Ed that identifies individuals currently teaching in the State of Indiana and lists the school that employs
                                                                                                                                                                                                                                                                                                                    them. *Required (years 1-6) written documents submitted to the Student Services Office indicating their current
                                                                                                                                                                                                                                                                                                                    teaching status. *MU Alumni Office’s maintained database of contact information on all MU graduates.




                                                                                                        *Classroom observations of scholar graduates in their 2nd year of teaching (using the NSF-CETP Core Observation Protocol)                                                                                   *Signed recipient acceptance agreement outlining terms of award *Annual mail correspondence to gather current
                                                                                                                                                                                                                                                                                                                    contact information and employment data *Fall 2005 (first year with Noyce graduates meeting teaching service
                                                                                                                                                                                                                                                                                                                    obligations) employment information to certifying teaching placements for service fulfillment
                                                                                                                 *Student Teacher Appraisal Reports (STAR) (10x in year 1 and 2x in year 2) to evaluate recipients' teaching and completed by school-based and field-experience mentor with observations by the          *Verification of placement in a qualifying school (1x during teaching years 1 and 2 with monitoring continuing
                                                                                                                 building principal. *Texas EXamination of Educator Standards (TEXES) content and pedagogy exams during final coursework semester. *Classroom Environment Scale (CES, Moos & Tricket,                    until service requirement is fulfilled)
                                                                                                                 2002) (pre/post measurements in teaching years both 1 and 2) measures student and teacher perceptions of the classroom and provides a way to examine the effects of course content, teaching
                                                                                                                 methods, teacher personality, and classroom composition. *Texas Assessment of Knowledge and Skills (TAKS) (1x during both teaching years 1 and 2) to measure the achievement of students in the
                                                                                                                 scholarship recipients’ classrooms. *Professional Development and Appraisal System (PDAS, Texas Education Agency, 2003) (4x during teaching year 1 and 2x during teaching year 2) to evaluate
                                                                                                                 teachers each academic year according to 7 criteria representing categories of teacher responsibilities and levels of instructional effectiveness. *Teaching portfolio (2x during teaching year 1; 1x
                                                                                                                 during teaching year 2) to determine if recipient is applying the knowledge, skills, and dispositions of effective teaching practice. *External evaluator designed plan to:
                                                                                                                 a) examine teacher candidate knowledge as evidenced by transcript evaluation and state exam results. b) evaluate teaching practice including technology integration, student achievement, and
                                                                                                                  student attitude.




*Evaluation to assess the organizational structure (collaborative support system) as monitored by                *Evaluation to assess professional development. *Evaluations conducted through: a) in-service workshops; b) practicum logs; c) mentor/Induction year teacher meetings; d) follow-up survey sent
attendance at orientation workshop and other indications of on-going collaboration between schools and the       each year beginning in fall 2004. *Success of classroom activities measured by an increase in quality content labs that focus on laboratory experiences and manipulatives designed to improve
university.                                                                                                      outcomes for students in high-need schools.




                                                                                                                 *Recipient self-report on plans for monitoring teaching success and 3 or more measures used to determine effectiveness


*Interviews and documentation of process to determine the strength and productivity of partnerships and
their role in facilitating student entry into the teaching profession with confidence.



*Compare our (the 9 Texas A&M system partners) data with data collected by the Texas Higher Education
Coordinating Board’s “Closing the Gap” effort to increase graduates in critical fields such as mathematics
and science




                                                                                                                 *Mentor teacher and other school personnel evaluations. *Annual evaluations by principals. *Other pertinent data.                                                                                       *Tracking system to monitor service requirement until completed or reimbursed. *Student completed inquiry
                                                                                                                                                                                                                                                                                                                         form (web page to collect demographic data) and principal-signed certification form attesting to current
                                                                                                                                                                                                                                                                                                                         employment each semester)




*FOCUS Project evaluator will collaborate to create a regional teaching database (that will include all
scholarship recipients) to track the educational progress and institutional choices of prospective teachers in
pre-credential, credential, and induction teaching phases of their careers. *Collaboration between FOCUS,
CTFMS, and Noyce programs.




                                                                                                                                                                                                                                                                                                                         *Review documents in the program files using quantitative methodologies to determine answers to questions such
                                                                                                                                                                                                                                                                                                                         as: do scholars sign necessary paperwork? *Doc review and subsequent quantitative analyses of the district
                                                                                                                                                                                                                                                                                                                         employment records database will assist in answering questions such as: a) how long do Scholars remain teaching
                                                                                                                                                                                                                                                                                                                         in high need districts? and b) what kind of schools do Scholars choose after graduation (diversity of students,
                                                                                                                                                                                                                                                                                                                         faculty, administration; SES; school grades)?




*Eval plan carried out under direction of the UMass Center for Learning, Teaching, and School Change             *Performance assessment data will be used to conduct a longitudinal study of the early stages of teachers’ careers using students’ standardized test scores as a measure of teacher effectiveness.      *Scholarship recipients required to participate in performance assessment process through end of obligation period
with "data tracking" conducted in conjunction with the office of the Director of Teaching.                                                                                                                                                                                                                               to evaluate teacher effectiveness




*Evaluation will: a) be managed and conducted (as well as having surveys and evaluation criteria                                                                                                                                                                                                                         *See "summative program effectiveness data" column *To assess achievement of program objectives
developed) by the University’s Office of Institutional Research (UOIR), b) utilize guidance and feedback
from the Advisory Committee and faculty advisors, and c)involve collaboration between the office of
enrollment management and UOIR to develop a database of baseline indicators and to track project impacts.




                                                                                                                 *Teaching effectiveness (comparison of teachers’ N.C. end-of-year and end-of-course passing rates with program teachers). *Detailed records maintained on teachers participating in professional        *Tracking of scholarship/stipend recipients (while fulfilling teaching obligation) will follow the NC Teaching
                                                                                                                 development activities                                                                                                                                                                                  Fellows model: a) recipients sign a promissory note; b) PI and Co-PIs verify academic success and post-graduation
                                                                                                                                                                                                                                                                                                                         employment compliance (with data collected during pre- and in-service development activities through agreements
                                                                                                                                                                                                                                                                                                                         with regional school systems).




                                                                                                                 *Survey data (of Noyce students) on what constitutes good teaching and the amount of time teachers should spend in various types of pedagogical activities (e.g., lecture, discussion, real world
                                                                                                                 problems).
                                                               Formative Program Effectiveness Data                                                                                                                                        Summative Program Effectiveness Data                                                                                         Document Analysis


                                                                                                                                                                                   *Summation of the formative data on the quality of teacher performance




R--Details on program financial stability (current and projected expenditures)                                                                                                     *Degree to which the interaction of the Noyce project and the UTSA-UTeach project actually produces a significant increase in the number of science and
                                                                                                                                                                                   mathematics teachers certified (measured as the number of students enrolled in the program and the number of appropriately certified teachers produced by the
                                                                                                                                                                                   program against the baseline data embodied in the statistics of the past five years and conducted at 6-month intervals throughout the project with info on both
                                                                                                                                                                                   enrollees and graduates of the program collected)




*Midsemester surveys of all courses; *Survey of graduates about portions of UTeach that were most and least useful and institute program modifications in response                 *Survey of graduates about portions of UTeach that were most and least useful and institute program modifications in response; R--graph of enrollment trends in        *All students submit portfolio as part of requirements
                                                                                                                                                                                   student teaching (by major) from 1996 to 2004.                                                                                                                         for recommendation for certification. It is evaluated
                                                                                                                                                                                                                                                                                                                                                          by trained readers.




                                                                                                                                                                                                                                                                                                                                                          *Program and student records


*Yearly evaluation to see if project goals are being met




*Evaluate short-term impacts of project activities                                                                                                                                 *Evaluate long-term impacts of program activities. *Standard evaluation forms: (created for BCM’s GK-12 program) to assess participants’ satisfaction with the
                                                                                                                                                                                   quality of the “Introduction to Science Teaching” experience (including documentation of the actual activities undertaken by each participant (number of hours spent
                                                                                                                                                                                   co-teaching, planning/teaching a lesson, conducting observations)).




*Formative implementation evaluation: follow project evolution to see how well it becomes an established functioning entity and to document changes in project plans               *Eval to determine extent to which FCEPT/Noyce activities led to major and long-lasting reform in: teacher preparation, support of new teachers, teacher retention
*Formative progress evaluation: baseline for components; assess progress toward project goals (i.e., how are activities and strategies aiding participants to move toward goals)   rates, and student classroom achievement




*Review of program materials and other project activities and records *Meetings/interviews with the PI and other key staff to ID strengths and weaknesses *Evaluator-led           *Exit or end-of-program interview survey *Annual follow-up questionnaires mailed to students starting one year after program completion
focus groups with students to get feedback on levels of support received during their participation in the program *Questionnaires mailed or emailed to students periodically


*Twice-yearly surveys of scholars, advisors, and districts to assess satisfaction with the program and to determine any adjustments or improvements that need to be made with      *Effectiveness data will be assembled and analyzed annually by the program manager, PI, and co-PI to determine the effectiveness of the program as a whole.
scholars asked to asses: 1) the obstacles or difficulties encountered, 2) the value of the supports provided




*Program description both as proposed and as implemented (including program objectives and enabling activities as well as resources allocated to accomplishing program             *Information on the impact of program services and activities on participants and other service recipients (including number of students employed in positions
services). *Science Teacher Efficacy Belief Instrument (STEBI) to determine teacher candidates' perceptions on the effectiveness of the science and math methods courses.          directly related to the program).
*Salish I Research Instruments to holistically use separately gathered data on: demographics, pedagogical philosophy, teaching (observation), and personal attitudes of the
teacher towards the nature of science, math, and technology to determine preservice math and science program effectiveness.




*Qualitative data to alert project management to strengths and weaknesses in order to adjust components and report the effectiveness and impact of key project elements.           *Initially counts and graphs of numbers representing each of the following outcomes (recruitment of math/science interns, graduation and certification, employment
*Benchmarking data on: key student expectations, perceived needs, assumptions, program satisfaction, and preparedness and success. *Data to formatively evaluate program           by many in low-income urban districts, and continued employment of teachers especially those of color) in pipeline format for a given cohort with subsequent time
effectiveness and make mid-course corrections will be gathered via: interviews, pre- and post- questionnaires, focus groups, and field observations.                               series data for key outcome measures and comparing cohorts to previous cohorts.
*Rubric-based analysis of Cornell Teacher Education (CTE) student projects to determine if: some students take longer to learn to reflect, if students beginning program at
later points in life progress differently than college juniors, and if some assignments better foster the thinking/acting desired in future teachers.




*Eval plan will provide effectiveness data of the program in preparing program participants.                                                                                  *Scholar exit interview and survey. *From academic year 2001-2002 on: a maintained comprehensive database with information on teacher and administrator
                                                                                                                                                                              preparation, turnover, in-service education and performance. *Eval plan will provide effectiveness data of the program in preparing individuals for teaching careers.
                                                                                                                                                                               *Number of Noyce Scholars who obtain math/science teaching positions in high need districts. *Number of Scholars who maintain these positions beyond required
                                                                                                                                                                              time (records maintained until 2014).




*Quarterly focus groups and interviews with STEM scholars, STEM professionals, and Drexel Faculty; analysis of student teaching journals, electronic porfolios, and           *Annual interviews and focus group meetings with key program staff representing partner institutions to evaluate the overall program and to obtain suggestions for
performance-based course assessments for STEM scholars. *Evaluate practices used to prepare Noyce scholars and professionals.                                                 improvement. *Determine long-term performance of scholars and professionals in terms of retention as a teacher, job satisfaction and identification with the
                                                                                                                                                                              profession



*KSDs (see"research questions/evaluation goals" column) measured: 1-16. *Mid-Internship: 1) evaluation of Noyce teacher internship by mentor teacher using the Summative *KSDs (see"research questions/evaluation goals" column) measured: 1-16. *End-of-Internship: 1) Mentor teacher evaluation of Noyce teacher internship using SET;
Evaluation Tool (SET); 2) conference with discipline area specialists to evaluate Noyce teacher internship; 3) program director evaluation of Noyce teacher internship using 2) Thesis committee rubric-based examination of Noyce teacher's Teaching Portfolio (i.e., final philosophy statement, technology portfolio, meta-reflection, student
SET; 6) 2nd Technology Assessment by program director (using survey & demonstration methods).                                                                                growth and development demonstration, and content knowledge assessment).




*External evaluator, PI, Co-PIs will collect year 1 baseline data to evaluate the project against in years 2-4; *Report formative findings on a continuous basis;             *Effectiveness data on the UMEP program in preparing individuals for teaching careers; *Report summative findings upon completion of the project, comparing              *Review of project documents and records;
                                                                                                                                                                              project outcomes against baseline data;
                                                                                                                                                                              *For TEEMS program, the frequency of graduates: *with a pass-rate of the PRAXIS II (teacher certification) exam at 100%; with a 3.0 GPA or above; with grades
                                                                                                                                                                              of “C” or better in mathematics courses; with grades of “B” or better in all field work; meeting or exceeding the technology standards and skills in appropriate and
                                                                                                                                                                              meaningful uses of technologies in teaching mathematics; demonstrating and documenting evidence of knowledge and skills across the ten Principles of Interstate
                                                                                                                                                                              New Teacher Assessment and Support Consortium (INTASC).




                                                                                                                                                                              *Annual data reported on the overall effectiveness of the NSP program to to attract, prepare, and retain highly qualified pre-service teachers; *Collection of data on
                                                                                                                                                                              the impact of the Noyce Scholars Program (NSP) teacher preparation on student achievement




*Scholar perceptions (after years 1, 2, and 4) on how well the program prepared them to teach in relationship to content knowledge, teaching for understanding, evaluating    *Scholar and immediate supervisor perception data analyzed to determine program effectiveness.
student learning, reflective practice, and professionalism.




*Formative Self-evaluation of Faculty Teams (considered important to monitor and modify program elements)                                                                     *Continuous data collection on performance of graduates (exiting student surveys, National Board certification data), *Program performance (specialty program
                                                                                                                                                                              reviews, accreditation reviews, university program reviews, NCATE reviews, faculty evaluations),



*Evaluation including formative feedback on the process and progress of project implementation. *Evaluation-documented effectiveness of web site information to the Noyce     *The quality of teacher produced with support of Noyce funding and her or his longevity in teaching within the timeline of Noyce funding. *Successful
recipients.                                                                                                                                                                   implementation of project as outlined in proposal. *Success of students becoming certified as determined via: (a) The number of teachers remaining in teaching
                                                                                                                                                                              through Noyce funding term (Teachers tracked through alt cert program with comparisons between % of Noyce students staying in teaching and % of either alt cert,
                                                                                                                                                                              but no Noyce funds or traditionally certified) and (b) Assessment of quality of Noyce teachers (Assessed through: interviews with Noyce recipients’ university
                                                                                                                                                                              instructors, university GPAs, and interviews with administrators at the school/district were recipients teach)




*Whether NSF and pre-service Noyce scholars participated in grant-related activities available to them, and how helpful these activities were in their development as a       *Survey responses will also be gathered from in-service teachers on unmet professional needs and preferences (see "Specific Analyses or Methodologies" column
mathematics / science teacher                                                                                                                                                 for greater detail); *How these scholars rated the quality of their teacher preparation programs
                                                                                                                                                                                  *Teacher technology survey (pre-test in semester 1; post-test prior to teaching year 1) (South Central Regional Technology in Education Consortium, 2004) to
                                                                                                                                                                                  establish baseline data about the technology skills of beginning teachers; *External evaluator plan to examine the relationship of content and pedagogy training to
                                                                                                                                                                                  student achievement on state exams.




*Formative evaluation to assess: organizational structure (collaborative support system), professional development, mentor to induction year teacher relationships, and           *Summative evaluation to assess: mentor to induction year teacher relationships, expectations (sustainability of project activities after the grant, information
expectations (sustainability of project activities after the grant, information dissemination, weaknesses). *Attitudinal instruments will measure satisfaction with project       dissemination, weaknesses), and outcomes (benchmarks of the program, final results of activities)
activities by participants. *Additional data will be collected through mentor notes and practicum logs. *Evidence of technology and best practices being integrated into
coursework assignments will be present in course syllabi. *Course evaluations by project participants will provide further evidence of project success.




*Online survey evaluation of particular courses. *Annual reports on all aspects of the program including appropriate recommendations. *Program documentation, interviews,
observations, and discussions to evaluate how well courses meet science/math content and pedagogical needs and that students are satisfied, challenged, and successful in
course and lab work. *Observation, interviews, and documentation to evaluate how well program administration establishes structures and roles and enables students to move
smoothly through the program with problems solved as needed.




                                                                                                                                                                                  *Qualitative student survey data on: their experience in their degree program, *satisfaction with the support services, *perceived value of program participation,
                                                                                                                                                                                  *impact of family and financial assistance on their college experience.




*Utilize doc reviews and surveys as well as both quantitative and qualitative methodologies with course enrollment documents, course syllabi to determine whether Scholars
have been initiated into the field of education and becoming part of a professional collaborative community (Scholar opportunities to develop a cohort group are relevant for
this outcome).


*See "summative program effectiveness data" column as these data may be intended for formative evaluation use.                                                                    *Gerdeman’s dissertation (a 2004 resource is cited in the received email) is being used as a baseline for change in the perception and interest of UCLA undergrads
                                                                                                                                                                                  in science and math towards teaching careers.


*Formative evaluation using: a) analysis of scholarship recipient surveys, b) questionnaires, c) focus groups, and d) data from other stakeholders (e.g., cooperating teachers,
school administrators).




*Evaluation of the Noyce Scholars program will use ongoing formative assessment tools as well as existing instruments for assessments (course grades, cumulative GPAs)        *Annual summative evaluation (to determine program impact) will involve interviews with involved faculty and BPS head administrators, yearly assessments and
whenever possible: a) to assess programmatic strengths and weaknesses (including quality with which the proposed benchmarks within the activities are obtained); b) to assess interviews with each of the participants (including those in new teaching positions for less than two years) to: a) assess achievement of the program objectives, b) to
the extent to and quality with which proposed benchmarks within activities are attained, and c) to assess the systems for documenting service delivery.                       measure the impact of the activities on Noyce Scholars around retention, academic performance in STEM courses, quality of teaching preparation experiences, and
                                                                                                                                                                              graduation with a baccalaureate degree, and c) to determine whether the UMB Noyce Scholars Program proved the intellectual merit of its teacher preparation
                                                                                                                                                                              model for UMB, BPS, and the public record for effective practices




*Outcome data including: assessments of the # of program participants receiving services and the # of contact hours (with efforts made to make services equally accessible to     *Summative (to determine the extent to which goals were achieved) activities utilizing ongoing analyses of quantitative and qualitative data will be conducted.
diverse groups). *Formative (to determine if program is implemented as intended and improve the project) activities utilizing ongoing analyses of quantitative and qualitative
data will be conducted. *Questions to guide the evaluation of the integrity of program delivery will be developed according to the NSF’s Division of Research, Evaluation and
Communication “User-Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering and Technology Education.”




*Possibly survey response change over time.                                                                                                                                       *Possibly survey response change over time.
                                       Observations                                                                              Self-Report Data                                                                                  Specific Analyses or Methodologies




*Observations of Noyce graduates underway and to continue in Fall.                                 * Large collection of items were collected in recent survey of graduates (sent to
                                                                                                   graduates from all years, not just students at time of graduation).




                                                                                                   *Recipient opinions on teacher preparation program(s) and first teaching
                                                                                                   assignment


                                                                                                   *1st year exit interviews with recipients.



*Of Advisory panel meetings                                                                        *Annual interviews with undergrad scholars *Annual surveys with scholars
                                                                                                   once teaching

*By co-PIs of scholar graduate's implementation of learned teaching strategies.                    *Interviews with scholar graduates on experiences in implementing learned           *Qualitative semantic domain analysis over time *Compare the experience of these Scholars with teachers coming from more
                                                                                                   teaching strategies                                                                 traditional backgrounds. *Using Data gathered from two groups of non-Noyce mathematics teachers (in- and pre-service) in addition
                                                                                                                                                                                       to Noyce Scholars: (*Instruments to test mathematical knowledge of fundamental ideas including ability to use that knowledge to solve
                                                                                                                                                                                       math problems, *Instruments to test ability to apply math knowledge in teaching situations and to analyze student responses to math
                                                                                                                                                                                       situations and suggest appropriate course of action, and Observations of Scholar classroom instruction especially the view of math they
                                                                                                                                                                                       portray to students).

                                                                                                   *Surveys: To assess participants’ science teaching efficacy beliefs (At three      *Evaluate each project objective both quantitatively and qualitatively with the evaluation embedded within ongoing evals of the
                                                                                                   time points: before any project activities, after preliminary teaching experience, OPTIONS program. *NOYCE objectives include: a) ID and recruit 10 graduate science students and/or post-graduate scientists each
                                                                                                   and after completion of all requirements) using the SETAKIST (Roberts, 2002) year; b) Provide support, instruction, and experiences to retain all 40 Noyce Fellows; c) Create reporting and accountability system to
                                                                                                                                                                                      assure full participant compliance and recover stipends from defaulted participants; d) Evaluate short- and long-term impacts of project
                                                                                                                                                                                      activities, and track Noyce Scholars through their teaching careers. OPTIONS objectives include: a) Promote innovative natural
                                                                                                                                                                                      science preparation through partnership; b) Complete and deploy modular online instructional program; c) Provide opps for ATC
                                                                                                                                                                                      candidates to observe classrooms and gain practical teaching experiences fulfilling Texas ATC requirements; d) Promote ongoing
                                                                                                                                                                                      professional development of all OPTIONS participants entering teaching via ATC; e) Utilize external evaluator to formatively and
                                                                                                                                                                                      summatively assess project activities attending to short-term outcome indicators and longer-term implications; and f) disseminate
                                                                                                                                                                                      results. *Specific methodologies include:
                                                                                                                                                                                      ANOVA and repeated measures to monitor diffs between and among observed groups; Multiple regression to
                                                                                                                                                                                       ID diffs in teaching behaviors among groups of observed teachers; Explore relationship among variables related to
                                                                                                                                                                                      classroom observations, science teaching efficacy beliefs and the science background of each OPTIONS
                                                                                                                                                                                      participant (vague).
*Initial or final student teaching scholars in credential program are evaluated by university      *Attitudinal surveys given yearly to Noyce graduates while teaching
supervisors (i.e., by a program already in place) using performance guidelines by SOE
*Intend to evaluate fully credentialed Noyce teachers next fall (PI applied for sabbatical to do
so)

                                                                                                   *Evaluation activities focused on Scholar attitudes and beliefs concerning the      *Pre- and post tests centered around Scholars *Focus group activities centered around the Scholars
                                                                                                   program's value; *Scholar "self-evaluative" journaling regarding experiences
                                                                                                   tutoring and student teaching; *Evaluation activities focused on Scholar
                                                                                                   attitudes and beliefs about high needs schools' attributes

                                                                                                   *Spot phone calls to alumni to get feedback on successes and challenges they        *Formative and Summative Evaluation to determine programmatic strengths and weaknesses using enrolled and completed student
                                                                                                   face as science teachers.                                                           data collected via evaluator observations, focus groups with participants mailed surveys, emailed websurveys, and phone interviews


                                                                                                   Scholar survey (administered at beginning of program and annually) to assess
                                                                                                   scholar opinion of their ability to teach science or math, scholar level of
                                                                                                   commitment to teaching, scholar opinion of the teaching profession, the impact
                                                                                                   of Noyce funds on their decision to become a teacher * …with questions
                                                                                                   adjusted once teaching to capture satisfaction or frustration with teaching,
                                                                                                   plans to continue teaching, and intention to continue teaching in their current
                                                                                                   school or another high-need school


*(Salish I) controlled observational instruments to supplement data from the Teachers'             *(Salish I) focus group exit interview of graduates' pedagogical philosophy to   *Pre- and post-program administrations of STEBI and Stalish I Instruments to measure growth/extent of change *After 1 year of actual
Pedagogical Philosophy Interview (TPPI).                                                           elicit teacher perspective on effectiveness of science teacher strategies and    teaching: randomly sample CSUSM credential graduates employed at 50-50 schools (experimental) and non-CSUSM credential
                                                                                                   evidence of student success on achieving science and math concepts. *(Salish candidates employed at similar schools (control) to examine teacher effectiveness (using Salish I Instruments).
                                                                                                   I) nature and implications of science and technology survey that explores the
                                                                                                   teacher candidate consistency or change in perceptions of the nature of science.



                                                                                                   *Interviews with participants each semester in the program and each year            *Two external evaluators *Internal evaluation with both formative and summative components *California Teaching Performance
                                                                                                   thereafter                                                                          Expectations (to measure teaching performance) for the 2 years they are observed. *These multiple methods will enable triangulation
                                                                                                                                                                                       of data to determine effective and less effective aspects of the program.
                                                                                                                                                                                  *Intend to conduct a comparative study of pre-service teachers, student teachers, and first/second year teachers who are Noyce Fellows
                                                                                                                                                                                  and those who were not Noyce Fellows (analysis of great interest because believe that the Undergraduate Learning Assistant (ULA)
                                                                                                                                                                                  experience (what Noyce-eligible students participated in) leads to better prep of K-12 teachers).


                                                                                                                                                                                  *PI (who is experienced in case study and interpretive research) will design interviews and questionnaires to obtain and analyze rich
                                                                                                                                                                                  data.




*Classroom observations and evaluations with program status and evaluation data published        *Qualitative data on scholar impressions of and overall satisfaction with the    *Project manual detailing every component of designing, implementing, carrying out and evaluating the program.
annually on the Dowling website. *Spring classroom observations in first critical years after    services of the project
graduation with follow-up consultation with Noyce teachers.




                                                                                                 *Annual survey of Noyce Scholars to determine whether they've developed an       *Obtain qualitative information on case studies of scholars *Evaluation conducted to assess the project’s effectiveness in attracting,
                                                                                                 identity with the teaching profession. *Compare Noyce and non-Noyce              preparing, and retaining teacher candidates. *Evaluation facilitates collecting demographic data on, and tracking recipients as they
                                                                                                 assisted teachers on surveyed teacher attitudes towards teaching during the      fulfill their teaching obligation in the Philadelphia Public Schools.
                                                                                                 first three years and surveyed job satisfaction and teacher motivation




*Years 1-5: on-site visits                                                                       *Years 1-5: focus groups, and interviews with UMEP Students and Leadership       *Longitudinal research component to investigate how the alternative certification program prepares high quality teachers for urban
                                                                                                 Team; *Survey of UMEP participants each year of participation in the project;    school environments. *Used mixed methods to determine the extent to which the project is successful in: a) increasing the retention
                                                                                                                                                                                  rates of UMEP grads teaching in urban schools; b) developing a set of indicators of teaching commitment to urban education; c)
                                                                                                                                                                                  developing a set of indicators of high quality mathematics teachers in urban settings; d) identifying the variables that contribute to the
                                                                                                                                                                                  disconnect the preparation of mathematics teachers and their induction experiences in urban schools; e) identifying what new teachers
                                                                                                                                                                                  need for the next phase of career development (moving from induction to advanced professional training).




                                                                                                                                                                                  *External evaluation (by Dr. Dia Sekayi) including developing formative assessment instruments, analyzing data, communicating and
                                                                                                                                                                                  sharing data, determining whether goals have been achieved, identifying indicators for successes and shortcomings, evaluating
                                                                                                                                                                                  management roles and responsibilities, conducting longitudinal study of progress of teacher candidates




                                                                                                                                                                                  *Assess the value of problem-based learning (PBL) in urban settings. *Some discrete evaluation tools will be developed to assess
                                                                                                                                                                                  targeted teacher training in urban environments and in PBL teaching strategies. *ISTEP+ and MAP exam performance will be
                                                                                                                                                                                  monitored for the students taught by Noyce Scholars and compared with district-wide results.




*3 classroom observations conducted in the first year teachers’ classrooms                       *Efficacy scale completed both upon program admission and at end of program
                                                                                                 with aggregated data used to determine if program impacted the scholars’
                                                                                                 perceptions of their skills, knowledge and dispositions toward teaching. *3
                                                                                                 interviews and a final questionnaire administered to the teachers


                                                                                                                                                                                  *Will be employing an independent individual to conduct the evaluation of the program



*Structured observations of graduates: an induction facilitator will conduct these and provide                                                                                    *External Evaluation Consultant - an evaluation expert will advise the design of the permanent evaluation system and provide written
data                                                                                                                                                                              recommendations.



                                                                                                 *Evaluation data in the form of: *interviews with selected stipend recipients    *Evaluation of the progress of the establishment of the database for tracking students.
                                                                                                 and members of the review committee as well as *interviews with the PI and
                                                                                                 developers of the database.




                                                                                                 *Both NSF scholars and pre-service Noyce scholars' opinions of various           *Spring 2005 data are tabulated, but not summarized (first year to include Noyce scholarship graduates now teaching) *(Fall 2004)
                                                                                                 “reformed” teaching techniques, and their philosophical leanings on key          Classroom observations of scholar graduates in their 2nd year of teaching (using the NSF-CETP Core Observation Protocol) *Since
                                                                                                 MMSTEC values (many of these survey items adopted from CETP Core                 2005-2006 is the first year with Noyce graduates in their 2nd year of teaching (a limited # of these Noyce graduates may be added to
                                                                                                 surveys) *Survey responses from in-service teachers on unmet professional        the observation pool)
                                                                                                 needs and preferences *By end of 3rd year of teaching survey all teaching
                                                                                                 scholars to determine if (and to what extent) the scholarship influenced their
                                                                                                 teaching.
*Intern survey (1x during teaching year 1 including Likert Scale and anecdotal     *Texas Examination of Educator Standards (TExES): State test of content and pedagogy. Pre-test (sample) scores are used to
information) on teacher efficacy, knowledge/skills, and commitment to              determine strengths and weaknesses and later compared with actual scores. *Using Texas Assessment of Knowledge and Skills
teaching.                                                                          (TAKS) in an effectiveness comparison: a stratified random sample of scores from students whose teachers participated in the program
                                                                                   will be compared with students whose teachers did not participate in the program.




                                                                                   *Use of both qualitative and quantitative measures




*Recipient self-report of 3 or more measures s/he will use to evaluate
satisfaction with teaching.

*Annual interviews of all Noyce scholars and a selection of Stony Brook
faculty and partner participants. *On-line surveys to evaluate other
components of the program.


                                                                                   *Compare our (the 9 Texas A&M system partners) data with data collected by the Texas Higher Education Coordinating Board’s
                                                                                   “Closing the Gap” effort to increase graduates in critical fields such as mathematics and science




*Survey data from: *A&M Texarkana faculty, *mentor teachers, *principals,          *Focus groups: with cohort and non-cohort students as well as students with other teaching majors, to determine the reasons math or
*the induction coordinator, *other school personnel, and *parents.                 science was or was not selected as their major (data will be entered in an Access database).




*FOCUS Project evaluator designed a survey for undergraduate CTFMS                 *Contracted independent evaluator (within the university) will analyze and synthesize all of the eval data and write annual reports with
participants that senior-year scholarship recipients will complete. *Contracted    formative results and a final report with summative conclusions.
independent evaluator (within the university) will design new surveys for
scholarship recipients: 1) in their credential year and 2) in their new teacher-
induction year.




*Survey data has recently been collected from the second cohort of Noyce
Fellows who are earning their credential this year.




*Email interview with cohort 1; develop questions on first semester of
teaching; personal interview with cohort 2, detailed questionnaire for cohort 1;
data on satisfaction with whole program; email interviews with cohort 2;
detailed questionnaire to cohort 2; emailed exit interview




*Survey instruments (with different math and science forms) are completed:       *Identify changes in survey responses over time.
upon entering the program, after completing certification program, and at the
end of the first induction year as a teacher. These instruments examine: locus
of control, general teaching beliefs, specific math or science teaching beliefs,
perceptions of self-efficacy in managing student behavior, commitment to
teaching in urban settings, perceptions of barriers that might prevent them from
reaching urban students
                                                                                        Research Questions/Evaluation Goals                                                                                                                                         School/District Recruitment                                                     School/District Retention                                                   Additional Notes


*Evaluated program in terms of its ability to attract, prepare and retain STEM career-change individuals to teaching careers in the Chicago Public Schools. *Both formative and                                                                         *Recruitment data for Noyce recipients is tracked across the   *Retention data for Noyce recipients is tracked across the two years of the service
summative processes and products                                                                                                                                                                                                                        two years of the service obligation.                           obligation.




*Process and product-oriented evaluation activities to determine how successfully the three objectives for the project have been attained: Objective 1) Provide scholarships for
STEM Juniors and Seniors and stipends for STEM post-baccalaureates who are willing to teach science and/or mathematics in high-need school districts; Objective 2) Provide a
focused teacher certification program through collaboration among the College of Education and Human Development, the College of Sciences, the College of Engineering, and
the Honors College; Objective 3) Provide a support system for new teachers in the high-need school districts through a partnership between UTSA and collaborating high-need
districts.




                                                                                                                                                                                                                                                                                                                                                                                                                   Realize that gains in student learning represent
                                                                                                                                                                                                                                                                                                                                                                                                                  the most compelling evidence of the effectiveness
                                                                                                                                                                                                                                                                                                                                                                                                                      of Noyce scholarship recipients and would
                                                                                                                                                                                                                                                                                                                                                                                                                   welcome future research partnerships to obtain
                                                                                                                                                                                                                                                                                                                                                                                                                                   this type of data.




                                                                                                                                                                                                                                                                                                                                                                                                                      Uncertainty about retention data gathering




                                                                                                                                                                                                                                                                                                                                                                                                                    One first year graduate; planned longitudinal
                                                                                                                                                                                                                                                                                                                                                                                                                                       analysis


                                                                                                                                                                                                                                                                                                                                                                                                                  Grad student data analyst; annual formative eval
                                                                                                                                                                                                                                                                                                                                                                                                                              reports; evaluator funds

*What are the effects of the increased mathematical content knowledge of pre-service teachers in terms of teacher pedagogical content knowledge, teacher practice, and student
learning? *How can the content courses be improved to inculcate pedagogical content knowledge and produce an appreciation for it so that it is used in the classroom?




How can the content courses be improved to inculcate pedagogical content knowledge and produce an appreciation for it so that it is used in the classroom?                                                                                              Impact of Noyce Fellowships in recruiting qualified teachers   *Impact of Noyce Fellowships in retaining qualified teachers                                Noyce scholarships are described as just a means
1. Are partners able to recruit 10 science graduate students/professionals to participate in OPTIONS as Noyce Fellows? In what ways do applicants vary in terms of science field, previous teaching experience and current position? Do these
                                                                                                                                                                                                                                                                                                                                                                                                                     to bring in more competitive candidates for
differences influence their success as teachers?
2. What are levels of satisfaction with the online instructional modules regarding access, content appropriateness and difficulty, functionality, relevance of information, etc.? Do participants demonstrate mastery of content in each module using                                                                                                                                                              existing programs with funding already in place
online assessment tools incorporated into the modules? Are some modules more effective than others, and in what ways?                                                                                                                                                                                                                                                                                              for delivering instruction and support as well as
3. Do the preliminary teaching experiences provide an appropriate introduction to secondary science teaching? Does the mentored teaching internship help participants develop classroom teaching skills?                                                                                                                                                                                                             for evaluating the OPTIONS ATC program.
4. Do OPTIONS participants, in general, and Noyce Fellows, in particular, demonstrate improved teaching skills (demonstrate more “expert” behaviors) over the course of the one-year internship, as evidenced by                                                                                                                                                                                                  *How do they, and how will we, disentangle Noyce
classroom observations? In which teaching areas do participants demonstrate greatest
improvement? Least improvement? Which programmatic variables are most related to the
                                                                                                                                                                                                                                                                                                                                                                                                                            effects from OPTIONS effects?
development of teaching expertise by participants? How many participants complete all
six modules, take the TExES, and complete teaching internships in satisfaction of the
ATC requirements? What factors contribute to failure or success of OPTIONS
participants in completing all ATC requirements? Is a clear strategy for addressing the
ongoing professional development needs of participants articulated, with sequential
activities and related resources targeted for development?
5. Do the Noyce Fellows persist as teachers in high need schools for the required two
years? What are the most challenging aspects of working in high need schools for Noyce
Fellows? How is the OPTIONS program adjusted in response to feedback from Noyce
*Was preservice education enhanced? *Has the minority enrollment in the preservice program increased? *Is the innovative teacher preparation program attracting a different                                                                                                                                            *Number of years (beyond obligation) Noyce teachers remain in low-performing schools
type of student? *How do preservice students perceive the program, and does that perception change? *FCEPT surveys will also be used and SMEC will survey principals of
schools where students from the program are placed.




*Eval focus is to determine extent, nature, and adequacy of support received by program participants *Eval objective is to assess overall efficacy of Noyce Scholarship Program                                                                                                                                                                                                                                             Evaluation timeline available
in producing practicing secondary science teachers.


                                                                                                                                                                                                                                                                                                                       *Evaluate success in attracting, preparing, and retaining Noyce Scholars in teaching        A clearly specified agreement will ensure that
                                                                                                                                                                                                                                                                                                                       careers via program manager, PI, and co-PI annual extraction from DB and education         Noyce Scholars understand their obligations and
                                                                                                                                                                                                                                                                                                                       plans to determine retention, attrition, and timely completion of a degree or credential                      commitment




                                                                                                                                                                                                                                                                                                                       *Number of teachers retained in the teaching field for at least two years                  External evaluator role and misnamed instrument




                                                                                                                                                                                                                                                        *Employment counts of the number of Noyce teachers in low- *Continued employment counts of teachers, especially of color (see "summative program
                                                                                                                                                                                                                                                        income districts (see "summative program effectiveness data" effectiveness data" column)
                                                                                                                                                                                                                                                        column)
                                                                                                                                                                                                                                                                                                                                                Provided "basic" plan with further eval plan info
                                                                                                                                                                                                                                                                                                                                                available; Noyce here is an extension of another
                                                                                                                                                                                                                                                                                                                                                                    program


                                                                                                                                                                                                                                                    *Data collected on personal and contextual variables contributing to retention. *Work       Part of the eval plan was described as no longer
                                                                                                                                                                                                                                                    with Noyce teachers to ID support mechanisms that work and fail to retain teachers in       planned, but the exact section was not specified.
                                                                                                                                                                                                                                                    high needs districts/schools.




                                                                                                                                                                                         *Number of Noyce Scholars who obtain math/science          *Number of Scholars who maintain these positions beyond required time (records              By 2004 GIS project should be fully operational
                                                                                                                                                                                         teaching positions in high need districts.                 maintained until 2014).                                                                     with "map" to guide stakeholder queries. Noyce
                                                                                                                                                                                                                                                                                                                                                     LISTSERV expected to help contribute
                                                                                                                                                                                                                                                                                                                                                         information to the database.




                                                                                                                                                                                         *Semi-annual observations, interviews and surveys with     *Annual interviews, surveys, and examination of documented retention data (school
                                                                                                                                                                                         Noyce Scholars. *Semi-annual interviews and surveys with   records) of Noyce scholars and professionals as well as Non-Noyce teachers in undergrad
                                                                                                                                                                                         school personnel and principals                            and alt cert programs (to enable comparisons (i.e., Noyce vs. Non-Noyce assisted and
                                                                                                                                                                                                                                                    Noyce scholars vs. Noyce professionals).


*All gathered data based on the Knowledge, Skills, and Dispositions (KSDs): 1) demonstrated knowledge of the subject area(s) in there area of licensure; 2) effective application
of a variety of content area relevant teaching strategies; 3) understanding of the needs of diverse learners; 4) knowledge of the basic principles of child and adolescent
psychology; 5) knowledge of the philosophical, historical, and societal roots and foundations of education; 6) knowledge and understanding of the ethical, legal, and policy issues
informing current educational debates; 7) knowledge and understanding of curriculum issues informing current debates; 8) collaborative work with family members, school
colleagues, and community resources for benefit of all learners; 9) development and implementation of effective methods for classroom management; 10) development and
implementation of effective methods for planning lessons and units; 11) development and implementation of fair an equitable assessment system; 12) knowledge and
understanding of uses of technology to facilitate teaching; 13) presentation of the characteristics of professional teachers and emerging leaders; 14) practice of regular reflection;
15) consistent engagement in ethical behaviors; 16) commitment to educationl equity

*Specific evaluation questions include:                                                                                                                                                                                                             *Retention data on TEEMS teachers across the metro-Atlanta area. *Effectiveness data        Still seeking an external evaluator to develop and
• To what extent has the UMEP been successful in preparing high-quality mathematics teachers in urban schools? • To what extent has UMEP been successful in preparing                                                                               on the UMEP program in retaining individuals in teaching careers.                             implement a mixed methods evaluation of the
teachers who are committed to teaching in urban schools? • Which components of these of program do graduates perceive as being most effective? • Which program components                                                                                                                                                                                             project
were perceived to be most effective for different types of students (based on undergraduate major, age, gender, ethnicity, etc.)? • What do the UMEP graduates perceive to be the                                                                   *Measurable objectives of the projects goals are: • An increase in the retention rate of
major barriers to their development as high quality mathematics teachers in urban schools? • What do the UMEP graduates perceive to be the major barriers to being committed                                                                        UMEP graduates in urban schools; • A list of valid and reliable performance measures
to remaining in urban schools for three years or more. • What additional professional development activities were identified and implemented during the project? • How actively                                                                     that indicate commitment to urban education; • A list of attitudes, values, and beliefs
did master teachers mentor new UMEP graduates in their schools?                                                                                                                                                                                     that correlate with commitment to urban education; • A list of mutable variables that
*Summative evaluation questions:                                                                                                                                                                                                                    contribute to the perception of a disconnect between teacher preparation and induction; •
• To what extent did UMEP meet its goal? Specifically, to what extent was the project able to:                                                                                                                                                      A decrease in TEEMS graduates perception of a disconnect between their preparation
-Increase the retention rates of UMEP graduates teaching in urban schools; -Develop a set of indicators of teacher “commitment” to urban education; -Develop a set of                                                                               and induction experiences;
indicators of “high quality mathematics teachers in urban settings;” -Identify the variables that contribute to the “disconnect” between the preparation of mathematics
teachers and their induction experiences in urban schools; -Identify what new teachers need for the next phase of career development, as they move from induction to
advanced professional training;
*A measurable objective of the projects goals is: • Dissemination of the project findings through papers at professional conferences, publications in scholarly journals, and
publication of a monograph describing the project and its findings.

*Mailed surveys with the following items: Do you feel that you have been successful during your first six weeks of teaching? What are some of the strategies and techniques you          *Increased recruitment of teachers from minority and       *Improved teacher retention within partner schools on an annual, five-and ten-year
learned in the “Science and Mathematics for All” program that have helped you to be successful? What have been your challenges during your first six weeks in your teaching              disadvantaged populations into partner schools.            period.
position? How were you prepared for such challenges in the “Science and Mathematics for All” program? What kind of support could you use going forward in your teaching
career? How can the “Science and Mathematics for All” program assist in the support you need?




                                                                                                                                                                                                                                                    *2002-2003 academic year information indicated that: 100% of 1st cohort graduates
                                                                                                                                                                                                                                                    continued into their third year of teaching;
                                                                                                                                                                                                                                                    100% of 2nd cohort graduates returned for their 2nd year of teaching.




                                                                                                                                                                                                                                                                                                                                                Have not begun eval process as only in 2nd year
                                                                                                                                                                                                                                                                                                                                                   with only 4 scholars in teaching positions

                                                                                                                                                                                                                                                                                                                                                An evaluation expert will advise the design of the
                                                                                                                                                                                                                                                                                                                                                permanent evaluation system and provide written
                                                                                                                                                                                                                                                                                                                                                               recommendations.
                                                                                                                                                                                        *External evaluator designed plan to examine the recruitment *External evaluator designed plan to examine teacher retention.
                                                                                                                                                                                        of high quality mathematics and science teachers for high
                                                                                                                                                                                        need districts.




                                                                                                                                                                                                                                                                                                                                                         Eval plan references a "strategic plan" that
                                                                                                                                                                                                                                                                                                                                                        provides specific evaluation and benchmark
                                                                                                                                                                                                                                                                                                                                                       indicators, but that portion of the plan was not
                                                                                                                                                                                                                                                                                                                                                                           received.


                                                                                                                                                                                                                                                       *Recipient self-report on factors/events that could lead to leaving teaching; *Recipient         Evaluation (as provided) is based entirely on
                                                                                                                                                                                                                                                       self-report on whether s/he anticipates remaining in teaching for at least the next 5 years.                  survey instruments




*The key question addressed with continual assessment and improvement of the Noyce Scholars Program is: to what extent has the TxCETP Noyce Scholars Program enhanced                                                                                                                                                                                 Noyce administrative funds will be used to hire a
the number of outstanding undergraduate mathematics and science students committed to and well prepared for a teaching career?                                                                                                                                                                                                                        master teacher to facilitate weekly seminars for
                                                                                                                                                                                                                                                                                                                                                                      novice teachers.




*Three preparation phase evaluation to examine: *recruitment strategies to attract a diverse and talented pool of STEM majors to early teacher preparation activities, to a teacher                                                                                                                                                                         More detailed description of CFTMS
credential program, and to a teaching position in a high-need district; *preparation of scholarship recipients for success in teaching in terms of knowledge and use of subject
matter expertise, effective teacher strategies for all learners, and strategies for engaging in teacher inquiry; *preparation of scholarship recipients to take the initiative, and a
leadership role, in examining and sharing knowledge and strategies with peers; *success in sustaining scholarship recipients’ commitment to teaching and to a district or school.




*Evaluation addresses: a) the importance of the financial assistance in the decision for students to come to UCLA; b) the environment they experienced as undergrads here at                                                                                                                                                                          UCLA has opened a Teaching Resource Center to
UCLA.                                                                                                                                                                                                                                                                                                                                                 promote science (and math) teaching as a visible
                                                                                                                                                                                                                                                                                                                                                             and accessible option for majors.

1) Evaluate the effectiveness of STEM Bridge in attracting, preparing, and retaining STEM individuals in teaching.
2) Evaluate the effectiveness of Noyce scholarship recipients as teachers.
3) Provide formative information for the improvement of the STEM Bridge project.
4) Track scholarship recipients during the period in which they are fulfilling their service obligation.

                                                                                                                                                                                                                                                       *See "summative program effectiveness data" column




                                                                                                                                                                                        *Data on entry into teaching (percent of students who obtain   *Retention in teaching (percent of program completers still teaching with a three-year
                                                                                                                                                                                        teaching jobs in designated areas)                             time period and after a five-year time period)




*Attitudinal change over time.                                                                                                                                                          *Survey data on commitment to teaching in urban settings.      *Survey data on perceived efficacy in managing student behavior, commitment to
                                                                                                                                                                                                                                                       teaching teaching in urban settings, and perceptions of barriers to reaching children in
                                                                                                                                                                                                                                                       urban settings.
                         Instrument(s)                              Link to Instrument

  Math achievement of Scholar-taught students (state-mandated
                                                                        None
         Alabama Reading and Mathematics Test (ARMT)
 Tests to measure Scholar-taught students' achievement of the big
                                                                        None
                      ideas taught to scholars
  Noyce grads included in yearly data collection by TEAM-math
                                                                        None
  (Transforming East Alabama Mathematics) evaluation process
 Survey of science teaching efficacy beliefs using the SETAKIST
                                                                  ERIC Accession No:
(Roberts & Henson, 2000): Self-Efficacy Teaching and Knowledge
                                                                      ED 448208
                 Instrument for Science Teachers.
  Observations: within classroom via district mentoring program             None
 Observations: 4 with Stallings Observation System (SOS) using
                                                                       Stallings Report
                        Microsoft Access
  FCEPT surveys used (at entry into, and upom completion of,
                                                                       Survey Themes
                     preservice component )
  FCEPT surveys used (at entry into, and upom completion of,
                                                                       Survey Themes
                     preservice component )

Science and Math Education Center (SMEC) will survey principals
                                                                           Survey
      of schools where students from the program are placed.


Science and Math Education Center (SMEC) will survey principals
                                                                           Survey
      of schools where students from the program are placed.

                 Student standardized test scores                           None
                 Student standardized test scores                           None
  Salish I (1997) Research Instruments (Teachers’ Pedagogical
  Philosophy Interview—TPPI) for holistic data on preservice           TPPI Website
                     program effectiveness.
Salish I (1997) Research Instruments (Nature and Implications of
                                                                    ERIC Accession No:
    Science / Technology Survey (NISTS) for holistic data on
                                                                        ED 463974
                 preservice program effectiveness.
     Teacher efficacy via the Science Teacher Efficacy Belief
                                                                           STEBI
            Instrument (STEBI, 1990, Riggs & Enoch)

California State University Systemwide Evaluation of Professional
 Teacher Preparation (uses reliable and valid survey instrument;            None
   includes CSU graduate survey data on teacher effectiveness)
California State University Systemwide Evaluation of Professional
 Teacher Preparation (uses reliable and valid survey instrument;
                                                                          None
     includes supervisory administrator survey data on teacher
                           effectiveness)
 Self-reflective assessment using Attributes of Highly Effective
                                                                  AHET (p. 10 of this link)
                         Teachers (AHET)
          California Teaching Performance Expectations                      CTPEs

NY State required prospective teacher tests assessing content and
 pedagogical knowledge (New York State Teacher Certification                 None
                            Exams)
      K-12 pupil performance on standardized math tests                      None
     K-12 pupil performance on standardized science tests                    None
  Framework for Teaching (FFT) classroom observation rubric
                                                                         FFT (Excerpt)
                       (Danielson, 1996)
                NTE exam (required for licensure)                            None
Semi-annual student achievement scores (standardized state tests?)           None
         Quarterly student classroom performance data                        None
           Knowledge, Skills, and Dispositions (KSDs)                        KSDs

                           GRE Scores                                        None
                         Praxis II Results                                   None

                         Praxis II Results                                   None
Demonstration and documentation of evidence of knowledge and
   skills across the ten Principles of Interstate New Teacher            INTASC Core
        Assessment and Support Consortium (INTASC)
              K-12 student standardized test scores                          None
                         Mailed Surveys                                  Mailed Survey

ISU’s Unit Assessment System (UAS) (using portfolio Live Text
                                                                             UAS
            Software) to track Noyce student progress
          K-12 students ISTEP+ exam results in Math                          None
         K-12 students ISTEP+ exam results in Science                        None
K-12 students NWEA Measures of Academic Progress (MAP) test
                                                                             None
                              scores
K-12 students NWEA Measures of Academic Progress (MAP) test
                                                                             None
                              scores
   Assessment of program-completing scholars guided by the
  Interstate New Teacher Assessment and Support Consortium            INTASC Core
                          (INTASC)
  Classroom Observation Protocol (COP) designed by Horizon
                                                                      Horizon COP
                     Research Inc. for NSF
                          Praxis scores                                  None
  National Board for Professional Teaching Standards (NBPTS)
                                                                         None
                         Certification data
Unit performance (NCATE reviews, university review of strategic
                                                                         None
                        plans and progress)
  Noyce student opinions of “reformed” teaching techniques and
philosophical leanings on key MMSTEC values (using many items            None
                    from CETP Core surveys)
             NSF-CETP Core Observation Protocol                        CETPCOP

Student Teacher Appraisal Report (STAR) to evaluate recipients’
                                                                         None
                             teaching
Teacher Technology Survey (South Central Regional Technology
                                                                         None
                in Education Consortium, 2004)
 Texas Examination of Educator Standards (TExES) state test of
                                                                         None
                     content and pedagogy
  Classroom Environment Scale (CES) developed by Moos and
                                                                        Citations
                         Tricket (2002)
Texas Assessment of Knowledge and Skills (TAKS) to measure
                                                                         None
     achievement of K-12 students in recipients’ classrooms
Professional Development and Appraisal System (PDAS) (Texas
 Education Agency, 2003) measures teacher responsibilities and           None
                   instructional effectiveness
           Noyce Student Survey at time of the award                Not Available Yet

         Noyce Student Survey after one year of teaching            Not Available Yet

       Survey for department chair at new teacher’s school          Not Available Yet
    Texas Higher Education Assessment (THEA) and other
                                                                         None
                     standardized test scores
Student reading standardized test scores of Noyce-taught students        None
 Student math standardized test scores of Noyce-taught students          None
Student science standardized test scores of Noyce-taught students        None
Standards-based observation protocols (e.g., Arizona Observation
                                                                         RTOP
          Protocol) to evaluate classroom performance
 University’s Office of Institutional Research (UOIR) developed
                                                                           None
                               surveys
      SAT II Math performance of UTeach program students                   None
 UTeach student performance on the ExCET teacher certification
                                                                           None
                          examination
                       Application Form                               Not Available Yet
 Survey of Locus of Control (Completed upon program entry and
                                                                      Not Available Yet
                          completion)
Survey of Locus of Control (Completed after first year of teaching)   Not Available Yet
   Mathematics Majors Survey of Beliefs about Teaching and how
 they change as students progress through the program (Completed      Not Available Yet
                 upon program entry and completion)
   Mathematics Majors Survey of Beliefs about Teaching and how
 they change as students progress through the program (Completed      Not Available Yet
                      after first year of teaching )
   Science Majors Survey of Beliefs about Teaching and how they
change as students progress through the program (Completed upon       Not Available Yet
                   program entry and completion )
   Science Majors Survey of Beliefs about Teaching and how they
change as students progress through the program (Completed after      Not Available Yet
                         first year of teaching )
  Survey of Self-Efficacy to Manage Student Behavior (Completed
                                                                      Not Available Yet
                 upon program entry and completion )
  Survey of Self-Efficacy to Manage Student Behavior (Completed
                                                                      Not Available Yet
                      after first year of teaching )
Survey of Commitment to Teaching in Urban Settings (Completed
                                                                      Not Available Yet
                 upon program entry and completion )
Survey of Commitment to Teaching in Urban Settings (Completed
                                                                      Not Available Yet
                      after first year of teaching )
  Survey of Perceptions of Barriers to Reaching Children in Urban
                                                                      Not Available Yet
      Settings (Completed upon program entry and completion )
  Survey of Perceptions of Barriers to Reaching Children in Urban
                                                                      Not Available Yet
           Settings (Completed after first year of teaching )
Additional Links      Content          Construct         Method


  ARMT Website      Mathematics       Achievement          Test

      None          Mathematics       Achievement          Test

TEAM-Math Website        ?                 ?                ?


      None            Science           Efficacy          Survey


      None               ?                 ?            Observation

      None               ?          Classroom Ecology   Observation

  FCEPT Home        Mathematics                           Survey

  FCEPT Home          Science                             Survey


  FCEPT Home        Mathematics            ?              Survey



  FCEPT Home          Science              ?              Survey

      None          Mathematics       Achievement          Test
      None           Science          Achievement          Test
                                        Program
      None           Pedagogy                             Survey
                                      Effectiveness

                                        Program
      None            Science                             Survey
                                      Effectiveness

  Multiple Links      Science           Efficacy          Survey


                                        Teacher
  Multiple Links    Not Specified                         Survey
                                      Effectiveness
                                        Teacher
 Multiple Links    Not Specified                             Survey
                                      Effectiveness

                                        Teacher
     None          Not Specified                             Survey
                                      Effectiveness
                                        Teacher
CTPEs (Detailed)    Pedagogy                                Criteria
                                      Effectiveness

   NYSTCE           Pedagogy       Teacher Knowledge          Test

       ?           Mathematics        Achievement             Test
       ?            Science           Achievement             Test
                                        Teacher
     None          Not Specified                          Observation
                                      Effectiveness
       ?            Pedagogy       Teacher Knowledge          Test
     None          Not Specified      Achievement             Test
     None          Not Specified      Achievement             Test
                                        Teacher
     None           Pedagogy                             Rubric Analysis
                                      Effectiveness
     GRE           Not Specified        Aptitude              Test
                     Subject
    Praxis II                         Competence              Test
                    Knowledge
    Praxis II       Pedagogy          Competence              Test


 INTASC Math       Mathematics        Competence            Criteria

     None          Not Specified      Achievement             Test
                                       Transition
     None          Not Specified                             Survey
                                       experience
UAS Article Link   Not Specified   Progress in Program      Criteria
    ISTEP+         Mathematics        Achievement             Test
    ISTEP+          Science           Achievement             Test
  NWEA MAP         Mathematics        Achievement             Test

  NWEA MAP           Science          Achievement             Test
  INTASC Math       Mathematics        Competence         Criteria

                                        Teacher
  Multiple Links      Science                           Observation
                                      Effectiveness
                      Subject
     Praxis                            Competence           Test
                     Knowledge
                                        Teacher
     NBPTS           Pedagogy                             Criteria
                                      Effectiveness
                                        Program
     NCATE          Not Specified                         Criteria
                                      Effectiveness

     None           Not Specified        Attitude          Survey

                                        Teacher
  Multiple Links     Pedagogy                           Observation
                                      Effectiveness
                                        Teacher
     None            Pedagogy                           Not Specified
                                      Effectiveness
SCRTE Information    Pedagogy         Not Specified        Survey
                     Content &
TExES Information                   Teacher Knowledge       Test
                     Pedagogy
    Website          Pedagogy       Classroom Ecology   Not Specified

TAKS Information    Not Specified     Achievement           Test

                                        Teacher
PDAS Information     Pedagogy                           Not Specified
                                      Effectiveness

     None            Pedagogy         Self-Reflection      Survey

     None            Pedagogy         Self-Reflection      Survey
                                        Teacher
     None            Pedagogy                              Survey
                                      Effectiveness
     THEA           Mathematics          Aptitude           Test
     None            Reading          Achievement           Test
     None           Mathematics       Achievement           Test
     None            Science          Achievement           Test
                                        Teacher
  Multiple Links    Not Specified                       Observation
                                      Effectiveness
Institutional Research   Not Specified    Not Specified     Survey

     SAT Math            Mathematics      Achievement        Test

       ExCET              Pedagogy         Competence        Test
        None             Demographics     Not Specified     Form
        None              Pedagogy       Locus of Control   Survey

        None              Pedagogy       Locus of Control   Survey

                                          Perception of
        None             Mathematics                        Survey
                                           Profession

                                          Perception of
        None             Mathematics                        Survey
                                           Profession

                                          Perception of
        None               Science                          Survey
                                           Profession

                                          Perception of
        None               Science                          Survey
                                           Profession

        None              Pedagogy           Efficacy       Survey

        None              Pedagogy           Efficacy       Survey

        None             Not Specified    Commitment        Survey

        None             Not Specified    Commitment        Survey

        None              Pedagogy         Competence       Survey

        None              Pedagogy         Competence       Survey
   Target


K-12 students

K-12 students
 Beginning
  Teachers
 Beginning
  Teachers
Beginning
 Teachers
Beginning
 Teachers
Pre-service
 Teachers
Pre-service
 Teachers

  District
 Personnel


  District
 Personnel

K-12 students
K-12 students
 Beginning
  Teachers

 Beginning
  Teachers

Pre-service
 Teachers

 Beginning
  Teachers
  District
 Personnel


 Candidates
 Beginning
  Teachers
Pre-service
 Teachers
K-12 students
K-12 students
 Beginning
  Teachers
 Pre-service
  Teachers
K-12 students
K-12 students
 Pre-service
  Teachers
 Candidates
 Pre-service
  Teachers
 Pre-service
  Teachers
Pre-service
 Teachers
K-12 students
 Beginning
  Teachers
 Pre-service
  Teachers
K-12 students
K-12 students

K-12 students

K-12 students
Pre-service
 Teachers
Beginning
 Teachers
Pre-service
 Teachers
Beginning
 Teachers
  Program

Pre-service
 Teachers
 Beginning
  Teachers
 Beginning
  Teachers
Not Specified
Pre-service
 Teachers
Not Specified

K-12 students

 Beginning
  Teachers
 Pre-service
  Teachers
 Beginning
  Teachers
   District
  Personnel
 Pre-service
  Teachers
K-12 students
K-12 students
K-12 students
 Beginning
  Teachers
Not Specified
Beginning
 Teachers
Pre-service
 Teachers
Candidates
Pre-service
 Teachers
Beginning
 Teachers
Pre-service
 Teachers

 Beginning
  Teachers

Pre-service
 Teachers

 Beginning
  Teachers
Pre-service
 Teachers
Beginning
 Teachers
Pre-service
 Teachers
Beginning
 Teachers
Pre-service
 Teachers
Beginning
 Teachers
                                            Instrument(s)

Cal State-Fresno (FCEPT) Survey Themes:
What influenced Noyce recipients’ decisions to pursue a teaching career?
What are their goals?
How comfortable are they with science/math?
How well prepared do they feel to teach math and science?

Cal State-Fresno (SMEC) Survey Questions
Are more positive views of science and mathematics being transmitted to students?
How has the program impacted pre-service teachers?
Do they feel more confident to teach math and science within the 7-12?
Are they staying in teaching longer?
Are they becoming leaders at their school sites?
Are their students achieving in math and science?
California State (San Marcos)
California State University Systemwide Evaluation of Professional Teacher Preparation (SEPTP)
California State University Systemwide Evaluation of Professional Teacher Preparation (SEPTP)
Horizon Research, Inc. Classroom Observation Protocol (COP)
NSF-CETP Core Observation Protocol
Arizona Observation Protocol (Revised Teacher Observation Protocol-RTOP)
Links to Additional   Links to Additional    Links to Additional
                                                                     ReturnLink to Main
   Instrument            Instrument             Instrument
                                                                          Database
   Information           Information            Information


                                                                   To FCEPT Survey Themes
   FCEPT Home
                                                                         Main Sheet




   FCEPT Home                                                      To SMEC Survey Main Sheet




 Review of Research   Research Instruments                         To Cal State-SM Main Sheet
       None                SEPTP_1                SEPTP_2            To SEPTP Main Sheet
       None                SEPTP_1                SEPTP_2            To SEPTP Main Sheet
   Horizon COAP          Horizon Home                                   COP Main Sheet
 COP Training Guide     COP Video Guide                            NSF-CETP COP Main Sheet
     Ref Manual          Training Guide       Manual Coversheet     (AOP) RTOP Main Sheet

								
To top