Evaluation by 7FvO6g3

VIEWS: 4 PAGES: 25

									                      Access to the World and Its Languages


INTERNATIONAL EDUCATION
PROGRAMS SERVICE



Evaluation
      CIBE, FLAS, IIPP, LRC, NRC
      Technical Assistance Workshop
      February 2-3, 2009




                  Access to the World and Its Languages
                                           www.ed.gov/ope/iegps
                                  www.ed.gov/ope/iegps
Title VI 50th
Anniversary Update
                          Access to the World and Its Languages

• Madeline Albright is the Keynote
  speaker
• Please make separate reservations
  – Conference registration
  – Hotel registration
  – Rooms are going quickly
• Contacts Rob Glew and Tomas Hult,
  Michigan State University
• http://titlevi50th.msu.edu

                                      www.ed.gov/ope/iegps
Evaluation
             Access to the World and Its Languages




             Love

                         www.ed.gov/ope/iegps
Panelists
                        Access to the World and Its Languages



• Eleanor Briscoe, Senior Budget Analyst
• John Clement, Director, Strategic
  Planning Services
• Susan Lehmann, Evaluation Analyst,
  FIPSE
• Karla Ver Bryck Block, Senior Advisor,
  IEPS


                                    www.ed.gov/ope/iegps
Why???
                                 Access to the World and Its Languages


• ED believes in and supports high quality
  program evaluations to help inform budgetary
  and program decisions

• ED Strategic Plan
  – Goal 3, Objective 1: Increase success in and
    completion of quality postsecondary education.
     • Strategy 3. Prepare more graduates for
       employment in areas of vital interest to the
       United States, especially in critical-need
       languages, mathematics, and the sciences.



                                             www.ed.gov/ope/iegps
Administration Update
                    Access to the World and Its Languages



• “Accountability and Transparency”
• GPRA continues
• PART may or may not continue but
  the need to measure program
  effectiveness remains



                                www.ed.gov/ope/iegps
Congress
                       Access to the World and Its Languages

• Clear interest in program
  evaluations
• Directed a $1.5 million review of
  Title VI and Fulbright-Hays
  programs by the National Research
  Council of the National Academies
  – International Education and Foreign
    Language – Keys to Securing
    America’s Future (March 2007)

                                   www.ed.gov/ope/iegps
Keys to Securing
America’s Future
                       Access to the World and Its Languages



• National Academies
  recommendation
  – The Department of Education should
    commission independent outcome
    and impact evaluations of all
    programs every 4 to 5 years




                                   www.ed.gov/ope/iegps
IEPS Program Level
Initiatives
                           Access to the World and Its Languages

• Evaluations are underway with AIR
  – Language Resource Centers
  – Fulbright-Hays Group Projects Abroad –
    Overseas Intensive Language Projects
• Analysis of the International Research
  and Studies Program is concluding
• National Resource Centers are next
• Future plans include Centers for
  International Business Education


                                       www.ed.gov/ope/iegps
IRIS
                         Access to the World and Its Languages

• IEPS gathers information via IRIS that
  is used for program evaluations
• Breakouts for 4 programs today (CIBE,
  LRC) and tomorrow (NRC, FLAS) to
  discuss:
  – Current and future program performance
    and efficiency measures
  – Changes to IRIS for the next clearance
  – Evaluation efforts
  – Trivia, prizes, chocolate!!!


                                     www.ed.gov/ope/iegps
Keys to Securing
America’s Future
                          Access to the World and Its Languages



• National Academies Recommendation
  – The Department of Education should work
    with universities to create a system of
    continuous improvement for Title VI and
    Fulbright-Hays programs. The system
    would help develop performance indicators
    and other improvement tools and should
    include networks of similar centers and
    university officials with overall
    responsibilities in language, area, and
    international studies.

                                      www.ed.gov/ope/iegps
Your Project
                      Access to the World and Its Languages



• Think…
  – Goal-oriented
  – Data driven

   ED does not have specific models,
   timelines, budgets, etc for your
   evaluation plans. Develop a plan that
   serves your needs.

                                  www.ed.gov/ope/iegps
Performance
Objectives & Measures
                           Access to the World and Its Languages



• A measurable indication used to
  determine how well goals are being met
  – How will progress be measured?
  – How much progress will constitute success?
  – How will it be known if an objective or part
    of an objective has been achieved?
• Are we measuring process or
  outcome?


                                       www.ed.gov/ope/iegps
Objective & Measure
Types
                        Access to the World and Its Languages



• A specific achievement/accomplishment

• Change in a level of
  achievement/accomplishment

• Achievement/accomplishment (or change)
  compared with other students or
  institutions



                                    www.ed.gov/ope/iegps
Components of
Objectives & Measures
                    Access to the World and Its Languages



• What will change
• How much change is expected
• Who will achieve the change
• When the change will take place




                                www.ed.gov/ope/iegps
Performance Objective
Example
                            Access to the World and Its Languages

• Help local businesses to
  internationalize
 “20% (how much) of local businesses with
 more than 200 employees (who) will open an
 office in a foreign country or enter into a joint
 venture with a foreign company (what) within
 two years (when).”




                                        www.ed.gov/ope/iegps
Performance Objective
Example
                         Access to the World and Its Languages

• Teacher workshop –
    Developing Materials for LCTLs
 “50 (how much) LCTL teaching professionals
 (who) will complete LCTL workshop (what)
 annually (when).”

 “90% (how much) of LCTL workshop
 participants (who) respond with 4 or higher
 on survey (what) one year after completing
 program (when).”

                                     www.ed.gov/ope/iegps
Performance Measure
Example
                         Access to the World and Its Languages


• Revise public health curriculum
  to enhance international
  knowledge
 “The required MPH courses (who) will be
 revised (what) to cover the 10 important
 international health areas (how much) by
 next fall (when).”




                                     www.ed.gov/ope/iegps
Performance Measure
Example
                         Access to the World and Its Languages




• Improve enrollment in
  summer language institute
 “Undergraduate student enrollment in the
 summer language institute (who) will
 increase (what) by five percent (how
 much) next summer (when).”




                                     www.ed.gov/ope/iegps
 Improving
 Performance Measures
                                  Access to the World and Its Languages
• Original Performance Measure: Provide
  overseas internships.

• What/Who:
  – Provide dynamic student internships with partners in a
    foreign country.

• When:
  – Provide dynamic student internships with partners in a
    foreign country each semester.

• How Much:
  – 20 students will engage in dynamic student internships
    with partners in a foreign country each semester.

                                              www.ed.gov/ope/iegps
Improving
Performance Measures
                                 Access to the World and Its Languages

• Original Performance Measure: Enhance
  Business Resource Center.

• What/Who:
  – Enhance Business Resource Center to advise
    businesses on international trade regulations.


• When/How Much:
  – Business Resource Center documents and staff will
    be updated on new international trade regulations
    within one week of passage.



                                             www.ed.gov/ope/iegps
Remember!!!
                     Access to the World and Its Languages



• Activities are not performance
  measures!




                                 www.ed.gov/ope/iegps
Resolving the “Activity-
as-Measures” Dilemma
                            Access to the World and Its Languages



• “Create opportunities for local faculty to
  strengthen area skills.” (activity)
  – Create “visiting” faculty training programs
    and train 2 per year. (outcome)


• “Enhance study abroad.” (activity)
  – Increase number of students studying
    abroad by X% each year. (outcome)


                                        www.ed.gov/ope/iegps
 Summary: Good
 Objectives & Measures
                               Access to the World and Its Languages

• Each objective should have one or a few specific
  performance measures to demonstrate how
  progress will be measured toward meeting the
  objective

• More objectives and measures ≠ better
  objectives and measures!

• Ensure data is collectible

• Quantify and define success

                                           www.ed.gov/ope/iegps
Summary: Strong
Evaluation
                         Access to the World and Its Languages
• Partner with a competent evaluator
• Develop quantifiable objectives and
  measures
• Measure what is important
• Measure what can be changed
• Get frequent progress reports
• Keep key stakeholders informed of
  results
• Use results to adjust project regularly
• Keep analyzing…
                                     www.ed.gov/ope/iegps

								
To top