UNIVERSITY OF ARKANSAS AT LITTLE ROCK Assessment by knb11734

VIEWS: 0 PAGES: 10

									UNIVERSITY OF ARKANSAS AT LITTLE ROCK                          Plan No. 70
Assessment Progress Report Form (Calendar Year 2005)

• Due To: Dean’s office                         On: March 3, 2006
 (college or law school office)          (date to be set by college or law school)


 Degree Program: Learning Systems Technology

 Department and College: Department of Ed. Leadership, College of Education

  Circle one: AA      AS BA BS BBA BSE BSW (Master's) EdS                    JD   EdD
PhD

 Prepared by: David S. Spillers
                                             Learning Systems Technology –M.Ed. Curriculum and Assessment Matrix
                                                                           4/3/06



Required Courses        How Outcomes are   Outcome # 1       Outcome # 2            Outcome #3 Be         Outcome# 4             Outcome # 5
and Activities          Addressed and      The graduate      Be able to develop a   able to design an     Translate objectives   Design a plan for for
                        Assessed           will be able to   systems approach in    instructional plan.   into concrete,         support services.
                                           determine entry   creating outcomes.                           mediated learning
                                           levels of                                                      applications.
                                           students..
LSTE 7303
Technology and          Emphasis:          Somewhat          Somewhat               Somewhat              Somewhat               none
Society                 Assessed:          Exams             Exams                  Exams                 Create presentations

LSTE 7305 Survey of
Computer-based          Emphasis:          Extensive         Extensive              Extensive             Extensive              none
Learning Systems        Assessed:          Lesson Plans      Lesson Plans           Lesson Plans          Create Tutorials
                                                                                    Flow charts
LSTE 7310 Systematic    Emphasis:          Extensive         Extensive              Extensive             Extensive
Integration of          Assessed:          Lesson Plans      Lesson Plans           Lesson Plans          Lesson Plans           Somewhat
Technology in                                                Paper                                        Tutorials              Floor plans
Learning Settings                                                                                                                Exams

LSTE 7320 Intranet
and Internet Learning   Emphasis:          None              Extensive              Extensive             Somewhat               Extensive
Systems                 Assessed:                            Exams                  Web pages             Exams                  Web site construction
                                                                                    Exams                                        Facility plans
LSTE 7325
Assessment in           Emphasis:          Extensive         Somewhat               Little                none                   Extensive
Learning Systems        Assessed:          Exams             Exams                  Not assessed                                 Exams, Research
Technology                                                                                                                       Project

LSTE 7330 Distance      Emphasis:          Somewhat          Somewhat               Extensive             Extensive              Somewhat
Learning Systems        Assessed:          Exams             HTML tutorials         Exams, Tutorials      Exams, Tutorials       Delivery Mapping
Technology
LSTE 7350 Practicum     Emphasis:          None                     None            None                  none                   Extensive
                        Assessed:                                                                                                On site experiences
EDFN 7303               Emphasis:          Not Assessed         Not Assessed        Not Assessed          Not Assessed           Not Assessed
Introduction to         Assessed:
Research and Its
Applications
EDFN 7313 Theories      Emphasis:          Not Assessed         Not Assessed        Not Assessed          Not Assessed           Not Assessed
and Principles of       Assessed:
instruction
EDFN 7314 Cognition     Emphasis:          Not Assessed         Not Assessed                              Not Assessed           Not Assessed
and Instruction         Assessed:
                                  Learning Systems Technology
                                M.Ed. Assessment Progress Report
                                  For the 2005 Academic Year
                                             4/3/06
Caveats for this report:
         1. This program is not a teacher certification program.
         2. This report is merely a progress report and not the creation of a new Assessment tool.
         3. Of the five program objectives, only objectives 3-5 are addressed since the first two have been
         addressed in previous reports.
         4. The Educational Foundations courses were not assessed directly since these courses are
         essentially pre-requisites for understanding the content of the LSTE courses.
         5. All of the LSTE courses are delivered on-line and are operated on a “mastery” philosophy
         which means that the students continue in producing the materials and solving the problems until
         accepted by the instructor based on the specified outcomes (for an example of these specified
         outcomes see the section titled, “Assessment Methods” below).


Program Mission

           a. Central Educational Mission of LSTE Program
The Learning Systems Technology (LSTE) program's mission is to prepare instructional designers and
facilities/resources administrators for careers in public schools, community colleges, higher education
institutions, business, industry, government, and medical settings or facilities. Specifically, the program
will enable instructional designers to act in teaching and administrative roles in order to analyze problems
and apply solutions for learning including planning, preparation, implementation, evaluation, and
management.

          b. Relation to UALR's Mission
This program strives to support the University's mission for the application of UALR resources to the
community in the preparation of instructional personnel who in turn foster a community of learning by
providing the necessary vehicles for a broad spectrum of learning applications. In addition, the program
specifically supports the development of teaching skills using a systems design approach based on the
awareness of the ways humans learn as well as the development and enhancement of resources for effective
instruction.

          c. Relation to College of Ed. Mission
Generally the program supports the COE mission in the professional development of individuals concerned
with education in a variety of settings. Specifically, the program's mission directly supports the COE
statement, "We strive to develop professionals who use state-of-the-art methodologies and technologies to
serve, to promote, and to strengthen the professional development of individuals concerned with education
and human resource development in a variety of settings such as schools, colleges and universities, private
and corporate organizations, and government agencies." By training personnel with the techniques for
utilizing modern instructional media based on sound instructional theory and practice, the program is an
integral part of the mission of the College of Education.

Program Conceptual Framework

The conceptual framework supported by the faculty in the LSTE program is a blend of mastery learning
based on humanistic-oriented constructivism. This approach was adopted as the effect of deeper attention to
a commitment to diversity. Mastery learning is used in the sense that the learning involved will be different
for different learners (Diversity), but that all learners meet a level of expectation defined in rubrics by the
facilitator of learning (Coherence), the teacher. Constructivism is the basic vehicle for the practice of
mastery learning in that the students "produce," "manage," and "implement" learning experiences for
themselves and others (Commitment and Disposition). The definition below is displayed in the executive
summary for Transylvania University's teacher education:

         "Constructivism is the educational philosophy based on the idea that all individuals
         are by nature "meaning-seekers." Constructivism is often contrasted to behaviorism
         whose adherents tend to divide the concepts to be learned into small pieces or steps
         to be mastered, and, then, they condition learners, usually through rewards (or
         punishments), to acquire the necessary skill or idea. In contrast, advocates of
         constructivism take a holistic view of knowledge and learning and seek to provide
         experiences in which the learners are actively involved in seeking meaning. Meaning
         is contextualized and rooted in experience. Research indicates that passive learning
         and memorization does not lead to understanding and long-term retention. On the
         other hand, considerable research indicates that, "active engagement in learning may
         lead to better retention, understanding, and active use of knowledge" (Perkins 8).
         Constructivists also acknowledge a social dimension to learning, advocating
         collaboration and cooperative learning, when appropriate.

         Humanistic learning climates provide a "safe" place for learners to ask questions, to
         research and to seek meaning. In an atmosphere of respect for each learner, the
         teacher seeks to form a learning community. Such a learning community is taught
         how to question, to solve conflict, and to dare to risk what might be a "wrong"
         answer or hypothesis in a search to understand. The diversity of the group is valued
         and celebrated.

         The learners are given responsibility for managing and owning their learning and
         behavior, with the teacher serving as the guide and facilitator."

Thus, the terminology, "mastery learning based on humanistic-oriented constructivism," best describes the
conceptual framework for the faculty and the students within this program. This definition is coupled with
the ISTE standards adopted by the program faculty for the preparation of teachers and instructional
personnel as well as the College of Education’s conceptual framework for developing specialized expertise,
professional development, communication, diversity and technology. (To see how the COE conceptual
framework is incorporated into the evaluation of the capstone experience of the portfolio presentation look
at http://edlrlab.ualr.edu/LSTE/PortfolioGuidelines.html and
http://edlrlab.ualr.edu/LSTE/PortfolioEvalForm.html.

Program Learning Goals:

For the Students

Goal#1:In the area of Instructional Program Development the student will be faced with a consideration of
the broad problem of developing a complete system of instruction; a total application of technology and
mediated instruction to facilitate learning.

Goal#2:In the area of Educational Technology Product Development the student will practice the creation
of packages of mediated instruction and the translation of specific instructional objectives into concrete
items that facilitate learning.

Goal#3:In the area of Educational Technology Management the student will conduct an investigation of
support services for both instructor and learner and consider principally a "responsive" service which
includes aspects of location, selection, acquisition, organizing, storage, retrieval, distribution, and
maintenance of both materials and devices.

For the Program

Goal #1:To prepare instructional designers and facilities/resources administrators for careers in public
schools, community colleges, higher education institutions, business, industry, government, and medical
settings or facilities.

Goal #2: To enable instructional designers to act in teaching and administrative roles in order to analyze
problems and apply solutions for learning including planning, preparation, implementation, evaluation, and
management.

Goal #3: To prepare instructional personnel who in turn foster a community of learning by providing the
necessary vehicles for a broad spectrum of learning applications.

Goal #4:To support the development of teaching skills using a systems design approach based on the
awareness of the ways humans learn, as well as the development and enhancement of resources for
effective instruction.

3. Program Learning Objectives

Objective #1:The graduate of this program will be able to assess and describe in at least two narrative
paragraphs the entry levels for a variety of learner populations by recording factors such as age, gender,
reading levels, math functioning levels, noticeable motivational aspects, ethnic structures, and physical
characteristics after viewing student files, viewing videotapes of such populations or visiting live sites of
practice.

Objective #2:The graduate of this program will be able to develop a systematic approach for determining
desired outcomes based on the mission specified by an institution which includes a listing of tasks
involved, a listing of the products desired, and a planning flowchart.

Objective #3:The graduate of this program will be able to design an overall plan for learning experiences in
order to reach the outcomes by selecting an appropriate application based on the readiness levels of the
learners, the nature of the outcomes, the environment constraints where the outcomes will be activated, and
the time frames involved.

Objective #4:The graduate of this program will be able to translate instructional objectives into concrete
applications by generating a variety of mediation samples which meet requirements for cognition by
learners. For example, the student will be able to generate an interactive PowerPoint lesson based on the
readiness levels of the intended audience and the expected outcomes involved.

Objective #5:The graduate of this program will be able to design a systematic plan for support services for
both the learner and the instructor which includes expected aspects or industry standards for access,
location, selection, acquisition, organizing, storage, retrieval, distribution, and maintenance of both
materials and devices.


Assessment Methods

The program goals are based on the International Society for Technology in Education Standards for Basic
Endorsement in Educational Computing and Technology Literacy (currently, an endorsement in the state of
Arkansas does not exist, however, the endorsement is a planned project by the State Department of
Education) and the ISTE Standards for Advanced Programs in Educational Computing and Technology
Leadership as noted above. Each objective is measured by the completion of production items, cognitive
test scores, judgments on research papers, and class participation in the online discussion forum, Web
Crossing. Rubrics for judgment are contained in each syllabus for each course. With the exception of the
portfolio and the cognitive testing, the “do until accepted” method is applied. Because of the nature of the
mastery learning approach as explained in the above conceptual framework, letter grades below a “B” are
very rare indeed. Essentially, lower grades are earned by non-submission or non-re-submission of some of
the course requirements.

As noted above, there are five general program objectives. The curriculum matrix (attached) displays the
level of treatment in the each of the courses within the program for the five objectives. Each objective is
approached from the perspective of a particular course. For example, the first course in the program
attends to objectives on a global/societal basis whereas the second course deals with specific computer
applications in learning settings. Therefore, as an example for all of the courses, the method of dealing
with the objectives at the course level, the objectives for the second course are presented here with the COE
conceptual frameworks and the goals for the program noted in the parentheses. (C=communication,
PD=professional development, SE=specialized expertise – the goal numbers are 1-4)

(The following are examples for only one course out of twelve LSTE courses.)
After completing this course the student will be able to perform the following-
1. Describe the development of computer technology and its applications to instruction over the past three
decades according to the textbook. (C, PD, SE, 1, 3)
2. Identify and briefly describe the internal and external components of a computer system using specific
keywords found in the text. (SE, 2)
3. Distinguish between computer assisted instruction (CAI) and computer managed instruction (CMI) using
a software evaluation model as a guide. (SE, 1,2,3,4)
4. Identify three instructional applications of computers in education and/or training other than CAI or CMI
according to the electronic lectures by choosing the correct application in the software evaluation form.
(SE, 3,4)
5. Outline the processes and materials needed to evaluate and select computer software using the software
evaluation form as the criteria. (SE, C, 2,4 )
6. Apply an "evaluation checklist" to a sample CAI program using the software evaluation form as the
criteria and email the result to the professor. (PD, SE, 4)
7. Describe a concomitant learning that may be developed through interaction with computer programs
according to the readings in the text and the electronic lectures. (C, 3)
8. Generate an example of the use of the computer for computer-assisted instruction to : a)individualize
learning, b)provide computer-based testing, and c)provide a student data file, and d)compute grade
averages all using the interactive applications taught in the course and according to the electronic lectures.
(SE, PD, 2,3,4)

All of the goals and objectives are assessed in the individual courses via, tests, research papers, product
development on a “do until accepted basis,” and adjudication of the portfolio presentation or the optional
comprehensive exam. In each course the objectives are assessed from the perspective of the purpose of the
course. As mentioned previously, the ISTE standards are used as guidelines and within the mastery
approach, these standards are either met or they are not met.

Stakeholders involved are the faculty, the students, the learned societies, and the practitioners. Feedback is
solicited from the students and practitioners using listserves such as the AAIM list (Arkansas Association
for Instructional Media) and electronic questionnaires. The faculty members are the sources for
determining the recommendations from the learned societies by their memberships and participation in
state, regional, and national conventions. Because instructional technology is becoming so common in our
society, the identification of the learned societies is increasingly complex. Almost every group, which
communicates via electronic means, has recommendations about instructional technology. Finally, there is
great confusion at the moment between the terms "instructional technology" and "informational
technology." This program stands for the application of technology to the teaching/learning processes based
on systematic approaches which are in turn, based on accepted learning theories.

The fundamental premise of "mastery learning" is a method that the faculty has chosen in the hope that all
learners in the program will be able to complete the program by "mastering" all of the concepts before
proceeding. Only by "giving up" or withdrawing from the program will a student not complete the program.

Reliability:
Reliability is problematic in this particular program because of the flux mentioned above, especially during
the time period of 1999-2001 because the national and state organizations as well as the learned societies
are also in a state of flux brought on by the infusion of digital technologies in all aspects of communication
and learning. Where the program faculty relied on the standards from single national and state
organizations and a learned society, the arrival of standards written at the national and state levels are now
extensive and even the learned societies number four. Nevertheless, the continued inclusion of the three
core areas of program development, product development and program management as demonstrated in the
sample questionnaire, provide a sense of reliability.

Validity:
Validity will be judged on the standards espoused for this program area from at least three sources:
International Society for Technology in Education, National Educational Technology standards for
Teachers, and Association for Educational Communications and Technology. The challenge to the program
faculty is to merge the standards and recommendations from these organizations with the perceptions and
recommendations of local and state practitioners. However, there is no current collection of perceptions and
recommendations by these practitioners which has been identified in a formal way. Therefore, the program
relies heavily on the standards and recommendations from the national sources mentioned above.

Summary:

The attached form titled, “Degree Program Assessment Progress Report for: Learning Systems
Technology,” indicates that the program outcomes 3, 4, and 5 were assessed in four different courses
during the 2005 academic year. Objectives 3 and 4 were met by all students who completed the LSTE
courses involved. Objective 5 was not met because of the level of understanding in the site managers or the
cooperating supervisor. Obviously, the program faculty must find ways to enhance the understanding of
the site managers as they supervise the students at the external site. It appears that most site managers
approach this activity as a “slave labor” opportunity rather than as an overall involvement in a practicing
instructional technology unit. The faculty is already examining open source software for collaboration
activities which will be used to involve the cooperating site managers in interactions for better
understanding the practicum experience.
                                             Learning Systems Technology –M.Ed. Curriculum and Assessment Matrix
                                                                           4/3/06



Required Courses        How Outcomes are   Outcome # 1       Outcome # 2            Outcome #3 Be         Outcome# 4             Outcome # 5
and Activities          Addressed and      The graduate      Be able to develop a   able to design an     Translate objectives   Design a plan for for
                        Assessed           will be able to   systems approach in    instructional plan.   into concrete,         support services.
                                           determine entry   creating outcomes.                           mediated learning
                                           levels of                                                      applications.
                                           students..
LSTE 7303
Technology and          Emphasis:          Somewhat          Somewhat               Somewhat              Somewhat               none
Society                 Assessed:          Exams             Exams                  Exams                 Create presentations

LSTE 7305 Survey of
Computer-based          Emphasis:          Extensive         Extensive              Extensive             Extensive              none
Learning Systems        Assessed:          Lesson Plans      Lesson Plans           Lesson Plans          Create Tutorials
                                                                                    Flow charts
LSTE 7310 Systematic    Emphasis:          Extensive         Extensive              Extensive             Extensive
Integration of          Assessed:          Lesson Plans      Lesson Plans           Lesson Plans          Lesson Plans           Somewhat
Technology in                                                Paper                                        Tutorials              Floor plans
Learning Settings                                                                                                                Exams

LSTE 7320 Intranet
and Internet Learning   Emphasis:          None              Extensive              Extensive             Somewhat               Extensive
Systems                 Assessed:                            Exams                  Web pages             Exams                  Web site construction
                                                                                    Exams                                        Facility plans
LSTE 7325
Assessment in           Emphasis:          Extensive         Somewhat               Little                none                   Extensive
Learning Systems        Assessed:          Exams             Exams                  Not assessed                                 Exams, Research
Technology                                                                                                                       Project

LSTE 7330 Distance      Emphasis:          Somewhat          Somewhat               Extensive             Extensive              Somewhat
Learning Systems        Assessed:          Exams             HTML tutorials         Exams, Tutorials      Exams, Tutorials       Delivery Mapping
Technology
LSTE 7350 Practicum     Emphasis:          None                     None            None                  none                   Extensive
                        Assessed:                                                                                                On site experiences
EDFN 7303               Emphasis:          Not Assessed         Not Assessed        Not Assessed          Not Assessed           Not Assessed
Introduction to         Assessed:
Research and Its
Applications
EDFN 7313 Theories      Emphasis:          Not Assessed         Not Assessed        Not Assessed          Not Assessed           Not Assessed
and Principles of       Assessed:
instruction
EDFN 7314 Cognition     Emphasis:          Not Assessed         Not Assessed                              Not Assessed           Not Assessed
and Instruction         Assessed:
        Degree Program Assessment Progress Report for: _Learning Systems Technology (LSTE)
                                              Calendar Year:__2005_______

               (2)            (3) Courses &            (4)Method:         (5) What are the assessment            (6) What conclusions were
gram           Associated     activities where                            findings?                              drawn and what decisions
(s)            learning       assessed:                                                                          were made as a result?
resse          outcomes/ob
is             jectives for
 :             those goals:
                                                                          Using the “mastery” system the         There is little need to
ome                                                    Lesson Plans,      lesson plans and tutorials reflected   change this overall
                              LSTE 7305, 7310          Tutorials          adequate learning for this outcome.    instructional practice.
                                                                          Much like outcome #3, this
                                                                          outcome was met in the creation of
                                                                          online tutorials and packaged
                                                                          tutorials containing the lesson        There is little need to
ome                                                    Lesson Plans,      plan, a flowchart, and the actual      change this overall
                              LSTE 7305, 7320          Tutorials,Exams    electronic tutorial.                   instructional practice.
                                                                                                                 The failure in meeting
                                                                                                                 this outcome appears to
                                                                                                                 be in the practicum
                                                                                                                 course. A plan will be
                                                                                                                 developed to include
                                                                                                                 cooperating site
                                                                                                                 managers in
                                                                                                                 constructing a
                                                                                                                 measurable experience
                                                                          The resulting floor plans and web      for the practicum. The
                                                       Floor plans, web   sites were adequate, but the           concept appears to be
ome                           LSTE 7310,7320,          sites,delivery     concepts in delivery mapping as        missing in the minds of
                              7350                     mapping            defined in the courses was not met.    the site managers.

        (Date submitted to college committee:_______________________ Date posted:__________________________)




        (LM:2/2/06)                              -1-
        Degree Program Assessment Progress Report for: _Learning Systems Technology (LSTE)
                                              Calendar Year:__2005_______

               (2)            (3) Courses &            (4)Method:         (5) What are the assessment            (6) What conclusions were
gram           Associated     activities where                            findings?                              drawn and what decisions
(s)            learning       assessed:                                                                          were made as a result?
resse          outcomes/ob
is             jectives for
 :             those goals:
                                                                          Using the “mastery” system the         There is little need to
ome                                                    Lesson Plans,      lesson plans and tutorials reflected   change this overall
                              LSTE 7305, 7310          Tutorials          adequate learning for this outcome.    instructional practice.
                                                                          Much like outcome #3, this
                                                                          outcome was met in the creation of
                                                                          online tutorials and packaged
                                                                          tutorials containing the lesson        There is little need to
ome                                                    Lesson Plans,      plan, a flowchart, and the actual      change this overall
                              LSTE 7305, 7320          Tutorials,Exams    electronic tutorial.                   instructional practice.
                                                                                                                 The failure in meeting
                                                                                                                 this outcome appears to
                                                                                                                 be in the practicum
                                                                                                                 course. A plan will be
                                                                                                                 developed to include
                                                                                                                 cooperating site
                                                                                                                 managers in
                                                                                                                 constructing a
                                                                                                                 measurable experience
                                                                          The resulting floor plans and web      for the practicum. The
                                                       Floor plans, web   sites were adequate, but the           concept appears to be
ome                           LSTE 7310,7320,          sites,delivery     concepts in delivery mapping as        missing in the minds of
                              7350                     mapping            defined in the courses was not met.    the site managers.

        (Date submitted to college committee:_______________________ Date posted:__________________________)




        (LM:2/2/06)                              -1-

								
To top