Points for Knowledge Survey Design
Shared by: bgu15404
Points for Knowledge Survey Design Concept of a Knowledge Survey 1. = I have insufficient knowledge to answer this question. 2. = I have partial knowledge or know where to quickly (20 minutes or less) obtain a complete answer to this question. 3. = I can fully answer this question with my present knowledge. A Framework for Planning •Responsibilities –Goals •Outcomes –How would I confirm student mastery? (questions, challenges, applications, skills) »Organize items in order of course presentation Responsibility Example --to meet a published goal statement "Goal 5: To understand how the physical sciences explain the natural world. Courses in the physical sciences which fulfill this requirement (1) examine the processes by which scientific knowledge is gained, (2) introduce the basic concepts and terminology of one or more of the physical sciences, and (3) explore how scientific knowledge influences human society." (Idaho State University Undergraduate Catalog, 2006) Important Definitions • GOAL: A general descriptive statement statement of intent, reason, or vision for a particular educational experience. Example - the 12 General Education Goals in the ISU Catalog. • LEARNING OUTCOMES: Measurable skills or knowledge that are a result of instruction. These are action statements—verbs—that specify what students should be able to do at the end of a planned educational experience. These are the most important assessment criteria in accreditation reviews. Example--Drafted outcomes for the 12 goals now in notes. Example of outcomes: Students will be able to: 1. describe the scientific method and provide an example of its application; 2. pick a single theory from the science represented by this course and explain its historical development; 3. provide two examples of testable hypotheses; 4. provide two specific examples that illustrate why it is important to the everyday life of an educated person to be able to understand science; 5. describe two current examples of the relationship between physical science and public policy…. (4 more) Evaluation versus Assessment • Evaluation - addresses measures applied to make decisions about individuals such as tests, graded papers & projects--grades of students and summative ratings of individual professors. • Assessment - addresses measures of students’ success in meeting specific learning goals through performance on outcomes. Uses aggregated data to describe performance of units --courses, classes, programs, institutions--as a whole. Assessment Instruments Contrasted with Evaluation Tools • Assessments examine learning • Evaluations examine individuals’ outcomes in aggregate by unit performance (class, specific major, specific • Based mainly on instructors’ goals & institutional degree) outcomes • Based mainly on a unit’s chosen • Traditionally based on a few faculty- goals & outcomes made tools of undocumented • Where possible, based on multiple reliability tools of known reliability • Examples of evaluative input for • Examples: pre-post tests of grades: short-answer tests & quizzes, established reliability, knowledge in-class essays, graded homework, surveys, standardized exams, participation, attendance --less often concept inventories, records of based on open-ended challenges, success on registration exams or group projects, rubrics, service successful applications for learning, multifaceted competency certification levels • Work when they inform instructional • Work when they are aligned with clear planning and improve instructional information about goals and quality outcomes and are enacted with truth and fairness. For students, we must test on what we teach. Assessment Instruments Are Beneficial • When they promote students’ learning • When they assist instructional planning • When they aid curricular design • When they have high benefit:cost • When information provided is unique with only some overlap with (is not redundant with) other measures An Ideal Assessment Instrument • (1) furnishes unique data that provides some overlap with but does not duplicate that of other tools • (2) provides reliable, quantifiable data about student understanding • (3) provides data useful to students’ cognitive and meta-cognitive growth • (4) helps faculty improve course design and instruction • (5) helps units improve curricula. Knowledge Surveys • Are very reliable assessment • Are not test substitutes tools • Are not primarily evaluative • Obtain valuable, unique • Are not replacements for information, some of which summative and formative overlaps with that of tests, evaluations student ratings • Are optimal for assessment when • Reveal levels of course used with other tools to inform challenge overall interpretations • Information is obtained under • Are not simply passive different conditions from tests measures. Their effect to • Are amenable to numerical enhance teaching and learning analyses used for tests and depends on the degree to which student ratings they are skillfully employed. • Are useful to developing self assessment skills in students KS items are not interpreted in isolation from one another. Specific items map to into more global concepts. 10. Outline your understanding of geologic time and discuss how this course opened your mind to the notion of four- dimensional science. The following KS items mapped to the above Goal 5 outcome were: 14, 15, 16, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 78, 101, 115, 120, 121, 141, 147, 148, 151, 154, 155, 159, 185, 210, 211, 222, 238, 263, 268.