Document Sample
nsf03212 Powered By Docstoc
					The National Science Foundation promotes and advances scientific progress in the
United States by competitively awarding grants and cooperative agreements for research and education in the sciences, mathematics, and engineering. To get the latest information about program deadlines, to download copies of NSF publications, and to access abstracts of awards, visit the NSF Web Site at:
• • • •

Location: For General Information (NSF Information Center): TDD (for the hearing-impaired): To Order Publications or Forms: Send an e-mail to: or telephone:

4201 Wilson Blvd. Arlington, VA 22230 (703) 292-5111 (703) 292-5090 (301) 947-2722 (703) 292-5111


To Locate NSF Employees:

Transfer of Learning: Issues and Research Agenda *† Report of a Workshop held at the National Science Foundation March 21-22, 2002 Jose Mestre Department of Physics University of Massachusetts-Amherst

* Workshop supported by funds from the National Science Foundation. The views expressed herein do not necessarily reflect the views, position, or endorsement of the NSF.

This report is dedicated to the memory of Rodney R. Cocking, who was instrumental in planning this workshop but who suffered a tragic death a month prior to the workshop.

Transfer of Learning: Issues and Research Agenda
On March 21-22, 2002, a workshop on transfer of knowledge was held at the National Science Foundation in Arlington, Virginia. The workshop was funded by the NSF and organized by the late Rodney Cocking from NSF and by Jose Mestre from the University of Massachusetts. For one and one-half days 12 NSF program officers and 22 participants, representing diverse fields such as developmental, social and cognitive psychology, educational media/technology, cognitive science, and research into teaching and learning, met both to discuss past and present issues of transfer as it pertains to education and to generate a research agenda for the future. The National Science Foundation is planning to fund several Science of Learning Centers (SLCs) in the near future; it is hoped that the research agenda contained herein will be useful and informative for future research planned in the SLCs. We begin with a broad introduction of transfer of knowledge, discussing why it is both central and timely to the country’s educational mission in science, mathematics and engineering. Next the background leading up to the workshop is presented, including the governing charge and organizing themes. We then present the research agenda that resulted from the participant’s deliberations, and conclude with final remarks to place the research agenda in perspective. An appendix contains a complete participant list and the agenda followed in the workshop.

We define transfer of learning (hereafter transfer) broadly to mean the ability to apply knowledge or procedures learned in one context to new contexts. A distinction is commonly made between near and far transfer. The former consists of transfer from initial learning that is situated in a given setting to ones that are closely related. Far transfer refers both to the ability to use what was learned in one setting to a different one as well as the ability to solve novel problems that share a common structure with the knowledge initially acquired. We note that there is an emerging third way to talk about transfer, one that meets a criterion of generativity. The idea here is that learners can, on their own, come up with novel instances or solutions. Some kinds of expertise involve generativity. So do our common ability to talk, almost always using novel utterances in everyday conversations; learnings that constitute conceptual changes or reorganizations of knowledge as a function of instruction; and changes that can be related to advances in cognitive development. We will not attempt an extensive review of the current transfer research literature; we refer the interested reader to an excellent overview of the topic in chapter 3 of the National Research Council report, How People Learn (Bransford, Brown, & Cocking, 1999) as well to Brown and Campione (1998), and Brown, Bransford, Ferrara & Campione, (1983); for topics related to generativity see Carey and Spelke (1994) and Gelman and Williams (1998). An excellent article by Barnett and Ceci (2002) provides a literature review of the salient research on transfer, a taxonomy that can help organize the field, and directions for future study. Rather, we develop the case that a research agenda for transfer is both important and timely. It is important because of the pivotal role that transfer plays in education. The application of knowledge taught through schooling or training to contexts other than those in which the knowledge was initially learned is among the most important goals in education (Halpern, 1998). Yet, we are only beginning to see how to do the relevant research. A research agenda is timely because we now know much about human cognition (of which transfer is a


subset) and are in a position to consider ways of extending this body of work to include research that is directly aimed at finding ways to improve teaching and learning. We now expand on these two points. A major, but often tacit, assumption in education is that the knowledge that students learn in school will transfer to situations and problems encountered outside of school. Indeed much of our investment in education is justified in terms of preparing students for future learning so that they may become productive members in a society where workplace needs and demands are in constant flux. Clearly there is ample evidence for the usefulness of education in an advanced service-based society. In a variety of ways we know that schools do teach many the basic skills of reading, writing and reasoning. Witness the extent to which employers use educational level as one aspect of a job descriptions. Still, many students cannot meet these requirements, a fact that hampers their ability to advance in the workplace. There is also evidence that schooling improves IQ, as well as increases income (Ceci & Williams, 1997). Yet, despite these tacit assumptions, our educational system is inefficient at teaching in ways that promote transfer. Whether measured by standardized tests, or by laboratory studies of transfer, it is evident that transfer of knowledge is elusive. Some classic studies of analogical transfer illustrate that transfer of relevant knowledge from one situation to a second where the task is isomorphic (that is shares the same structure), but the context changes, is not common (Gick & Holyoak, 1980; Hayes & Simon, 1977; Reed, Dempster, & Ettinger, 1985; Reed, Ernst, & Banerji, 1974). Only after receiving hints pointing out that two situations are isomorphic are students able to transfer relevant knowledge. More recently Blanchette and Dunbar (2002) have found that, although students can spontaneously draw analogical inferences from one domain to another, they do not make enough inferences to support a fully fledged transfer from one domain to another. If, as these studies suggest, the ability to apply knowledge flexibly is so context-bound then an important question for education is how to structure instruction to ensure transfer, short of the impossible task of covering in detail all the relevant contexts in which the knowledge being taught could be applied. What, then, is known about successful transfer and what does it depend on? Research suggests that several factors affect transfer. First, some initial acquisition of knowledge is necessary for transfer (Brown, et al., 1983; Carey & Smith, 1993; Chi, 2000). Although this seems obvious, it is noteworthy that many failures to produce transfer have resulted from inadequate opportunities for students to learn effectively in the first place (e.g., see discussions by Brown, 1990; Klahr & Carver, 1988; Littlefield, Delclos, Lever, Clayton, Bransford, & Franks, 1988). Attention to initial learning is very important for transfer, especially when measures of transfer are used to evaluate the degree to which educational interventions are or are not “effective.” Whereas rote learning does not tend to facilitate transfer, learning with understanding does (Bransford, Stein, Vye, Franks, Auble, Mezynski, & Perfetto, 1983; Mandler & Orlich, 1993; see also literature review in Barnett & Ceci, 2002). Thus, attempts to learn too many topics too quickly may hamper transfer since the learner may simply be memorizing isolated facts with little opportunity to organize the learned material in any meaningful fashion or to link it to related knowledge. Although previous learning can enhance transfer, it can also obstruct it (Bransford, et al., 1999; Gelman & Lucariello, 2002). For example, new learning may not proceed rapidly if knowledge that the learner possesses that is relevant to the new learning remains inactivated; on the other hand, when tasks share cognitive elements, transfer is facilitated (Glaser, 1994; Singley & Anderson, 1989). This is true even for young children (Brown & Kane, 1988). 4

Context also plays a pivotal role in transfer. If the knowledge learned is too tightly bound to the context in which it was learned, transfer to superficially different contexts will be reduced significantly (Bjork & Richardson-Klavhen, 1989; Carraher, 1986; Eich, 1985; Lave, 1988; Mestre, 2002; Saxe, 1989). For example, students who learn to solve arithmetic progression problems can transfer the method they learned to solve similar physics problems involving velocity and distance, but students who learn to solve the physics problems first are unable to transfer the method to solve isomorphic arithmetic progression problems (Bassok & Holyoak, 1989). The transfer from physics to arithmetic was apparently blocked by embedding the physics equations within that specific context which then precluded students from seeing their applicability to another context. These findings also suggest that because students had more general knowledge about arithmetic/algebra, those who learned to solve the problems within a math context first were able to screen out the content-specific details of the problem-solving procedures, whereas those who learned to solve the physics problems first appear to attribute the underlying physics context as crucial to the application of the problemsolving procedures, and hence were unable to apply those procedures to a math context. Further, the context within which quantities/variables in a problem are presented affects transfer, as a study by Bassok (1990) demonstrated; students exhibited spontaneous transfer from problems involving speed (meters per second) to problems involving price rate measured in dollars per minute, but not to problems involving salary rate measured in dollars per year. It appears that dollars per minute was interpreted by students as a continuous rate similar to meters per second, but dollars per year was interpreted more like a discrete quantity rather than a rate, and hence the lack of transfer. Apart from factors that affect transfer, the issue of how to measure transfer has attracted recent attention (Bransford & Schwartz, 1999). These researchers observe that most measures of transfer use a paradigm of “sequestered problem solving” (SPS) in which people complete tasks isolated from additional knowledge resources that are typically available in non-laboratory settings. They question whether these SPS assessments have blinded the field to the role of transfer in facilitating subsequent learning, for example, when progressing from one mathematics class to the next. What was learned on day one in the math class may not produce strong transfer when tested in an SPS manner on day two. However, day one experiences may facilitate students’ abilities to learn new information on day two. An emphasis on facilitating learning is the hallmark of an approach to transfer that focuses on “preparation for future learning” (PFL) rather than only on SPS transfer per se. Overall, Bransford & Schwartz (1999) note that SPS measures of people’s abilities to retrieve spontaneously problem solving procedures and relevant facts in a new context is a useful measure of well-developed expertise, but it is often too blunt an instrument to measure the role of transfer for novices who know less about a subject area. Having a less blunt measure of transfer is important for educators, because transfer is one of the major criteria used to assess the relative effectiveness of different instructional procedures. For example, Schwartz & Bransford (1998) found that students who received different instructional treatments looked similar on standard knowledge assessments. However, when given an opportunity to learn from additional resources, one method of instruction looked far superior. If only SPS measures of transfer had been used, very different conclusions about instructional effectiveness would have been reached. PFL measures of transfer may also help the field differentiate good versus poor ways to “teach to the test.” Many ways of “teaching to the test” may prepare students to do well on SPS “near transfer” problems yet fail to prepare them effectively for future learning. The development of new 5

models of transfer-minded instruction and assessments that target PFL appears to be a fruitful focus for further research. In summary, research suggests that transfer is enhanced when the learner abstracts the deep principles underlying the knowledge being learned, and that abstraction is facilitated by opportunities to experience concepts and principles in multiple contexts. People’s prior knowledge and experience in a domain affects their subsequent transfer, although sometimes the effect is initially negative because previously learned concepts and routines must be changed to deal with new settings (e.g. Barnett & Ceci, 2002; Bransford, et al., 1999; Bransford & Schwartz, 1999; Hartnett & Gelman, 1998; Singley & Anderson, 1989). In educational settings, this is frequently referred to as “the implementation dip” or the “J curve effect” (e.g., Fullan, 2001). Despite our rapidly growing knowledge about transfer and factors that facilitate it, we know little about how to construct classroom environments for learning at levels that foster transfer. The question of how to formulate research questions so as to study in-class learning and transfer is just in its infancy (e.g., Silva & Kellman, 1999; Stigler & Fernandez, 1995). That is, how this research gets translated into better schooling remains to be investigated. There are some obvious obstacles to applying research findings in instructional settings to enhance transfer, such as the intransigence of the “teach-as-you-were-taught” cycle and the fact that there is often a long lag from when research findings are known to when they find their way to pre-service teacher education programs. But perhaps more importantly, there is also research evidence suggesting that in order to facilitate transfer we may need to structure instruction in ways that are counter-intuitive to both teacher and learner— counter-intuitive in that instructional changes suggested by research may fly in the face of tradition and common sense. Examples of this emerge in training motor skills where it has been known for some time that random practice is superior to blocked practice (Kerr & Booth, 1978; Shea & Morgan, 1979).* Yet, if participants are asked their perception as to which method is better for their own learning, they believe it to be blocked practice rather than random practice, a perception that is contrary to measured performance (Baddeley & Longman, 1978; Nitsch, 1977; Simon & Bjork, 2001). This erroneous perception may be shaped by tradition and our past educational experiences since blocked practice is how schooling is structured. That is, school curricula are structured in a sequenced (blocked) fashion where students learn topic A, then move to topic B, then C, etc.; yet, when students have to deal with all these topics simultaneously (e.g. at the end of the course or after the course is over) they falter unless they have had some random practice. In more cognitive tasks such as learning from textual material, research indicates that perceived efficiency or fluency may not be what is best for the type of learning that promotes problem solving and inference. For example, students given an outline of textual material prior to reading it perform inference and problem solving tasks better if the outline does not follow exactly the same organization as the text (Mannes & Kintsch, 1987). In addition, evidence also suggests that students possessing high knowledge in a domain do better at inference and problem solving after reading a passage related to that domain if the passage is sparse, requiring significant active processing of its meaning; this is in contrast to the poorer performance of students when the passage was much more complete, requiring less processing of its deep meaning (McNamara, Kintsch, Songer & Kintsch, 1996). Furthermore, research by Dunbar and colleagues has revealed that when students are required to process information for meaning there is a large influence on transfer. They found that students can 6

use underlying structural information when they are asked to generate analogies, but use superficial information when asked to recall similar information (Dunbar, 2001; Blanchette & Dunbar, 2000). These findings are consistent with a large body of research which shows that “effort after meaning” (or “generative processing”) facilitates learning (e.g., Auble & Franks, 1978; Bartlett, 1932; Slamecka & Graf, 1978; Wittrock, 1990). Another example of potentially “counter intuitive” approaches to instruction consists of beginning lessons by first having students generate their own, perhaps incorrect, thoughts about phenomena versus simply telling students the correct answers. One reason for advantages of the “generate first” approach is that it provides an opportunity for students to contrast their own thinking with that of others, including experts in an area. This sets the stage for appreciating the critical features of the new information that is presented to them. For example, Schwartz and Moore (1998) find that students are better prepared to appreciate the formula for variance if they first receive opportunities to generate their own (usually incorrect) ideas as they differentiate the elements of “spread” for which the formula accounts. To help differentiate these elements, students are shown an initial pair of distributions, say {2, 4, 6, 8, 10} and {4, 5, 6, 7, 8}. After pointing out to students that one number—the average—can be used to capture the similarity between these two sets, students are then asked to come up with a method for determining a single number for each set that could capture what is different (i.e., the variance). After they invent their own methods (often a range formula) they receive a new pair of distributions, say {2, 4, 6, 8, 10} and {2, 10} and determine whether their formula works for these sets as well, and if not to fix their formula. This continues for several cycles where students generate a formula and then try to apply it to new distributions that highlight new quantitative properties (like sample size and density). At the end of these exercises, students may be shown the formula for variance used by experts. The findings of Schwartz and Moore (1998) suggest that even though the students generated faulty formulas, these experiences helped students become aware of the quantitative properties of distributions that a formula should take into account. This set the stage for noticing critical features of experts’ formulas; for example, that it yields a smaller number for smaller variances; that it elegantly solves the problem of set size, and so forth. As a consequence, students in the “generate first” group were much better able to appreciate the strengths and weaknesses of different nonstandard formulas for capturing variance (e.g., a formula that summed the deviations from the median instead of the mean). In contrast, students who had been directly taught the standard formula (with no previous attempts to generate their own thoughts) simply declared that the non-standard formulas were “wrong.” They were not as prepared as the other students to learn about the expert formula. In Broudy’s terms (1977) they had a less well-differentiated “field” of knowledge for appreciating the nuances of the expert formula. What these previous examples suggest about effective learning does not reflect societal values, or even what we consider efficient in education. In school students study a topic until reaching some level of mastery and then move on to the next topic, whereas research suggests that transfer is improved by visiting the topics often rather than once intensely. We hold in high esteem and reward instructors who make clear, lucid, logical presentations that are totally understandable and easy to follow by the learner, yet research suggests that a) passively listening to an argument lacks the mental engagement necessary to promote deep learning, and b) some initial explorations on the part of the student when learning a new topic, even if those explorations generate erroneous notions, can be profitable in terms of eventually generating a deeper understanding of the topic. 7

Suggesting that the research cited earlier indicates that we should make textbooks and lectures more incoherent is clearly an overstatement. On the other hand, some middle ground is worthy of investigation. Perhaps clarity and coherence is most efficient at helping learners achieve some core knowledge, but after reaching some threshold knowledge it may serve the learner better to rely less on instruction and more on her/his own mental efforts to make sense and distill the knowledge into a form useful for future use.

The workshop was organized around the following three themes: 1. Context dependence of transfer. The acquisition of knowledge or a skill does not ensure that the knowledge or skill will be appropriately (or inappropriately) applied to relevant situations. What do we need to know about the contexts of learning or application of learning in order to facilitate transfer? 2. Conditions for transfer. What are the conditions that affect the appropriate (or inappropriate) application of knowledge or skill. What do we need to know about the contexts of learning, or application of learning, in order to facilitate transfer? 3. Metacognitive issues for transfer. Metacognitive research indicates that a significant part of effective learning is to be aware of, and in control of, one’s own learning. What research base is needed to promote reflective learning? What are the implications for classroom practice? Workshop participants were divided into three groups, with each group working on one of the themes above. Participants were given the governing charge for the workshop at the opening plenary, namely that by the end of the workshop, each group should have formed answers to three organizing questions for each of the three themes: 1) What are the primary issues?, 2) What is the research knowledge base? (What do we know?) and 3) What do we need to find out? (How can we learn more?) A chair and a co-chair were named for each group to keep the group on task and to capture the consensus of the discussions. More details concerning the agenda and charge are provided in the Appendix.

Outcome of Workshop: Research Agenda
The three sub-sections below are the reports from the co-chairs of the breakout groups that discussed the three themes outlined in the previous section. The three reports are included here in largely the same form as they were submitted, and hence the format and style of each is unique. To have “sanitized” each report into a consistent format would have not captured the unique flavor of what transpired in each group. It is interesting to note common threads running across the three reports below, such as whether or not current assessment practices are structured to measure, or even promote, far transfer, and how instruction might be structured to promote far transfer as opposed to doing well on mandated tests.


Context dependence of transfer There are two contexts to consider: The context of original learning and the context of the transfer problem. Context includes those things that are formally relevant to the core concepts being taught in the learning situation and which are supposed to be applied in the transfer situation. But context also includes many things that, while not formally relevant, may strongly influence learning as well as the likelihood that the learning will be recalled and used in transfer. These additional context factors include the learner’s prior knowledge, the learner’s expectations about the purpose of the original learning, the teacher’s expectations of the learner, the learner’s internal states during learning and transfer, group versus individual learning in relation to group versus individual application of knowledge, as well as the physical environments of the learning and transfer situations. Context can be important as a basis for cueing recall of the original learning or it can interfere with recall. • • A central question of transfer is: What remains invariant from the learning context to the transfer context? How can the learner best be taught this invariance? What are the cues and strategies that can be used to trigger appropriate knowledge to be applied in a particular situation? This question recognizes that the learner often has the relevant knowledge in a transfer situation but fails to make use of that knowledge. How important are the learner’s expectations during original learning and transfer? How important is it that the learner believe he/she will actually need the knowledge in “real life” as compared to just being able to pass an exam? Does the learner actually need to have the expectation that he/she is learning something? This is relevant to the use of informal learning situations such as educational television and computer games.


The research questions generated by the context group are: 1. How can education prepare people to recognize the cues that signal application of appropriate knowledge? What are the cues that produce or inhibit application of appropriate knowledge? 2. When is instruction for near transfer most appropriate as compared to instruction for far transfer? Are these dissociable? Does one type of learning hinder the other? In specialized training, such as for operating a nuclear power plant, should the learning situation always be an exact copy of the transfer situation? Should training emphasize near transfer at the expense of far transfer? If so, will the learner be able to think analytically if emergency situations arise that do not replicate the training context? 3. How does context affect deep understanding and preparation for learning as opposed to influencing whether relevant knowledge is retrieved?


4. When should tests match the learning context? 5. Will teaching for far transfer influence performance on traditional educational assessment? There were arguments for both enhancing and diminishing effects. 6. How does assessment of varying kinds influence teacher practice and student performance with respect to near and far transfer? 7. What do employers of various types want with respect to near and far transfer? To what extent do they want employees with narrowly defined skills as compared to broad abilities to solve problems and learn new skills? 8. What are the contextual factors (especially socio-cultural factors) that shape attitudes when entering new contexts that could involve transfer? 9. Is transfer hindered by compartmentalization of instruction? How integrated should instruction be? Conditions that Facilitate Transfer Cognitive research has converged on the conclusion that transfer is better if people have learned initially in a way that fosters deep, abstract understanding of central principles of a field, rather than emphasizing the rote application of rules triggered by surface similarities among problems. One method for achieving such deep, abstract understanding is to involve students in multiple examples illustrating a central principle drawn from an equivalence class where the surface characteristics of problems vary as widely as possible. It should be emphasized, however, that drawing this conclusion does not entail saying that automatization of skills and of facts is not relevant to transfer. On the contrary, research in areas such as reading and early mathematical development suggests that both conceptual learning and automatization (e.g. of decoding skills for reading, of number facts for early mathematics) are important. It is also clear that evaluating transfer requires clear delineation of the nature of mature competence as well as, preferably, knowledge of the sequence of states that typically leads to mature competence. However, we know relatively little about these issues in many domains. Thus, there is a research need to characterize such sequences and to determine if some errors or failures to transfer are potentially productive errors of growth or expectable restrictions on generalization, whereas other errors or failures to transfer indicate that the learner has encountered a significant barrier to continued growth. Representational requirements to facilitate deep understanding also need to be better characterized: both internal mental representations and external representations (e.g., maps, systems of notation, etc.). Learning for understanding may be more compatible with the implicit or explicit learning goals of some communities of learners than of others. There is a research need to consider whether there are barriers to such learning and teaching in the perceived needs of some learners to learn material of immediate practical relevance. If there is, there is obviously a further need to consider in detail how to work with learners with these assumptions.


Implications for Education and Assessment If we accept that cognitive research suggests the value of certain instructional strategies and goals, there are important implications for educational practice and for assessment that require further research. 1. Taking interventions based on these insights to scale may be quite difficult, as is true for many new educational programs. NSF is currently funding projects through IERI that involve the scaling problem, but there should also be consideration of funding projects to explicitly compare different techniques of scaling up. 2. One vital aspect of the scaling problem is what points of influence are available. Practicing teachers are often the target, but it can be hard to reach them. Pre-service teachers are one additional audience that can pay long-term dividends. School psychologists could be an entry point. In all these cases, the aim of researchers should go beyond discussion of a particular program or curriculum. It should be to foster learning of a deep, conceptual kind about the learning process. With such understanding, teachers and other professionals will be equipped to make good on-line decisions about such issues as, for example, how much students should struggle with new material, when research suggests there should be some struggle but not too much. 3. There is increasing concern about “teaching to the test” in an era when there is much demand for accountability. We need to devise tests that assess broad transfer so that accountability does not end up fostering teaching that will not support transfer. Science tests are in special need for attention. Technology can aid in the project of devising assessments that look at deep conceptual change. Assessments focused on preparedness for future learning (e.g., solving a relatively complex novel problem) may be more revealing of transfer than those focused strictly on analogous problems across isolated domains. 4. Many participants believe that “teaching to the test” may actually be self defeating, resulting in lower performance than teaching in a way that supports broad transfer. Research to evaluate the hunch that teaching for understanding may improve scores even on fairly traditional assessments should be supported. Metacognitive Issues for Transfer Primary Issues Research on transfer has emerged from two distinct traditions: associationist-learning and cognitive approaches (the latter including Gestalt psychology). Definitions, and empirical examples, of transfer reflect these intellectual roots. For example, stimulus generalization grew out of the associationist-learning tradition, whereas contemporary studies of analogical transfer reflect a cognitive approach. The classic transposition effects, studied extensively in rats and humans, were explained from both perspectives. Although metacognition is the most recent addition to the research landscape, it is a natural continuation of the cognitive tradition emphasizing “mentalism.”


Transfer can be defined as the cognitive benefits derived from prior learning or problem solving that accrue to problems that are not identical to studied exemplars. Issues that emerge from such a definition include distinguishing learning from transfer, near from far transfer, the peculiar role of negative as opposed to positive transfer, and interactions with consciousness, strategies for learning and problem solving, and retention of information across study and transfer episodes. The latter interactions between transfer per se and its active application by learners and problem solvers involve metacognition. Scientists attending the Transfer Workshop agreed that metacognition refers to any cognitive act about cognition, but they differed in their emphasis on the centrality of consciousness (explicit awareness of cognitions about cognition); verbalization (whether learners or problem solvers can articulate their metacognition); strategies (e.g., practical intelligence about effective tools and techniques); insight about quality (whether metacognition involves an evaluative or critical stance toward mental processes); or insight about self (a way of thinking that involves self-assessment, rather than merely skill-assessment). Issues of definition connect inexorably to those of measurement. For example, if verbalization and consciousness are not essential elements of metacognition, then measures that rely on verbal reports may underestimate this ability. Regardless of whether these elements must be strictly included in a definition, attendees concluded that these are important, researchable dimensions of metacognitive transfer. Scientists also identified several timely issues of national significance. Although these issues are only enumerated in this section, they are discussed in more detail in the sections that follow. Foremost among these is the issue of assessment, which has taken on increased significance because of recent national legislation. Another key issue that emerged was socio-cultural context, and contextual factors generally. These included identification of the kinds of learning required to adequately prepare the workforce in a fast-moving, knowledge-based economy (e.g., generalized problem solving, more narrow specific skills, or both). Participants cautioned against assuming that lower-paying jobs do not require substantial transfer and metacognitive awareness, but they acknowledged that some jobs appear to require near transfer and automaticity (e.g., piloting a plane, which might best be taught through highly realistic flight simulation). The group underlined the importance of focusing on skills that transfer from school to work, from home to school, and from one cultural context to another, without foreclosing educational and job opportunities for students by narrowing or lowering the quality of the curriculum. Finally, advancements in technology continue to make available many new options to enhance learning, transfer, and assessment. However, scientists stressed that most of these technological advances find their way into classrooms and work contexts without a solid foundation of research about their effectiveness for learning and transfer. Background of Prior Research Despite research in the learning tradition, which does not mandate understanding to explain near and far transfer, attendees agreed that research demonstrates that teaching for understanding produces more robust transfer. Thus, although transfer can occur without a pedagogical priority on understanding, rote learning is less effective than teaching for understanding in promoting transfer. In addition, instruction of the same underlying concept in multiple contexts facilitates transfer. (The precise relation among those contexts that optimally facilitates transfer is uncertain.) Research on direct instruction in critical thinking skills, ranging from deductive inference to statistical thinking (e.g., applying sample size in judging evidence quality), has produced reliable results. Much of this 12

instruction emphasizes metacognitive awareness. Little is known, however, about cross-domain transfer (e.g., critical thinking in English courses transferring to Biology), and some research on analogical reasoning suggests that such transfer is difficult. Assessment or testing has the potential to enhance metacognitive awareness, dispelling illusions of knowing and providing feedback about areas of weakness. Prior research on the psychometric properties of different tests must be brought to bear on the issue of their usefulness as a pedagogical tool. For example, tests that lack validity and reliability are unlikely to produce useful feedback to learners, teachers, or other decision makers. Moreover, tests differ in what they measure, some aiming at specific content knowledge and procedural skills, whereas others tap more general knowledge and skills. Distinctions in the research literature between rote versus conceptual understanding, concrete versus abstract knowledge, declarative versus semantic knowledge, verbatim versus gist representations, as well as task analyses of the cognitive processes involved in deep understanding of, say, physics or mathematics, may help to clarify the conditions under which “teaching to the test” could be beneficial. Given a valid and reliable test of some knowledge or skill worth assessing, the question arises as to whether teaching directly to the test or teaching for transfer produces better performance. If teaching for transfer raises metacognitive awareness, then the latter might mediate improved performance. Prior research on metacognitive awareness, such as judgments of the quality of one’s own learning, indicates that metacognitive judgments are generally abysmal in the absence of formal instruction. Indeed, naïve intuitions about the quality of one’s learning or the effectiveness of strategies for learning are often precisely the opposite of what has been shown to be true through empirical research. Closely allied with prior work on assessment, research on individual differences has identified stable personality traits and motivational factors that should affect learning, transfer, and metacognition. Promising connections between such factors (e.g., need for cognition) and transfer or metacognition have not been explored in any depth. Expertise is another dimension on which individuals differ, and this area of research can be plumbed for insights about the organization of knowledge and how development does or does not progress toward adaptive thinking, depending on one’s theoretical perspective. Notably, cognitive consistency (transfer across superficially different situations that tap the same cognitive competence) has been treated as an essential element of rationality in theories of judgment and decision-making. An emerging theme of the latter research is that of task variability, namely, that the same individual can exhibit bona fide competence in one context, and yet fail to apply that same competence in an equally relevant, but different, context. Finally, participants pointed out that memory has been a highly active area of research, but most of this work has occurred since transfer was intensively studied by learning or Gestalt theorists. In particular, research suggests that representation and retrieval are relevant to explaining and predicting transfer. The data suggest that learners who transfer have different mental representations of problem information, compared to those who do not. In addition, cues in the problem or environment govern retrieval of prior learning, and, hence, the likelihood of transfer. Many of the conundra of earlier theorists, it was argued, could be explained by recourse to these newer ideas.


Key Questions for Future Research Three main themes emerged from the discussion of transfer and metacognition. These themes are ones that build most efficiently on the foundation of prior research, and yet speak to timely issues of national significance. First, there is a need to explore habits of mind that produce better transfer across domains. Scientists’ best judgment at this time is that an attitude of active inquiry, insight that seemingly disparate contexts can involve similar kinds of thinking, and an evaluative stance towards one’s own cognition enhance transfer. However, they raised the possibility that better metacognition might be unrelated to better cognition. The importance of situated metacognition (and, conversely, contextual transfer) was another pervasive theme. Research should be directed toward blending laboratory-based and real-world observations (i.e., twinning or close coordination of carefully constructed studies across domains). Finally, scientists agreed that there was an overarching need for theory-driven research. Many of the open questions and practical problems that had been discussed could be addressed through a greater emphasis on processes and mechanisms underlying transfer or metacognition. Accurate assessment, for example, hinges on a sound analysis of such processes and mechanisms. Three areas seem especially promising for scientifically rigorous theory development: 1. Representation and retrieval 2. Domain specificity versus generality 3. Adaptive metacognition These choices reflect the groups’ judgment of enabling prior research (e.g., in memory), important questions (e.g., transfer across home, school, and work environments), and key conceptual challenges (e.g., determining whether metacognition is adaptive). Additional Recommendations The participants noted that inter-disciplinary collaboration is beneficial to research about scientific and mathematical learning, and that research conferences can facilitate that kind of collaboration. However, they pointed out that there was a need to encourage better communication about research within disciplines, and that this has received virtually no attention or support. This need is heightened by the growing expansion of research knowledge. Areas of cross-fertilization that would be germane for the study of transfer and metacognition include: • • Developmental and adult cognitive scientists Applied and basic scientists working on similar topics

It would be essential to continue to include content specialists (scientists and mathematicians) in such conferences. Although some content specialists might be naïve about cognitive components of problem solving (and would benefit from concepts, measures, and data generated by cognitive psychologists), cognitive psychologists who lack sufficient content knowledge are liable to construct naive theories about scientific or mathematical understanding. 14

Concluding Remarks
The research agenda that resulted from this workshop is clearly not “the” research agenda for transfer of knowledge but rather “a” research agenda. The research agenda presented here, however, reflects the views and consensus of the workshop attendees. By design, this was an invitational workshop and the group invited to attend was small, albeit comprised of individuals with rather impressive credentials. It was never the intention of the organizers to assemble a “complete” group representing all relevant fields and views related to transfer. Thus, the agenda presented may contain holes that will need to be filled out at some future date.

Acknowledgement & Dedication
This report is dedicated to the memory of Rodney R. Cocking, who suffered a tragic death a month prior to the workshop. Rodney was instrumental in planning the workshop, and he had great ambitions for it. As Program Director for the Developmental and Learning Sciences Program at NSF, Rodney was devoted to strengthening the connections between basic research on cognition and educational practice. His tireless efforts made the workshop possible, and his influence was noticeable throughout. Rodney was a dear friend and colleague of most who participated. It is our hope that we have come close to delivering on his vision.


Auble, P.M., & Franks, J.J. (1978). The effects of effort toward comprehension on recall. Memory and Cognition, 6, 20-25. Baddeley, A.D., & Longman, D.J.A. (1978). The influence of length and frequency of training session on the rate of learning to type. Ergonomics, 21, 627-635. Barnett, S.M., & Ceci, S.J. (2002). When and where do we apply what we learn?: A taxonomy for far transfer. Psychological-Bulletin, 128(4), 612-637. Bartlett, F.C. (1932). Remembering: A study in experimental psychology. Cambridge, UK: Cambridge University Press. Bassok, M. (1990). Transfer of domain-specific problem-solving procedures. Journal of Experimental Psychology: Learning, Memory and Cognition, 16, 522-533. Bassok, M., & Holyoak, K.J. (1989). Interdomain transfer between isomorphic topics in algebra and physics. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 153-166. Bjork, R.A., & Richardson-Klavhen, A. (1989). On the puzzling relationship between environment context and human memory. In C. Izawa (Ed.), Current Issues in Cognitive Processes: The Tulane Flowerree Symposium on Cognition (pp. 313-344). Hillsdale, NJ: Erlbaum. Blanchette, I., & Dunbar, K. (2000). How analogies are generated: The roles of structural and superficial similarity. Memory & Cognition, 28, 108-124. Blanchette, I. & Dunbar, K. (2002). Representational change and analogy: How analogical inferences alter target representations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 672-685. Bransford, J.D., Brown, A.L., & Cocking, R.R. (1999). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press. Bransford, J.D. & Schwartz, D. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.), Review of Research in Education (Vol. 24 pp. 61-100). Washington, DC: American Educational Research Association. Bransford, J.D., Stein, B.S., Vye, N.J., Franks, J.J., Auble, P.M., Mezynski, K.J., & Perfetto, G.A. (1983). Differences in approaches to learning: An overview. Journal of Experimental Psychology: General, 3, 390-398. Broudy, H.S. (1977). Types of knowledge and purposes of education. In R.C. Anderson, R.J. Spiro, & W.E. Montague (Eds.), Schooling and the Acquisition of Knowledge (pp. 1-17). Hillsdale, NJ: Erlbaum. Brown, A. L. (1990). Domain-specific principles affect learning and transfer in children. Cognitive Science, 14, 107-133. Brown, A. L., Bransford, J. D. , Ferrara, R.A., & Campione, J. (1983). Learning, remembering, and understanding. Cognitive Development. Vol.3 (pp. 78-166). J. Flavell and E. M. Markman. New York, John Wiley and Sons. Brown , A. L. & Campione, J. (1998). Designing a community of young learners: Theoretical and practical lessons. In Lambert, Nadine M. and McCombs, Barbara L. (Eds). (1998). How students learn: Reforming schools through learner-centered education. (pp. 153-186). Washington, DC: American Psychological Association. Brown, A.L. & Kane, M.J. (1988). Preschool children can learn to transfer: Learning to learn and learning from examples. Cognitive Psychology, 20, 493-523. Carey, S. & Smith, C. (1993). On understanding the nature of scientific knowledge. Educational Psychologist, 28 (3), 235-251.


Carey, S. & Spelke, E. (1994). Domain-specific knowledge and conceptual change. In L. A. Hirschfeld & S. A. Gelman (Eds.), Mapping the mind: Domain specificity in cognition and culture (pp. 169-200). New York: Cambridge University Press. Carraher, T.N. (1986). From drawings to buildings: Mathematical scales at work. International Journal of Behavioral Development, 9, 527-544. Ceci, S.J., & Williams, W.M. (1997). Schooling, intelligence and income. American Psychologist, 52(10), 1051-1058. Chi, M. T. H. (2000). Self-explaining: The dual processes of generating inference and repairing mental models. In Glaser, R. (Ed). (2000). Advances in instructional psychology: Educational design and cognitive science, Vol. 5. (pp. 161-238). Mahwah, NJ: Lawrence Erlbaum Associates. Dunbar, K. (2001). The analogical paradox: Why analogy is so easy in naturalistic settings, yet so difficult in the psychology laboratory. In D. Gentner, K. Holyoak, & B. Kokinov (Eds.). Analogy: Perspectives from cognitive science (pp. 313-334). Cambridge, MA: MIT press. Eich, E. (1985). Context, memory, and integrated item/context imagery. Journal of Experimental Psychology: Learning, Memory, & Cognition, 11, 764-770. Fullan, M. (2001). Leading in a Culture of Change. San Francisco, CA: Jossey-Bass. Gelman, R. & Lucariello, J. (2002). Learning in cognitive development. In Pashler, H. & Gallistel, C.R. (Eds.), Stevens’ Handbook of Experimental Psychology, Third Edition, Vol.3 (pp. 396443).New York: Wiley. Gelman, R. & Williams, E. (1998). Enabling constraints for cognitive development and learning: Domain specificity and epigenesis. In D. Kuhn and R. Siegler, (Eds.), Cognition, perception and language. Vol. 2. Handbook of Child Psychology (Fifth Ed) (pp. 575-630). New York: Wiley. Gick, M.L., & Holyoak, K.J. (1980). Analogical problem solving. Cognitive Psychology, 12, 306355. Glaser, R. (1994). Learning theory and instruction. In G. d’Ydewalle, P. Eelen, & P. Bertelson (Eds.), International Perspectives on Psychological Science, Vol. 2: The State of the Art (pp.341-357). Hove, UK: Lawrence Erlbaum Associates. Halpern, D. F.(1998). Teaching critical thinking for transfer across domains. American Psychologist, 53(4), 449-455. Hartnett, P. & Gelman, R. (1998). Early understandings of numbers: Paths or barriers to the construction of new understandings? Learning and Instruction: The Journal of the European Association for Research in Learning and Instruction, 8(4), 341-374. Hayes, J.R., & Simon, H.A. (1977). Psychological differences among problem isomorphs. In N.J. Castellan, Jr., D.B. Pisoni, & G.R. Potts (Eds.), Cognitive Theory (Vol. 2, pp. 21-41). Hillsdale, NJ: Lawrence Erlbaum Associates. Kerr, R., & Booth, B. (1978). Specific and varied practice of a motor skill. Perceptual and Motor Skills, 46, 395-401. Klahr, D., & Carver, S.M. (1988). Cognitive objectives in a LOGO debugging curriculum: Instruction, learning, and transfer. Cognitive Psychology, 20, 362-404. Lave, J. (1988). Cognition in Practice: Mind, Mathematics and Culture in Everyday Life. Cambridge, UK: Cambridge University Press. Littlefield, J., Delclos, V., Lever, S., Clayton, K., Bransford, J., & Franks, J. (1988). Learning LOGO: Method of teaching, transfer of general skills, and attitudes toward school and computers. In R.E. Mayer (Ed.), Teaching and Learning Computer Programming (pp. 111136). Hillsdale, NJ: Lawrence Erlbaum Associates.


Mandler, J. M. & Orlich, F. (1993). Analogical transfer: The roles of schema abstraction and awareness. Bulletin of the Psychonomic Society, 31(5), 485-487. Mannes, S.M., & Kintsch, W. (1987). Knowledge organization and text organization. Cognition & Instruction, 4, 91-115. McNamara, D.S., Kintsch, E., Songer, N.B., & Kintsch, W. (1996). Are good texts always better? Text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1-43. Mestre, J.P. (2002). Probing adults’ conceptual understanding and transfer of learning via problem posing. Journal of Applied Developmental Psychology, 23, 9-50. Nitsch, K.E. (1977). Structuring decontextualized forms of knowledge. Ph.D. dissertation, Vanderbilt University. Described in Bransford, J.D. (1979). Human Cognition: Learning, Understanding & Remembering. Belmont, CA: Wadsworth Publishing Company. Reed, S.K., Dempster, A., & Ettinger, M. (1985). Usefulness of analogous solutions for solving algebra word problems. Journal of Experimental Psychology: Learning, Memory, & Cognition, 11, 106-125. Reed, S.K., Ernst, G.W., & Banerji, R. (1974). The role of analogy in transfer between similar problem states. Cognitive Psychology, 6, 436-450. Saxe, G. B. (1989). Transfer of learning across cultural practices. Cognition and Instruction, 6(4), 325-330. Schwartz, D.L., & Bransford, J.D. (1998). A Time for Telling. Cognition and Instruction, 16, 475522. Schwartz, D.L., & Moore, J.L. (1998). The role of mathematics in explaining the material world: Mental models for proportional reasoning. Cognitive Science, 22, 471-516. Shea, J.B., & Morgan, R.L. (1979). Contextual inference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5, 179-187. Silva, A. B., & Kellman, P. J. (1999) Perceptual learning in mathematics: The algebra-geometry connection. In Han, M., & Stoness, S.C. (Eds.). Proceedings of the Twenty-First Annual Conference of the Cognitive Science Society. (pp. 683-688). Mahway, NJ: Lawrence Erlbaum Associates. Simon, D.A., & Bjork, R.A. (2001). Metacognition in motor learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 907-912. Singley, K., & Anderson, J.R. (1989). The Transfer of Cognitive Skill. Cambridge, MA: Harvard University Press. Slamecka, N.J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning & Memory, 4, 592-604. Stigler, J. W. & Fernandez, C. (1995). Learning mathematics from classroom instruction: Crosscultural and experimental perspectives. In C.A, Nelson (Ed.), Basic and applied perspectives on learning, cognition, and development (pp. 103-130). Mahwah, NJ: Lawrence Erlbaum Associates. Wittrock, M.C. (1990). Generative processes of comprehension. Educational Psychologist, 24, 345376.


Appendix: Workshop Agenda and Participant list Memorandum
To: Participants in the Transfer Workshop From: Jose Mestre and Rod Cocking Subject: Workshop organization, agenda, and schedule Date: February 25, 2002 The workshop is organized around three overarching themes outlined below. We have grouped participants into one of the three groups, with a designated chair and co-chair/scribe for each group. The roles of the chair and co-chair are to keep the group on task, to summarize groups discussions, to present a summary of the group’s views in the plenary session, and to submit a summary by the end of the workshop to the workshop organizers that addresses the three questions in the governing charge (described below). Each group should begin by making a list of the subtopics that fall under their broad topic. That list can then be edited down to those subtopics that embody core principles that we need to know more about and those for which we can summarize a knowledge base. Group discussions should then proceed to refine the critical areas that need to be refined. We have taken a first stab at listing some subtopics that each group may want to consider as a staring point.

WORKSHOP THEMES: • Context dependence of transfer (Dan Anderson: Chair; Janice Earle: Co-chair/scribe;
Miriam Bassok, Robert Bjork, Milt Hakel, Maria Ruiz-Primo, Gerhard Salinger, Dan Schwartz, Guy Van Orden, Brian Vaughn, Dean Zollman). The acquisition of knowledge or a skill does not ensure that the knowledge or skill will be appropriately (or inappropriately) applied to relevant situations. What do we need to know about the contexts of learning or application of learning in order to facilitate transfer? • Domain dependence (e.g., physics vs biology [principles vs taxonomy]; using what we learn in math class in physics or chemistry class) • What contexts facilitate transfer? which impede it? • Are context issues different as a function of cognitive maturity or differentiation within a domain? • Group work vs. individual in terms of: creativity; divergent thinking; distributed expertise • Education vs. training (i.e., formal education vs. trade learning) • Formal versus informal learning (including museums, television, media, games).


• Conditions for transfer (Nora Newcombe: Chair; Ken Whang: Co-chair/scribe; John
Bransford, Steve Breckler, Suzanne Donovan, Joyce Evans, Rochel Gelman, Drew Gitomer, Joe Redish). What are the conditions that affect the appropriate (or inappropriate) application of knowledge or skill. What do we need to know about the contexts of learning, or application of learning, in order to facilitate transfer? • Factors that facilitate, and that hinder transfer, including social and group processes • How can education in K-12 or 13-18 be structured to facilitate transfer, and to promote the integration of knowledge • Assessment and transfer: How do we assess transfer (especially far transfer)? Can assessment be used as a vehicle to promote transfer (just as tests can be used to drive the teaching and learning)? • What conditions facilitate transfer from a teaching perspective and from a learning perspective

• Meta level issues for transfer (Valerie Reyna: Chair; Elizabeth VanderPutten: co-chair/
scribe; Susan Barnett, Ian Beatty, Kevin Dunbar, Bill Gerace, Diane Halpern, David Hammer, Keith Holyoak, Michael Martinez). Metacognitive research indicates that a significant part of effective learning is to be aware of, and in control of, one’s own learning. What research base is needed to promote reflective learning? What are the implications for classroom practice? • Learning to learn and transfer • Learners’ self awareness of learning and transfer • The role of epistemology in transfer • Self-assessment as an avenue for transfer • Facilitating reflection on one’s own learning (e.g. via concept mapping, hierarchical organization of knowledge, …)

By the end of the workshop, we should have formed answers to the following three organizing questions for each of the three themes: 1) What are the primary issues? 2) What is the research knowledge base (What do we know?)? 3) What do we need to find out (How can we learn more?).


Schedule Transfer Workshop
Thursday 3:00 PM-6:00PM 3:00 Welcome and introductions. Mestre, Ramalay, Bradburn & Breckler. 3:30-4:15 Launching demonstration highlighting some important transfer issues in learning and teaching in the natural sciences. Mestre 4:15-5:00 Goals and structure of the workshop: Issues and agenda. Plenary discussion relating to workshop structure, issues, and agenda. Mestre 5:00-6:00 Organizational meeting of the three break-out theme groups. Goal: Reach consensus for plans to reach workshop goals. 6:00 Dinner Friday 8:15 PM-5:00PM 8:15-8:30 Coffee 8:30-10:30: Breakout groups meet (10 minute break at 10:15) 10:45-11:45 Breakout groups meet 11:45-12:00 Breakout groups prepare plenary presentation on the three organizing questions. [Each group presents summary of group consensus in a couple of tranparencies.] 12:00-1:00 Lunch (on your own) 1:00-2:30 Plenary reports. 20 minutes of subgroup presentation, 10 minutes of Q&A 1:00-1:30 Group consensus report: Context dependence of transfer 1:30-2:00 Group consensus report: Conditions for transfer 2:00-2:30 Group consensus report: Metacognition 2:30-2:45 Break 2:45-3:30 Synthesis—relationship between expertise and transfer: Bransford 3:30-4:00 Research innovations and opportunities: Breckler & Mestre 5:00 Adjourn


Participants in the Transfer of Knowledge Workshop NSF, March 21-22, 2002

Daniel Anderson Dept. of Psychology Tobin Hall University of Massachusetts Amherst, MA 01003 (413)545-2069 Susan Barnett Dept. of Human Dev MVR Hall Cornell University Ithaca, NY 14851 Miriam Bassok Department of Psychology University of Washington Box 351525 Seattle, WA 98195 (206) 543-6940 Ian Beatty Dept. of Physics, LGRT 416 University of Massachusetts Amherst, MA 01003 413-545-9483 Robert A. Bjork Dept. of Psychology UCLA Franz Hall 405 Hilgard Avenue Los Angeles, CA 90095-1563 310-825-7028

John Bransford Box 45 Learning Technology Center Peabody College of Education & Human Development Vanderbilt University Nashville, TN 37203 615-343-0432 bransfjd@ctrvax.Vanderbilt.Edu Suzanne Donovan Phone. 202-334-2080. National Academies 2101 Constitution Ave. HA-178 Washington, DC 20418 Kevin Dunbar Psychology & Education Department of Education Silsby Hall Dartmouth College Hanover NH 03755 Rochel Gelman Psychology & Cognitive Science Rutgers University 152 Frelinghuysen Rd. Piscataway, NJ 08854-8020 (732) 445-6154 William Gerace Dept. of Physics LGRT 416 University of Massachusetts Amherst, MA 01003 413-545-3774


Drew Gitomer Statistics and Research Division Educational Testing Service Princeton, NJ 08541 (609)734-1239 Milton Hakel Dept. of Psychology Bowling Green University Bowling Green, OH 43403 (419) 372-8144 Diane Halpern Director Berger Institute for Work, Family and Children Claremont McKenna College 850 Columbia Ave. Claremont, CA 91711-6400 (909)607-8023 David Hammer Dept. of Physics & Astronomy University of Maryland College Park, MD 20742 (301) 405-8188 Keith Holyoak Dept. of Psychology UCLA Franz Hall 405 Hilgard Avenue Los Angeles, CA 90095-1563 310-826-1646 Jose Mestre Dept. of Physics LGRT 312 University of Massachusetts Amherst, MA 01003 413-545-2040

Nora Newcombe Dept. of Psychology Temple University 1701 N. 13th St., Rm 565 Piladelphia, PA 19122-6085 215-204-6944 Edward (Joe) Redish Dept. of Physics & Astronomy University of Maryland College Park, MD 20742 (301) 405-6120 Valerie Reyna Senior Research Advisor, OERI/U.S. Department of Education 555 New Jersey Avenue, NW Washington, DC 20208 Telephone: 202-219-2138 Dan Schwartz Stanford University School of Education 485 Lasuen Mall Stanford, CA 94305-3096 (650) 723-2109 Brian Vaughn Auburn University Human Development and Family Studies 203 Spidle Hall Auburn, AL 36849 Phone: 334-844-3235 Dean Zollman Department of Physics 116 Cardwell Hall Kansas State University Manhattan. KS 66506-2601 785-532-1619


NSF Participants Norman Bradburn Philip Rubin Judith Ramalay Eric Hamilton Elizabeth Vanderputten Ken Whang Gerhard Salinger Steve Breckler Maria Ruiz-Primo Janice Earle Joyce Evans Larry Suter Mike Martinez Guy Van Orden


The National Science Foundation (NSF) funds research and education in most fields of science and engineering. Awardees are wholly responsible for conducting their project activities and preparing the results for publication. Thus, the Foundation does not assume responsibility for such findings or their interpretation. NSF welcomes proposals from all qualified scientists, engineers and educators. The Foundation strongly encourages women, minorities and persons with disabilities to compete fully in its programs. In accordance with Federal statutes, regulations and NSF policies, no person on grounds of race, color, age, sex, national origin or disability shall be excluded from participation in, be denied the benefits of, or be subjected to discrimination under any program or activity receiving financial assistance from NSF (unless otherwise specified in the eligibility requirements for a particular program). Facilitation Awards for Scientists and Engineers with Disabilities (FASED) provide funding for special assistance or equipment to enable persons with disabilities (investigators and other staff, including student research assistants) to work on NSF-supported projects. The National Science Foundation has Telephonic Device for the Deaf (TDD) and Federal Information Relay Service (FIRS) capabilities that enable individuals with hearing impairments to communicate with the Foundation about NSF programs, employment or general information. TDD may be accessed at (703) 292-5090, FIRS at 1-800-877-8339. The National Science Foundation is committed to making all of the information we publish easy to understand. If you have a suggestion about how to improve the clarity of this document or other NSF-published materials, please contact us at


Presorted Standard POSTAGE & FEES PAID National Science Foundation Permit No. G-69

Description: National Science Foundation