Document Sample
scaffolding_study Powered By Docstoc
					                                        ePortfolio Scaffolding Study
                                                   Sandra A. Lathem
                                                     Joyce L. Morris
                                                         Jing Qi
                                                  University of Vermont

                                                    Kirk Vandersall
                                                Arroyo Research Services

             Abstract: The e-Portfolio Scaffolding Study is an exploratory study that examines the process by
             which pre-service teachers in the elementary, middle, and physical education programs at the
             University of Vermont are taught to construct their licensure portfolios (in paper or electronic
             formats) and attempts to identify relationships between instructional scaffolding (in both portfolio
             development and technological proficiency) and the quality of the artifacts and reflections produced
             by pre-service teachers in these three programs.

         The e-Portfolio Scaffolding Study is one of several research projects being conducted by the Electronic
Portfolio Connection, a three-year grant awarded in October 2003 to the University of Vermont (UVM) by the U.S.
Department of Education Preparing Tomorrow‟s Teacher to Use Technology (PT3) initiative. Recognizing a lack of
substantive research about the effect of electronic portfolios on teaching practice (Carney, 2006; Ma & Rada, 2006;
Wetzel & Strudler, 2005), this research seeks to provide additional insight about the implementation of electronic
portfolios in pre-service teacher education programs. The purpose of the e-Portfolio Scaffolding Study is to examine
the process by which pre-service teachers in the elementary, middle, and physical education programs at the
University of Vermont are taught to construct their licensure portfolios (in paper or electronic formats) and identify
relationships between instructional scaffolding (in both portfolio development and technological proficiency) and the
quality of the artifacts and reflections produced by pre-service teachers in these three program areas.

Licensure Portfolio Development at UVM
         A licensure portfolio, along with course grades, disposition evaluations and PRAXIS results, is one of the
required assessments that determine whether or not a pre-service teacher is competent to receive a Level I teaching
license. It represents a gateway to the profession and as such is a high-stakes assessment. In investigating licensure
portfolio development at The University of Vermont, a number of questions arise:
         What is the evidence that instructional scaffolding in portfolio development and technology skills
          occurs at the micro (faculty member/student) and macro (department/program) levels?
         What is the relationship between the instructional scaffolding that pre-service teachers receive
          and the quality of artifacts and reflections found in the licensure portfolio?
         What are UVM student attitudes about the licensure portfolio process in their programs?
         Will the quality in artifact development and reflection differ between paper and electronic
We are curious to understand the intentional strategies that faculty members use to teach artifact construction,
reflection, and technology skills and if these strategies are producing desired results. Similarly, we are interested to
learn whether and how pre-service teachers are prepared to construct the licensure portfolio in an electronic format.

          Teacher licensure in Vermont is regulated by Rule Series 5100 of the Vermont State Board of Education
Manual of Rules and Practices and administered by the Vermont Department of Education Office of Licensing and
Professional Standards. The Vermont Standards Board for Professional Educators (VSBPE), a twenty-three member
board composed primarily of teachers, is appointed by the State Board of Education and charged with oversight of the
regulatory policies, procedures, and practices concerning teacher licensure. VSBPE oversees the accreditation process
for all colleges and universities offering teacher education programs through a process called ROPA (Results Oriented
Program Approval). VSBPE requires all teachers to prepare a portfolio to support their application for licensure that
provides evidence that the candidate has successfully met Five Standards and 16 principles articulated in another
VSBPE document called Five Standards for Teachers: A Vision for Schooling. The five standards address the
following areas: 1) Learning (Expertise in Endorsement Area), 2) Professional Knowledge (Methodology and
Pedagogy), 3) Colleagueship, 4) Advocacy, and 5) Accountability. ("Five Standards for Vermont Educators: A
Vision for Schooling," 2003) For assessment purposes, VSBPE has organized these five standards and 16 principles
into six required entries by which the licensure portfolio is scored. They are: Entry 1 – Teaching Episodes (two
episodes are required); Entry 2 – Understanding Student Learning; Entry 3 – Accommodating Students Identified;
Entry 4 – Teaching Over Time; Entry 5 – Colleagueship and Advocacy; and Entry 6 – Self-Reflection and Vision.
          Approved teacher education programs are charged with the responsibility to assess Level I licensure
portfolios according to the six entries but have the freedom to make internal and local decisions about format, criteria,
and development of licensure portfolios. VSBPE supplies a rubric that all postsecondary institutions must use to score
the six required portfolio entries, thereby implementing a common, statewide assessment instrument in all teacher
education programs. The methods and procedures to construct and develop portfolios, however, are determined by
each institution and then by program areas within each institution, allowing significant variability in the portfolio
process to occur. At the University of Vermont, requirements in addition to State criteria vary by program areas.
          The search for software tools to manage the portfolio process has evolved as faculty and student interest in
electronic portfolios has grown. When the Electronic Portfolio Connection grant began in October 2003, the grant
team implemented a new e-portfolio product under development by IBM called Teachers’ Workplace (TWP) to assist
faculty members in their evaluation of web-based portals designed for portfolio construction and management.
Unfortunately the development of Teachers’ Workplace did not meet teacher education program needs for a flexible
and reliable system. As a result, the PT3 grant team created several licensure portfolio templates that students (using
generic HTML editors such as Mozilla Composer) could modify and adapt. Whereas these pre-constructed templates
assisted students by providing a framework of the e-portfolio‟s organization, students nevertheless needed to possess
technology skills in web publishing, digital video, image manipulation and editing, and file management to be able to
complete their electronic licensure portfolio. Generic HTML portfolios seemed a solution that provided dual benefits
to the programs: 1) the portability and flexibility that portfolios disseminated on CD-ROMs offered, and 2) a
summative performance task that enabled the pre-service teacher to demonstrate technological proficiency.

Scaffolding and Contingency
         Scaffolding has been used successfully as an instructional technique for many years. In this approach, the
teacher initially models the task or strategy and then gradually turns responsibility over to the student. This practice is
based on the work of Vygotsky and then Bruner who postulated that with an adult's assistance, children could perform
tasks too complex for them to perform independently. Hammond and Gibbons (2001) note that “A major feature of
the term „scaffolding‟ is its ability to capture the role of the „expert,‟ or more knowledgeable other (typically the
teacher), in assisting students‟ learning, and the role of that knowledgeable other in extending students‟ current levels
of understanding or current capabilities.” A scaffolding experience allows a student to move away from assistance, to
become an independent learner able to transfer the acquired understanding to new contexts. Given this premise, a
teacher must recognize when to withdraw support and to move intentionally away, allowing the student to apply and
transfer knowledge in new ways. This ability to recognize when to provide support and when to withdraw, labeled
contingency, can be an indicator of good pedagogy and teaching. (Hammond & Gibbons, 2001) Scaffolding is also
seen in instructional design as the intentional planning of curricula to organize tasks and activities in such a way as to
build understanding. Thus, scaffolding occurs at both the micro (the individual teacher/student interaction) and the
macro (the program or organization) level, and both elements are presumably active in learning environments where
good pedagogy takes place.
          Contemporary learning theory research echoes much of the work of Vygotsky, Bruner, and others, and by
association, the concepts of scaffolding. Findings from the National Research Council‟s How People Learn add
credence to the notion of contingency by acknowledging that teachers must first come to understand one‟s prior
knowledge in order to uncover “false beliefs, and the naïve renditions of concepts that learners bring with them to a
given subject.” (Bransford, Brown, & Cocking, 2000) This research recommends pedagogical practices that begin
with student prior knowledge, that uncover and reveal misconceptions, that build a learner‟s metacognition and
reflection skills, and that employ formative assessment practices to make student thinking visible. These
contemporary notions about learning support constructivist learning and the social construction of knowledge and
recognize scaffolding and formative assessments as strategies teachers “use to move learning forward in the zone of
proximal development.” (Shepard, 2005) As Shepard (2005) states” “Formative assessment uses insights about a
learner‟s current understanding to alter the course of instruction and thus support the development of greater
competence. From a sociocultural perspective, formative assessment – like scaffolding – is a collaborative process and
involves negotiation of meaning between teacher and learner about expectations and how best to improve
          Scaffolding is therefore important in many ways. Strong instructional programs recognize the need to design
curricula that purposefully builds knowledge, introducing skills and concepts in logical and meaningful increments
that provide students a firm, factual foundation in content. This macro level design constructs the path for deep
learning to emerge. At the same time, teachers in the program must be adept at scaffolding at the micro level by
providing instruction and assistance that challenges and motivates individual students to construct and transfer new
knowledge and understanding.

     The ePortfolio Scaffolding Study seeks to identify correlations between classroom instructional practices and the
resulting artifacts that comprise a licensure portfolio. The research methodology is exploratory in nature and uses a
mixed methods approach that includes gathering both qualitative and quantitative data through a variety of
instruments, as follows: faculty assignments/syllabi; structured observations of artifact introduction and/or classroom
support; semi-structured faculty interviews; faculty and student surveys about the artifact development and reflective
process; student focus group sessions; formal, rubric review of the quality and depth of artifacts of final licensure

Data Sampling
          Student Surveys. Student surveys were distributed and collected during class time by members of the PT3
team. One hundred (n=100) students completed the survey. The survey was given to first year students (23),
sophomores (4), juniors (28), seniors (37), and post baccalaureate students (8) in selected courses.
          Faculty Interviews. Face-to-face interviews with faculty members in the elementary, middle level, and
physical education programs were also conducted during the Spring 2006 semester. A total of ten (10) faculty
members were interviewed, as follows: four (4) faculty members in elementary education; five (5) faculty members
in middle level; and one (1) from the physical education program.
          Faculty and Student Observations. Three classroom observations of at least one hour were conducted in
each of the three program areas.
          Student Focus Groups. Three, hour-long student focus group sessions were held with elementary (13
volunteers from a senior portfolio seminar of 38 students), middle level (15 students held during a class session), and
physical education students (5 students held during a class session).
          Course Syllabi. Thirty-four (34) course syllabi were analyzed and scored according to a pre-designed
scoring rubric.
          Licensure Portfolios. Licensure portfolios were collected at the close of the Spring 2006 semester from the
elementary and middle level programs. Elementary students were given the option to construct their portfolios in
either a paper or electronic format. Middle level students were required to develop electronic portfolios using a pre-
designed template. Thirty-five (35) portfolios were collected and scored using a pre-designed scoring rubric.
Preliminary Findings

         Analysis of the data began during Summer 2006 and preliminary findings are described below:

What is the evidence that instructional scaffolding in portfolio development and technology skills occurs at the micro
(faculty member/student)and macro (department/program) levels?
          To determine the extent to which instructional scaffolding occurs at the micro and macro levels, an analysis
of UVM College of Education and Social Services syllabi was conducted on 34 syllabi drawn from the elementary,
middle, secondary, and physical education programs. The syllabi dated from Fall 2004 to Spring 2006 and were
categorized into a matrix and analyzed by Arroyo Research Associates regarding their relevance to Vermont‟s
licensure portfolio requirement. This analysis found that The University of Vermont‟s education and teacher
preparation classes offered students many portfolio related experiences and artifacts as part of their coursework. More
often than not, these were not technology related, but in some cases, courses‟ technology components were quite rich.
Many of the courses in all the different program areas offered a wide variety of portfolio-relevant experiences for
students. Some courses were entirely dedicated to student-teaching experiences that track very closely to the
requirements of Vermont‟s licensure portfolio. Student teaching courses or parts of courses included reviewing
lessons or mini-lessons with peers, mentor teachers, and instructors; discussion of teaching online and student teacher
reflections in journals. Students were also encouraged to interact with the school staff during their field experiences
including working with parents. Their field experiences were not limited to student teaching in the classroom although
this was often the focus of the fieldwork.
          Although in the majority of syllabi, the licensure portfolio was not explicitly mentioned, in almost all cases,
the syllabi described student coursework on artifacts that could reasonably be expected to become part of a student‟s
licensure portfolio. Many of the course syllabi examined involved students creating a variety of lesson plans for
students, from entire instructional units to 10-15 minute mini-lessons. These artifacts fit very closely with the
requirements of the licensure portfolio that require teachers to demonstrate knowledge of student learning and the
ability to design effective lessons. The courses also required many other artifacts useful to portfolios -- many of them
graded assignments that weighed heavily on the courses‟ final grades. Some syllabi required teachers to create
reflective journals on their teaching experiences which could be adapted to their licensure portfolios. Again, in most
cases this was not done with technology, but nevertheless was very relevant to the requirements of the portfolio.
          In the majority of courses class discussion was encouraged in the syllabi and was counted as part of the final
grade. There were also many classes where student teaching was video taped and analyzed and discussed among
peers. Overall, the UVM education course syllabi encourage class discussion, teamwork, peer review, and give
students opportunities to revise their work with feedback from instructors. Most of the portfolio-relevant experiences
and artifacts were introduced into the courses early on.
          In most cases, there was no evidence that the portfolio-related artifacts and experiences were required to be
completed using technology tools. The most common technology tools used in the courses overall were WebCT for
online discussions and journaling. WebQuests were also fairly common. In other cases, students were required to keep
a journal online. For courses where student teaching lessons were recorded, iMovie was often used for review by
peers and instructors. In only a few courses were other technology tools integrated into the course. Typically, these
were courses where using technology in education was the main course theme. Overall, there was not a great emphasis
on technology in the syllabi compared to a focus on the portfolios in general.
          At a macro level, the elementary and middle level programs have intentionally earmarked portfolio artifact
development in specific courses but differ significantly in designing a developmental approach that spans the program
from beginning to end. The physical education program has the least developed approach and depends primarily on
one instructor‟s oversight. The middle level program has established a comprehensive developmental strategy that
aligns each education course with at least one of the six required portfolio entries and requires three portfolios during
the program: Application for Professional Block, Application for Student Teaching, and the Senior Licensure
          The ROPA portfolio scoring rubric influences many of the learning activities that middle level students
experience. This connection between assignments and ROPA entries was mentioned frequently in conducted
interviews. The following remarks from one of the middle level faculty is representative of other faculty interviews:

         … that [developmental portfolio system] has been a nice thing because in return it has provided
         some structure in the course that I had but I wasn‟t as necessarily intentional about. So for example,
         one of the things that we expect them to do in the portfolio is demonstrate colleagueship and
         advocacy. One of the things they do in my course is have all the students simulate an
         interdisciplinary team for the entire semester. They construct web pages as if they were an actual
         middle school team and on the web pages, they communicate their approach to groups and
         scheduling, family involvement, and all the things that I just named. So, it‟s a way to demonstrate
         their mastery of those concepts without a traditional written piece. … So that‟s a big piece of what
         we do in that course and it lends itself very nicely to the colleague artifact that we expect to be in the

What is the relationship between the instructional scaffolding that pre-service teachers receive and the quality of
artifacts and reflections found in the licensure portfolio?
          Thirty-five (35) final licensure portfolios were collected at the end of Spring 2006 semester and were
evaluated using a pre-designed scoring rubric developed by Arroyo Research Services, the grant evaluator. Of these,
twelve (12) paper portfolios and fifteen (15) electronic portfolios were collected from the elementary education
program. Elementary education pre-service teachers were provided a pre-designed HTML template, developed with
Mozilla Composer by the PT3 grant team. Middle level program portfolios (8) were all electronic portfolios and were
also constructed using a template. Each portfolio was scored by the grant evaluator and associates using a scoring
rubric designed to measure outcomes of interest to the Portfolio Connection project (not the ROPA standards).
Readers underwent a series of calibration exercises to ensure reliability of scores, although each portfolio was scored
by a single reader. The scoring rubric evaluated the portfolio using 16 criteria that evaluated each portfolio in terms of
artifact selection, reflection, collaboration, assessment, and technology proficiency. Each criterion was scored using a
scale of 1 to 4 with a score of 1 signifying no evidence; a score of 2 denoting limited or negative evidence; a score of
3 signifying some evidence that is true (but needs better development); and a score of 4 signifying ample evidence.
Preliminary analysis of these scores is summarized as follows:
    Artifact Selection. Overall, the portfolios contained evidence of artifacts being central to portfolio themes (mean
    3.4571 with std of .6572). Artifacts contained rich information about the themes (mean 3.4857). Of the 35
    portfolios scored, the mean score of 2.5429 (std .8521) demonstrated a weak connection between earlier and later
    work and experiences in the program. Of the two programs, elementary education portfolios revealed a stronger
    connection between earlier and later work experiences in the program with 60.2% elementary education
    portfolios receiving a score of 3 or 4 as compared to 37.5% of middle level portfolios receiving a score of 3 or 4.
    Reflection. Evidence concerning reflection was weak in the portfolios sampled. Overall, the mean score for
    reflection resulting from University education courses, including theory and practice was 2.6857 (std. 7581),
    reflection on student teaching and related teaching experiences was 2.7429 (std .95), and reflection on P-12
    students and their response to the portfolio creator as a teacher received a mean score of 2.6857 (std .9322).
    Pearson Chi Square tests comparing the elementary education and middle level programs revealed statistically
    significant relationships between the program and pre-service teacher reflections on students teaching and related
    teaching experiences (chi square=9.805; df=3; p=0.02) and on individual P-12 students and their response to the
    portfolio creator as a teacher (chi square=11.301; df=3; p=0.01).
    Collaboration. Scores referencing collaboration showed almost no participation in professional organizations
    (mean 1.0571 with standard deviation of .3381) and limited collaboration with fellow pre-service teachers (mean
    1.6286 with standard deviation of 1.0314), although Pearson Chi Square calculations did not prove to be
    statistically significant. In terms of evidence demonstrating collaboration and communication with in-service
    teachers, a significant relationship was found (chi square=13.307; df=3; p= .004). Eighty-five percent (85.2 %) of
    elementary education portfolios achieved scores of 3 or 4, whereas 37.5% of middle level portfolios received
    these scores. A Pearson chi-square test was calculated to compare the programs in terms of collaboration with
    in-service teachers. Elementary education students also scored higher with evidence of collaboration regarding
    instructional decision making (e.g., how to handle a particular student learning needs) with 62.9% scoring 3 or 4
    as compared to 25% of middle level students scoring a 3 or 4. A Pearson Chi Square test was calculated and a
    significant relationship was found (chi square=8.025; df=3; p=0.45).
    Assessment – Portfolios provided only weak evidence that teachers use a variety of tools to assess the students
    they teach (2.5143 with std of .9194). Of these scores, however, 51.8% of elementary pre-service teachers
    received a score of 3 or greater, whereas only 25% of middle level pre-service teachers received scores of 3 or
    greater. A Pearson Chi Square test calculated a statistically significant relationship between teachers using a
    variety of tools to assess students and program area (chi square=9.574; df=3; p=0.023). Our speculation here is
    that the elementary education students development reading and math assessments in course work and that these
    artifacts are included as evidence in their licensure portfolios.
    Technology - Most notable in all portfolios is a lack of evidence that pre-service teachers are using technology
    to analyze student assessment and achievement data (1.00 with std of 0). Of the 35 portfolios reviewed, 80%
    showed no evidence; 8.6% contained some evidence; and 11.4 % contained direct evidence in activities of these
    types. No portfolios received a score of 4 that would indicate ample evidence in using technology supported
    teaching strategies with students. Portfolios also failed to provide evidence that pre-service teachers are using a
    variety of technology supported teaching strategies with students (1.3143 with std of .6761). In evaluating the
    portfolio for making research-based decisions about selecting and using technology in his or her classroom,
    97.1% of the portfolios contained no evidence with only 2.9% showing some evidence.
What are UVM student attitudes about the licensure portfolio process in their programs? Do they feel adequately
prepared by their course experiences?
           A student survey was administered to 100 (79 female, 19 male, 2 missing) University of Vermont pre-service
teachers to all students in selected courses within three program areas (elementary education, middle level education,
and physical education) during the Spring 2006 semester. Each course contained instructional aspects that addressed
portfolio construction and development. Purposive sampling was conducted to identify students in courses where the
PT3 grant team provided professional development or consultation concerning an aspect of eportfolio construction.
Students in program areas were represented as follows: 69 elementary education; 16 middle level; 13 physical
education; 1 secondary education; 1 no program identified. Within this sample, only two (2) students had completed
a licensure portfolio. Thirty-two students (32) students indicated that they anticipated creating paper portfolios and
fifty-nine (59) students anticipated that they would create electronic portfolios.
           Several survey questions queried students about programmatic support of the portfolio process. The survey
revealed, for example, that 51.6% of the students could agree or strongly agree that they were introduced to the idea of
maintaining a portfolio early in their program, leaving approximately half of the students to agree only slightly or not
at all. When asked if students have been collecting artifacts that demonstrate their learning from the beginning of
their pre-service training, 55.7% could agree or strongly agree, with 26.8% only agreeing slightly. This 50-50 split of
responses was found in similar questions regarding relationships between artifact collection with personal learning
and reflection, indicating a lukewarm response to questions that made inferences between the portfolio process and the
individual‟s learning and/or reflective processes.
           Although 82.3% of students could agree (to some degree) that professors provide feedback on artifacts or
reflections, approximately 45% of the students could agree that they frequently share artifacts and reflections with
other pre-service teachers. Whereas students were generally evenly-split (yes or no) concerning their collection of
artifacts (including examples of work from courses, examples of student work collected, video footage of themselves
teaching and/or of students learning), sixty-eight percent (68%) of the students sampled indicated that they were not
collecting examples of electronic resources that they use to support teaching.
           Analysis of student attitudes gleaned from the student focus groups also provides a mixed response to the
licensure portfolio process. A number of students viewed the portfolio process as an extremely stressful exercise with
little relationship to future purpose. One middle level pre-service teacher commented that the portfolio that he was
being asked to create according to the six ROPA entries was equivalent to “taking a test.” An equivalent number of
students, on the other hand, recognized that the process of constructing a portfolio had value in understanding their
personal learning and growth during their preparation programs. Similarly, students had mixed responses concerning
the creation of electronic portfolios using generic HTML software. Undergraduate students who had taken the first-
year technology course were better able to assemble and link their portfolio artifacts. Graduate level students in
middle level program who had not received this professional development were forced to learn these new technology
skills during an already stressful time period and expressed dissatisfaction with the program for expecting them to
have the prerequisite technology skills necessary to accomplish this task satisfactorily (Morris, Lathem, Vandersall, &
Qi, 2007).

Will the quality in artifact development and reflection differ between paper and electronic portfolios?
        As noted previously, 35 licensure portfolios were scored for the purposes of this study. Of these, twenty-
seven of these portfolios were completed by students in the elementary education program, 12 of which were
submitted in paper format and 15 were produced electronically using pre-designed templates that could be modified
by an HTML editor (generally Mozilla Composer). The remainder of portfolios were also electronic and produced in
a similar manner due to program mandates. It is difficult to draw conclusions on differences between paper and
electronic submissions given this small sample size from only elementary education students and the lack of format
choice for students in the middle level program. Recognizing these limitations, the following summary is presented:
          In comparing scores between paper (n=12) and electronic portfolios (n=23), paper portfolios receiving scores
of 3 or 4 (showing some or ample evidence of meeting the criteria) predominate in every scored criterion. For
example, artifacts demonstrating connections between earlier and later work and experiences in the program receiving
a score of 3 or 4 were found in 75% of paper portfolios as compared to 43.5% in electronic portfolios. Similarly,
portfolios that demonstrated reflection arising from university education courses, including reflection on education
theory and practice, that received a score of 3 or 4 were found in 66.6% of paper portfolios as compared to 52.2% of
electronic portfolios. Reflection on student teaching and related experiences scored at 3 or 4 were found in 91.6% of
paper portfolios and in 47.8% of electronic portfolios. Paper portfolios scored highest (16.7% receiving a score of 3)
for the criterion evaluating whether the pre-service teacher uses a variety of technology supported teaching strategies
with students, whereas only 8.7% of electronic portfolios received a score of 3 for this criteria. A Pearson Chi Square
calculation found a significant difference between portfolio format (chi square=8.518; df=3; p=0.036) concerning
technology supported teaching strategies with students.

          Licensure portfolios are critical assessments for pre-service teachers and their development and construction
entail a complex process that attempts to provide evidence of teaching competency. Preliminary findings from this
study reveal that University of Vermont faculty are scaffolding instruction at both the macro and micro levels. Both
the elementary education and the middle level programs have at a macro level introduced field experiences and course
work to ensure that students are able to construct artifacts and reflections which are essentially “portfolio ready.” The
physical education program has also identified courses in which artifacts may be developed, but to a lesser degree than
the other two programs, with oversight assigned to a single faculty member.
           Of the three program areas, the middle level program has adopted a developmental approach to the portfolio
process and require their undergraduate students to construct three different portfolios – the first as an application to
the professional block, the second as an application to student teaching, and the third to become a licensure teacher.
Undergraduate middle level students therefore experience curricula that purposefully builds knowledge and portfolio
construction skills and concepts are introduced in logical and meaningful increments. Post baccalaureate students in
middle level program, in contrast, enroll in a year long intensive program and this developmental approach is not
possible. Graduate level students in the middle level program do not have the same opportunity in field experiences
as the undergraduate middle level students nor have the opportunity to develop their portfolio over an extended period
of time as do the undergraduate students. Whereas undergraduate students in all three programs are required to enroll
in a computer applications course where they learn basic technology skills that include web development software and
procedures, middle level post baccalaureate students are not required to take a course that ensures they have
technological proficiency.
          The elementary education program ensures that students are able to develop artifacts that address ROPA
entries through course assignments. For example, students are required in one of their field experiences to identify one
student identified as having special needs. Course assignments require the student to demonstrate how the teacher
would modify instruction to meet individual needs. This assignment neatly mirrors ROPA Entry Three –
Accommodating Students Identified – and students are encouraged to include this assignment as evidence for Entry
Three. Because the ROPA scoring criteria influences the performance tasks that the pre-service teachers are asked to
produce, the final portfolio leaves little room for alternative evidence to emerge.
          UVM pre-service teachers express ambivalence about the portfolio process, and many view the portfolio as a
final assignment rather than a document that mirrors a personal portrayal of their professional growth and competency
to teach. Many students express dissatisfaction and frustration because of the stress caused while constructing the
portfolio and are unclear about the future purpose of the portfolio after it is completed. For most, the licensure
portfolio is finalized during the last week of the student teaching semester because a number of required artifacts and
reflections can only be accomplished during this field experience. The weak scores for reflections evaluated in this
study may be an indication that students simply do not have the time or opportunity to reflect deeply about their
student teaching experience and may also reflect the haste with which the final portfolio is constructed.
          ROPA criteria require colleges and universities to prove that students have received and mastered pre-
determined standards. This assessment has caused faculty to “teach to the test” and in the process inhibited
development of self-designed, expressive portfolios. The following excerpt from one faculty member captures this
         Historically, we started in a completely different place, and we wanted it [the portfolio] to be a more
         interpretive statement by the students about their own teaching. That was at a time when the state
         requirements just said put in a lesson plan, put in a unit plan, put in an evaluation of your student
         teaching, and everything else was up to you. [We] thought of it as – the portfolio – as a text, a
         narrative, an essay that the student writes about themselves with documents and so on. And that‟s
         pretty much how the portfolio operated for its first years until they revised the portfolio in – [..] - the
         second ROPA. And at that point they introduced the six requirements. They – I was on the
         committee that developed them so and certainly know what they were. But the rubrics for each one
         certainly took over control of how people looked at that portfolio because faculty who were asked to
         evaluate the portfolios were given the rubrics for the six entries and asked to evaluate the material
         provided against that rubric. So we started teaching to the test – the State. And now I have to say
         that you know I‟ve done this consciously and willingly. The focus of the course is on developing
         drafts for those six entries so that when they come to the end of the semester and they‟ve got those
         drafts, and if they had nothing else, they could polish those and put them in. Then they‟d have a
         portfolio that was 90% complete.

         The transition from paper to electronic portfolios created with HTML editors has added complexity to the
licensure portfolio process because of the technological proficiency it demands. Faculty members and pre-service
teachers must be skilled in two ways – both pedagogically and technically – to be able to construct efficiently a
representative portfolio of high quality. Scant evidence is available that students have been taught the technical skills
they need to construct generic electronic portfolios using HTML editors or to apply the use of technology in
classroom instruction and assessment. The overall weak results from licensure portfolios scored by Arroyo Research
Services may indicate that students are spending more time getting their electronic portfolios to link properly than
they are developing comprehensive and representative reflections and artifacts that document their teaching practice.
The quality and integrity of the paper portfolios scored in this study were higher than the electronic portfolios
analyzed in this study. However, much more research is needed to determine whether or not there is a difference in
content quality between paper and these types of electronic portfolios.


Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How People Learn: Brain, Mind, Experience, and
         School (Expanded Edition ed.). Washington, D.C.: National Academy Press.
Carney, J. (2006). Analyzing Research on Teachers' Electronic Portfolios: What Does It Tell Us About Portfolios and
         Methods for Studying Them? Journal of Computing in Teacher Education, 22(3), 89-96.
Five Standards for Vermont Educators: A Vision for Schooling. (2003). Retrieved April 11, 2006, 2006, from
Hammond, J., & Gibbons, P. (2001). What is scaffolding? (Publication. Retrieved April 15, 2006, from ERIC
         Document Reproduction Service No. ED 456477:
Ma, X., & Rada, R. (2006). Individual Effects of a Web-Based Accountability System in a Teacher Education
         Program. Journal of Computing in Teacher Education, 22(3), 111-119.
Morris, J., Lathem, S., Vandersall, K., & Qi, J. (2007). Electronic Portfolios: From learning and assessment to
         supporting reporting. Unpublished accepted paper for Society for Information Technology and Teacher
         Education (SITE) conference in San Antonio, Texas, March 26-30, 2007. University of Vermont.
Shepard, L. A. (2005). Linking Formative Assessment to Scaffolding. Educational Leadership, 63(3), 66-70.
Wetzel, K., & Strudler, N. (2005). The Diffusion of Electronic Portfolios in Teacher Education: Next Steps and
         Recommendations From Accomplished Users. Journal of Research on Technology in Education, 38(2), 231-

Shared By: