Template for Case Study Report

Document Sample
Template for Case Study Report Powered By Docstoc
					Evidence of Learning in Blogs

1

Data Analysis Final Report

Evidence of Learning in Blogs:

Kathleen Borsos Wooley

and

Kacy Whittenburg

San Diego State University

Submitted May 5, 2006

Evidence of Learning in Blogs

2

Overview Blogs are changing the face of education. A recent literature review revealed that Blogs are becoming an increasingly integral component of social, educational, political, and professional discourse in our society. Blogs are becoming so entrenched in our culture that new terms are being coined as a result of the use of this tool. The cyberspace of the Blog is now known as the Blogosphere (Drezner & Farrell, 2004) and the captured audience of a Blog has been referred to as the Blogerati. Research Purpose After conducting the teams’ two literature reviews, which included the education, corporate, and political arenas, we surmised that Blogs may be a rich resource for promoting learning in Higher Education, as well as Professional Development for Educators and Instructional Designers. The purpose of this study was to determine if Blogs are an appropriate learning tool. In order to do this, we conducted a content analysis of 12 current Blogs in order to quantify the amount and type of learning demonstrated. The 12 Blogs were categorized in to three similar groups. An ancillary comparison in this case study was to determine if there was any significant difference between the three categories chosen to study.

Evidence of Learning in Blogs Impact of the Literature Review

3

Although there are a number of definitions for Weblogs, none really capture their true essence. In order to understand the Blog’s nature requires participation, or at minimum, reading other individuals’ postings. The Pew Internet & American Life Project conducted a national phone survey in 2003 regarding Internet use. Their findings suggested that nearly 44% of all U.S. Internet users, or 53 million users, had used the Internet to publish their thoughts, respond to others, post pictures, share files and otherwise contribute to the explosion of content available online (Pew Internet & American Life Project Website, ¶1). More current estimates are that there are 200 million Blogs on the web today (http://www.comagz.com/webmagazine/news/number_of_Blogs_estimated_at_200_million.) Blog topics include anything from art to politics, science to religion. Wikipedia identifies fourteen categories of Blogs: personal, cultural, topical, business, science, moblog (mobile blog), collaborative (group blog), eclectic, educational, directory, forum, splog (spam blog), sketch, and political blog (http://en.wikipedia.org/wiki/Blog#Types_of_Blogs.) The use of Blogs is apparently only limited by an authors’ imagination. This literature review was limited to Blogs related to learning, in both the educational and corporate arenas. Even in its infancy, it is apparent that the rich social environment of the Blog promotes a type of cooperative learning by its very nature of collaboration. There is interest in blogging as a tool to promote deeper cognitive processing, which can lead to deeper

Evidence of Learning in Blogs

4

understanding of the material being taught. Research indicates that the conversational nature of the Blog leads to interactivity, inspires active learning, strengthens teacher/student relationships, increases higher order thinking skills, and improves flexibility in learning (Ferdig & Roehler 2003-2004).

An example of this is reflected in a statement by Barbara Ganley, professor at Middlebury College, who wrote that weblogs “allowed the writers the time to slow down their thinking, to consider the contributions of their peers and to respond thoughtfully and reflectively.” With a little practice student bloggers were writing better arguments and improving their communications skills (Ganley in Richardson 2003).

This professor’s opinion supports the idea that, in addition to capturing events and disseminating information, the Blog environment is truly being teased into an amazing learning tool. An example of the need in education for assistance in the learning process is provided by Graesser, Person and Hu (2002): The painful fact that students rarely acquire a deep understanding of the material they are supposed to learn in their courses is widely acknowledged in the field of education….What is missing are the deep coherent explanations that organize the shallow knowledge….

Evidence of Learning in Blogs

5

Anyone who has a computer and internet access can now have a slice of cyberspace. Since Blogs are so accessible, these researchers believe they offer a profound opportunity for learning in any given content area and for all learner types. Armed with this encouraging information, and because little quantitative data exists currently, these researchers set out to collect data on evidence of learning in Blogs. Because of the wide array of Blog categories, it was agreed that the research would be limited to adult learners in three categories: Higher Education, Professional Development for Instructional Designers, and Professional Development for Educators. Contextual Factors As mentioned earlier, the massive area of content that can be covered and the vast array of learner types involved in blogging led to a decision to limit the types of Blogs to be studied. Therefore, any content outside of the three designated categories was not addressed in this study. In addition, no data was collected on younger learners. One further limitation is that the study was limited to a recent one month period post for each Blog. Therefore, the study may not be an accurate reflection of an ongoing Blog.

Methodology After conducting the literature review, the team identified three categories of Blogs related to education and learning. These categories are Higher Education, Professional Development for Instructional Designers, and Professional Development for Educators. It was agreed to select 4 Blogs from each of the three categories for a content analysis.

Evidence of Learning in Blogs

6

Blogs included in this study were located through resources listed in the literature review, snow-balling and Google-search. After selecting appropriate Blogs for the analysis, the team elected to review one month of posts from each Blog. After an initial review, the team determined that the most limiting Blog contained 9 posts in a month. The team decided to code 9 posts from each Blog as a representative sample. The posts were selected in a random manner. The team developed a coding scheme to capture various elements that would show evidence of learning throughout the posts, (See Table 1). The coding scheme included a comprehensive categorization of learning related content. Each code was defined and the definitions were used to guide the evaluation of posts. To reduce bias, two types of triangulation were included in this study. They are as follows:  Data Triangulation – data was collected from different sample Blogs in three education related content areas.  Inter-rater Triangulation – data was analyzed by two researchers. Although there was no guarantee of inter-rater reliability, the team nevertheless made an attempt by using this strategy.

Evidence of Learning in Blogs Table 1. Coding scheme used for Instructional Designers, Higher Education, and Primary/Secondary Educators. Category Interpersonal Skills Codes (see below) Information States facts, concepts, “need to know” items, assignments, etc. Meanings Example

7

An interesting article on the terrific Tomorrow's Professor Blog, a collaboration between MIT and Stanford, Preparing Faculty for Pedagogical Change: Helping Faculty Deal with Fear by Linda Hodges, outlines underlying fears that make shifting to new pedagogies, including collaborative and problem-solving learning, so overwhelming to many teachers. And regarding effectiveness, I don't think Blogs are dead, just sometimes not used effectively. Like using a baseball bat to play 18 holes of golf...you can launch the ball, but actually sinking the putt will be extremely challenging. People will eventually know what tool to use to get the job done right, it just may take some time.

Example/NonExample

Provides an illustration or case in point/provides illustration or case in point that does not do so.

Knowledge Construction

(see below)

Provides reflection, evidence of applying information to “real life” situations, sees “big picture”, combines several ideas, draws inferences, relates to other similar experiences Provides a breakdown of fact,
It seems to me that we may have another kind of gap looming in our future. Europe and the United

Analysis

Evidence of Learning in Blogs idea or concept

8
Kingdom appear more eager to put funding into worker skills enhancement than we are in the United States. If, as predicted, many positions in the future will go to China and India, will the U.S. and Europe/U.K. be left fighting over the scraps? If so, who will win that battle? I am really starting not to see gaming and instructional design as rivals or obvious alternatives but instead as different species which do not spring from some UR-progenitor but which are different from the DNA up. I think it is not trivial to consider the military/industrial roots of ISD as compared to and markedly different from the ancient roots of gaming. Remember, I am not talking about learning here - which I think is inextricably linked with gaming - but rather with ISD. Here, for the first time anywhere, is what I really want to say: "Listen. I have been gaming classrooms for my entire life in order to get better evaluations, comments, grades, or certification scores. I have been dressing appropriately, feigning interest in topics that bore me beyond belief, cramming for tests in a way where my command of the information has a half life of hours desperately hoping that I forget the information moments after I write it down not moments before, skimming tangential sources to ask the one question that makes me seem much more knowledgeable than I really am, interviewing past students to see what will be on the test, playing back what the instructor said without understanding it at all, and pretending to take notes when I am really designing a biosphere in the margin. You want to talk about gaming? What

Evaluation

Provides an assessment of fact, idea or concept

Synthesis

Incorporates information into a “bigger picture”

Evidence of Learning in Blogs

9
do you think all of your students are doing all of the time?"

Reflection

Provokes contemplation or musing of an fact, idea or concept

It seems to me that we may have another kind of gap looming in our future. Europe and the United Kingdom appear more eager to put funding into worker skills enhancement than we are in the United States. If, as predicted, many positions in the future will go to China and India, will the U.S. and Europe/U.K. be left fighting over the scraps? If so, who will win that battle? Gaming Classrooms Response: Well put, Clark. Let's make the game more explicit, since it's been implicit for years. I did wonder aloud about whether you could really pull it off in our current educational system--do teachers have the time and energy to bring one more thing into their classrooms, especially if it means more for teachers to learn and to orchestrate.

Interpretation

Provides explanation or another way of stating fact, idea or concept Contests or questions previous statement(s)

Debate

Comments

(see below) Agree Concur with previous statement(s) Differ or oppose previous statement(s)
Thanks for the shout out! What a lovely round-up of links--so much to explore! But you are mistaken - I was (am?) a special needs case! And I was taught in a separate stream until the age of 10 when we moved to an area that didn't offer that option. Within a year, my results had deteriorated and never again rose much above mediocre until I left school Hey, what a great idea! And let me say thanks back, you've been a continual source of inspiration for me. Whenever it seems like exciting teaching and learning (with or without technology,

Disagree

Understand

Evidence of understanding information or concept

Evidence of Learning in Blogs

10
but especially with) can't or won't fulfill its potential, I think of all the exciting things you've done. I do believe that good teaching is teachable, and that people can change even the most firmly engrained practices; we've got a lot to learn from you.

Don’t Understand

Evidence of not understanding information or concept Expands on or provides additional information on previous item or point of discussion

I wonder, is there something ironic about the comment above, clearly a form of spam? Is it an oblique comment on the notion of games? The response to the T+D cover feature on the Skills Gap has been pretty exciting. I have had some very positive phone calls and emails. I hope you have had time to read it. If you would like an opportunity to dive a bit deeper into the subject, and perhaps quiz some experts, I would urge you to join in the follow-up webcast tomorrow. (That would be Tuesday, Feb 14, 2 p.m. Eastern). To get ready for the webcast, please check in at this site. It will walk you through the very short technology checkup that will allow you to participate in this event as well as other future webcasts from ASTD. Thanks. I look forward to talking with you on Tuesday. Peters is a hell of a feisty interview, but he is equally aggressive as an interviewer, too. For fun, check out his interview of author and behavioral science expert Aubrey Daniels. It is both revealing and uncomfortable to watch Peters tackle a subject and one of its leading experts. But it reminds me, as a journalist, to always ask the uncomfortable questions. It may make the interview subject squirm, a bit, but the end result is always worth it. Now I'm looking forward to learning even more from you. When I have time I am also looking forward to responding to Karyn. In short, Karyn, yes I think you are missing something

Elaboration

Offers Additional Provides additional Resources information or links to other sites

Constructive Criticism

Points out shortcomings of previous content or of other’s ideas in a

Evidence of Learning in Blogs postive manner

11
very important. Actually several things - and I bet you will be able to think of them by yourself. Or perhaps others with "special needs" (not you?) will help. Microsoft Police? Submitted by Stuart Yeates (University of Oxford) on Tue, 2006/03/21 2:14am. Microsoft have launched a new round of legal actions, describing it as a law enforcement campaign that will target cyber-criminals ...and here was me thinking that we have police forces to do law enforcement. The cynic in me suspects that the new anti-spam (which Microsoft usually calls anti-phishing to avoid trademark issues with SPAM), is advance groundwork for the marketing effort for the anti-spam features of the next iteration of windows including antispam measures.

Negative Criticism

Points out shortcomings of previous content or other’s ideas in a negative manner

Instrumentation Due to the nature of the research, it was determined by the team and Dr. Wang that the appropriate method of investigation for this study would be a content analysis of the Blogs. For this study, the instruments are the researchers themselves. They created a coding scheme and used it consistently to code all the sample Blogs. The technical tool used to collate and organize data was Microsoft Excel. Analyse-It was used for inferential statistical analysis of the data. See Appendix A for Excel Document.

Evidence of Learning in Blogs Collecting Data Data in this study was collected using the following procedure:

12

1. Assign two Blog’s from each category to each rater; for a total of 6 Blog’s per rater. 2. Locate and access Blog via internet. 3. Cut and paste posts and comments for a representative one month period, and insert in Excel document. One post with related comments per cell. 4. Calculate word counts for each post and record results in cell. 5. Read and analyze post. 6. Code content, and enter results in cell. 7. Collate coding numbers in tabular form for each Blog. 8. Assemble data from both raters in to one Excel document. 9. Consolidate raw data into a single table for analysis. 10. Calculate mean for each content code, and each Blog category. 11. Create tables from data for descriptive analysis. 12. Perform 1-way ANOVA using Analyse-It, for inferential analysis.

Data Analysis Data collected during this study is subjective in nature. Raters analyzed Blog posts using a coding scheme (see table 1). Table 2 shows two representative posts from the Blogs sampled for analysis. Both posts are representative of education-related Blogs sampled in this

Evidence of Learning in Blogs

13

case study. Both authors’ offer information about an education or instruction related idea, and offer personal insights about the information. The content of both posts focuses on current technologies and their potential applications for education, which is also typical of posts found in the representative Blogs sampled in this case study. You will also see sample coding in table 2. Note that in a single post, there may be multiple codes assigned to learning related content. See two representative posts, Table 2.

Table 2.

Representative coded posts from two Blogs.

DATE

POST
Weblog-Ed, The read/write web in the classroom http://www.weblogg-ed.com/ Bloggy Research The folks over at the CCCC Blogging SIG are taking the blog by the horns in terms of beginning to gather some empirical research about the effects of Blogs in the classroom. I still think it’s weird that no one has published any results of studies with this tool yet. I may have to carve out a few hours to go digging around some more. They’ve also got some other things on the agenda. One of my favorite snippets is this one: …we need to move the profession towards a space where we’re more aware of blogging as professional activity. To what degree can we “get credit” for blogging? And, deriving from that, how can we start thinking about blogging as professionals? (One question that was asked in response: if blogging becomes a professional activity, does it lose some portion of its value as teaching/writing tool?) Wow…we’re finally getting serious about this stuff, huh? Good questions that we’re all grappling with on some level, and I’ll be interested to see how things progress. Posted by Will Richardson

CODE

3/30/2006

Information, Reflection, Analysis, Debate

Evidence of Learning in Blogs

14

3/31/2006

BLOG: Educational Technology Weblog http://www.psesd.org/weblogs/edtech/ LATEST TRENDS IN ONLINE LEARNING Yet another article about online learning, this time in the Christian Science Monitor. There isn't too much radically new, but it does reflect this continues to be an increasing trend. The Florida Virtual High School is growing at the phenomenal pace of 40-60% per year, and now Michigan is poised to be the first state to require that every student take at least one online course before graduating from high school. There's still some muddy thinking floating around out there, though. One expert is quoted as thinking that virtual classes "...could help ease the nationwide shortage of math and science teachers." All of the online classes referred to in the story use real teachers, so I don't see how that affects the overall shortage problem. Unless, of course, the online teacher supports more students than a face-to-face teacher - which they don't in any (good) program I've seen. I'll be really happy when we have stamped out the notion that online learning somehow costs less than classroom learning. Posted by Conn McQuinn at 11:31 AM

Information Interpretation Evaluation Analysis Interpretation

Descriptive Analysis The data was organized in to tables for descriptive analysis of content. Tables 3, 5 and 7 display the summative results for each Blog category: Instructional Design / Educational Technology, Higher Education, and K-12 Educators Professional Development. Each table contains data from four Blogs. Nine posts from each Blog were randomly sampled, and coded. There are sixteen possible code categories. Tables 4, 6 and 8 display summative results for the each Blog category: Instructional Design / Educational Technology, Higher Education, and K-12 Educators Professional Development. Each table contains data from four Blogs. Tables 4, 6 and 8 only contain data from the BLOGGER posts code classifications which include: information, example/nonexample, analysis, evaluation, synthesis, reflection, interpretation, and debate. The code classification from the comments is not included in these tables.

Evidence of Learning in Blogs Table 3.

15

Collated results from four Instructional Design / Educational Technology related BLOGS.
Inst. Design / Ed Technology Professional Development

20 18 16 14
Total # Occurances

Information Example/Non-Example Analysis Evaluation

12 10 8 6 4 2 0 Ed Tec Ed Tec Weblog Training & Development Learning Circuits

Synthesis Reflection Interpretation Debate Agree Disagree Understand Don't Understand Elaboration Offers Additional Resources Constructive Criticism Negative Criticism

BLOG Name

Evidence of Learning in Blogs Table 4.

16

Collated results from four Instructional Design / Educational Technology Related Blogs. Data solely from BLOGGER Post Code Classifications. Comments not included.
ID/EdTec Data for BLOGGER Post Only
20 18 Information Example/Non-Example

16 14
Number of Occurrences

12 10 8

Analysis

Evaluation

Synthesis

Reflection

6 4

Interpretation

Debate

2 0 Ed Tec Ed Tec Weblog Training & Development Learning Circuits

Blog Name

In table 3, there is a wide range of learning related content evident in the four Instructional Design/Educational Technology related Blogs. For instance, the posts from Learning Circuits, an online e-learning magazine, are richer, with learning-related content across 15 out of 16 coding categories, and 100 total code occurrences per nine posts. Training & Development’s Blog contains content across all 16 coding categories, and a total of 73 code occurrences. Both Blogs are in contrast to Ed Tec, a weblog by Ray Schroeder in the Office of Technology-Enhanced Learning. The Ed Tec Blog contains content in 9 code categories, and a total of 27 code occurrences. It is also in contrast to Ed Tec Weblog, authored by Conn McQuinn of the Educational Technology Support Center, who posted learning related content in 7 code categories, with 50 total code occurrences.

Evidence of Learning in Blogs

17

Most codes occur four to eight times per total posts, but the range of code occurrences is from 0 to 18 per total posts. Information has the highest number of occurrences (49 total for four Blogs), followed by “disagree” with 22 and “understand” with 19 total occurrences for four Blogs. “Don’t Understand” has the least amount of occurrences (1 in 4 Blogs), followed closely by “disagree” with 5 and “negative criticism” with 6 total occurrences. When you look at the data from the same four Blogs, but only consider the first 8 code classifications, as seen in Table 4, fewer differences are apparent. All four Blogs have codes in at least 7 out of 8 classifications. Learning Circuits has the highest number of total codes, with 52, followed by Ed Tec Weblog with 50 codes, and Training & Development with 39. Ed Tec still has the least amount of learning-related content with a total of 24 codes per 9 posts.

Evidence of Learning in Blogs Table 5. Collated results from four Higher Education related Blogs
Higher Edcuation Related BLOGS
45

18

40 Information 35 Example/Non-Example Analysis 30
Total # Occurances

Evaluation Synthesis

25

Reflection Interpretation

20

Debate Agree

15

Disagree Understand

10

Don't Understand Elaboration

5 Offers Additional Resources Constructive Criticism Thoughts, Arguments & Rants EDUCAUSE Creative Writing Class BLOG Name Barbara Ganley Negative Criticism

0

Evidence of Learning in Blogs Table 6.

19

Collated results from four Higher Education Related Blogs. Data solely from BLOGGER Post Code Classifications. Comments not included.

Higher Ed Data for BLOGGER Post Only
45

40

35

Number of Occurrences

30

25

20

15

Information Example/Non-Example Analysis Evaluation Synthesis Reflection Interpretation Debate

10

5

0 Thoughts, Arguments & Rants EDUCAUSE Creative Writing Class Blog Name Barbara Ganley

The data in table 5 is from four higher education related Blogs. Overall, there is a diversity of learning-related content contained in all four Blogs. Barbara Ganley’s Blog (a professor of creative writing at Middlebury College,) contains learning related content in all 16 classifications, with a total number of 69 code occurrences in 9 posts. Similarly, the Blog from her Creative Writing Class, has 15 content codes, and 70 total occurrences. Brian Weatherson’s Blogs, “Thoughts, Arguments & Rants,” a Blog from his philosophy class, contains 130 total code occurrences in 14 learning related content classifications. Educause, a nonprofit association whose mission is to advance higher education by promoting the intelligent use of information technology, has the least number of code classifications and code occurrences, with 9 and 32 respectively.

Evidence of Learning in Blogs

20

Most code classifications occur between 3 to 9 total times. The range of code occurrences is from 0 times to 41 total times. The highest code occurrence across all four Blogs is “information” with a total of 68 occurrences. “Elaboration” is the second highest code classification (32 occurrences), followed closely by “analysis” with 30 occurrences. The lowest code occurrence across all four Blogs is “disagree” with a total of 4 occurrences. “Negative criticism” is the second lowest code classification (6 occurrences), followed closely by “constructive criticism” and “don’t understand” both with 7 total occurrences. The data in table 6 is similar to table 5. In table 6, data from the BLOGGER post only, without any comment related codes, is shown. You can see that Thoughts, Arguments & Rants is by far the Blog with the most learning related content (a total of 103 codes), and Educause has the least number of codes (24 total). The Creative Writing Class Blog and Barbara Ganley’s Blog are intermediate, but very similar to each other (both with 38 total codes per 9 posts).

Evidence of Learning in Blogs Table 7.

21

Collated results from four K-12 Educator Professional Development related Blogs.
Primary/Secondary Educators Professional Development

60

50

Information Example/Non-Example Analysis

40

Evaluation Synthesis Reflection

Total # Occurances

30

Interpretation Debate Agree

20

Disagree Understand Don't Understand

10

Elaboration Offers Additional Resources Constructive Criticism

0 Blogging for Ed Ed Tec Ed Weblogs BLOG Content Weblog-Ed

Negative Criticism

Evidence of Learning in Blogs Table 8.

22

Collated results from four K-12 Educator Professional Development related Blogs. Data solely from BLOGGER Post Code Classifications. Comments not included.

K-12 Data for BLOGGER Post Only
18

16

14

Number of Occurrences

12

10

8

6

Information Example/Non-Example Analysis Evaluation Synthesis Reflection Interpretation Debate

4

2

0 Blogging for Ed Ed Tec Blog Name Ed Weblogs Weblog-Ed

The data from four K-12 Educators Professional Development Blogs is shown in Table 7. Again, there is a rich variety of learning-related content evident in these Blogs. WeblogEd, Will Richardson’s Blog, has content from every classification code, and a total of 99 total classification code occurrences. Likewise, Al Delgado’s Ed Weblogs has content across all 16 codes, and 59 total occurrences. Although Blog Savvy’s Blogging for Ed, has slightly fewer codes, with 15 out of 16 code classifications, there are a total of 150 occurrences of learning related content in the Blog. Ed Tec, Tim Lauer’s Blog, has both the least number for code

Evidence of Learning in Blogs

23

classifications and the least number of code occurrences, with 10 codes, and 41 total code occurrences. Most code classifications have between 4 and 8 occurrences per 9 Blog posts, with a range from 0 to 51 occurrences. Unlike Table 3 and 5, the learning-related code classification with the highest incidence is “elaboration,” with a total of 71 occurrences total. However, 51 out of 71 occurrences are from a single Blog: Blogging for Ed. “Information” has the second highest number of total occurrences: 47, followed by “offers additional resources” (33) and “constructive criticism” (30). Note that over 50% of the code classifications for “offers additional resources” and “constructive criticism” are from Blogging for Ed as well. Code classifications with the least number of occurrences include: “synthesis” (6), “negative criticism” (6), “don’t understand” (7), and “disagree” (8). After looking at table 8, you will see that the four K-12 Blogs are much alike. Weblog-Ed has content in 8 out of 8 code classifications, and 59 total codes per 9 posts. Blogging for Ed has codes in 7 out of 8 classifications with 40 total codes per 9 posts. Ed Weblog contains content codes in 8 out of 8 classifications, and a total of 31 codes per 9 posts. Ed Tec has both the least number of code classifications (6 out of 8) and total codes (28).

Evidence of Learning in Blogs Table 9. Mean values organized by content code, and Blog category.
Content Comparison
2 1.8
Average # Occurances / BLOG Post

24

1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0
pl Info e/ N rma on t -E ion xa m p An le al Ev ys al is ua t Sy ion nt he si R s e In flec te tio rp re n ta tio D n eb at e Ag re D is e ag U re n D e on de rs 't O t U ffe nd and rs er Ad E sta di tio lab nd C on nal ora tio R st e n ru ct sou rc N ive es eg C r at iv itici e sm C rit ic is m

ID/EdTec Professional Development AVERAGES Higher Ed Related AVERAGES Primary/Secondary Educators AVERAGES

Ex

am

Type of Content

Table 9 denotes a side-by-side comparison of mean values for each category of Blog, by content code. The data in table 9 shows that over-all learning related content is similar in the three Blog categories: Instructional Design / EdTec Professional Development, Higher Ed Related, and K-12 (Primary / Secondary Educators). Out of a total of sixteen content codes, twelve are very similar for all three categories. For instance, in the “negative criticism” column, the means from each category are identical with an average value of 0.1 instances /post. Classifications that are dissimilar include: “information”, “elaboration”, “offers additional resources”, and “constructive criticism.”

Evidence of Learning in Blogs

25

All three categories contain mostly “information”. Information is the highest code category for both ID / EdTec and Higher Ed. The K-12 (Primary / Secondary Educators) show a high number “elaboration” occurrences, with an average of 1.775 instances/post. The content codes that are least common in the representative posts include “disagree” and “don’t understand”. There are an average of 0.1, 0.125, and 0.2 instances of “disagree” per post. And there are 0.025, 0.175, and 0.175 average occurrences of “don’t understand” per post. In general, K-12 (Primary / Secondary Educators) has the most elaboration, offers additional resources, and constructive criticism in their Blog posts. While Higher Ed shows the most information, analysis, and reflection. Understand is the only learning related code in which ID / EdTec has a higher average number of codes per post than either Higher Ed or K-12. Inferential Analysis In order to determine the magnitude of difference between learning related content areas, we conducted a one-way ANOVA on the raw data for each code classification within the twelve Blogs considered in this study. ANOVA (analysis of variance) is a statistical test to determine if there is significant difference on the emergence of the following categories in twelve Blogs: information, example/non-example, analysis, evaluation, synthesis, reflection, interpretation, debate, agree, disagree, understand, don’t understand, elaboration, offers additional resources, constructive criticism and negative criticism. ANOVA is the statistical

Evidence of Learning in Blogs

26

tool of choice in this case study because the data is non-parametric (the data does not lie on a normal curve). In this case study there are 16 codes. ANOVA was performed for each code classification including data from all twelve Blogs. The degree of freedom for each ANOVA, was 2 and 9, with n=12 in each test. Table 10 contains tabulated results from the inferential analysis of the data in this study.

Table 10.

One-Way ANOVA Results, Organized by Content Code

At a confidence level of 95%, ANOVA analysis revealed no significant differences. Findings Summative results for each Blog category indicated a wide range of learning related content in the twelve Blogs analyzed. For instance, in table 3, the Ed Tec Weblog contained 6 different content classifications, and a total of 50 instances recorded. In table 7, Ed Weblogs contained content in sixteen classifications, and a total 56 instances recorded.

Evidence of Learning in Blogs

27

The data represented by Table 9 shows that over-all learning related content is similar in the three Blog categories: ID / EdTec, Higher Ed, and K-12 (Primary / Secondary Educators). Out of a total of sixteen content codes, twelve are very similar for all three categories. For instance, in the “negative criticism” column, the means from each category are identical with an average value of 0.1 instances / post. Categories that are dissimilar include: Information, Elaboration, Offers Additional Resources, and Constructive Criticism. All three categories contain primarily information. The K-12 (Primary / Secondary Educators) show a high number of posts with Elaboration, for an average of 1.775 instances / post. The content codes that are least common in the representative posts include “disagree” and “don’t understand”. There are 0.1, 0.125, and 0.2 average instances of disagree per post. And there are 0.025, 0.175, and 0.175 average instances of “don’t understand” per post. In general, K-12 (Primary / Secondary Educators) have the most elaboration, offers additional resources, and constructive criticism in their Blog posts. While Higher Ed shows the most information, analysis, and reflection. Understand is the only learning related code in which ID / EdTec has more of than either Higher Ed or K-12. When you compare the Blogs with respect to content from the BLOGGER post only, and remove codes from the comments about each post (tables 4, 6 and 8) the data within the Blogs is more similar. Variation is still evident, for instance Weblog-Ed has almost twice as much learning-related content (8 out of 8 code classifications and 59 total codes per 9 posts) compared to Ed Tec (6 out of 8 code classifications and 24 total codes per 9 posts).

Evidence of Learning in Blogs

28

At a confidence level of 95%, ANOVA analysis revealed no significant differences between the twelve Blogs in any learning related code classification.

Discussion and Conclusions In this case study, 12 Blogs were analyzed in order to quantify and identify learningrelated content within the Blog. Our primary interest for conducting the study was to determine if Blogs are an appropriate learning tool. Clyde (2005) reported: There are a number of authoritative educational weblogs that can be used as sources of information and provide professional development material for teachers. There are weblogs that can be used by teachers and students as sources of information for curriculumrelated activities. The 12 Blogs used in this study, represented three Blog categories based on intended audience: Instructional Design / Educational Technology, Higher Education, K-12 Educators. The coding scheme identified learning-related content according to the following classifications: information, example/non-example, analysis, evaluation, synthesis, reflection, interpretation, debate, agree, disagree, understand, don’t understand, elaboration, offers additional resources, constructive criticism and negative criticism. Through inferential analysis of the data collected in this case study, it was determined that there was no significant variation in learning-related content across the three Blog

Evidence of Learning in Blogs

29

categories. Each content code was analyzed using one-way ANOVA, and no code showed significant variation. Through descriptive analysis of the data collected in this case study, it was determined that the greatest differences between the 12 Blogs in all three Blog categories was in the comments. Tables 3, 5 and 7 contained data in all sixteen code classifications (for both BLOGGER post and comments) and the tables appeared to be dissimilar. When the comment data was removed, in Tables 4, 6 and 8, the tables showed greater similarities between Blogs. Overall, Blogs contained information in all learning-related classifications, and showed a rich amount of content (high number of occurrences in each code classification). All three Blog categories (Instructional Design / Educational Technology, Higher Education, and K-12 Educators) showed evidence of containing learning related content. Although more research needs to be conducted, to determine the degree of learning Blogs afford, and the best practices for incorporating Blogs as a learning tool, our preliminary case study shows that Blogs across three categories, show evidence of learning-related content. After reading and analyzing posts from twelve different Blogs, and after observing interactions between Blogger and audience via comments, Harper’s (2005) assertion that “a Blog extends the learning experience well beyond the face-to-face classroom, creating a more complete learning experience.” has merit.

Evidence of Learning in Blogs

30

As mentioned earlier, the massive area of content that can be covered and the vast array of learner types led to a decision to limit the types of Blogs to be studied. Therefore, any content outside of the three designated categories was not addressed in this study. In addition, no data was collected on younger learners. One further limitation is that the study was limited to a recent one month period post for each Blog. Therefore, the study may not be an accurate reflection of an ongoing Blog.

There is evidence to suggest that learning and Blogs are linked. The best uses for Blogs in educational settings still need to be explored. As educators in both corporate and formal education settings adopt the use of Blogs to promote learning, more data will become available to determine the extent to which Blogs and learning are linked.

This research focused on adult learners only. In order to gain a more comprehensive view on the use of Blogs as a learning tool, further research is recommended on younger learners as well.

Evidence of Learning in Blogs References Clyde, L. A. (2005). Educational blogging. Teacher Librarian, 32(3), 43-45.

31

Drezner, D. and Farrell, H. (2004). The Power and Politics of Blogs. Retrieved February 27, 2006 from http://www.danieldrezner.com/research/Blogpaperfinal.pdf Du, H. S., & Wagner, C. (January, 2005). Learning with Weblogs: An Empirical

Investigation. In HICSS '05: Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS-38). Los Alamitos:
IEEE Computer Society. Ferdig, R.E., & Roehler, L.R. (2003). Student engagement in electronic discussions: Examining online discourse in literacy pre-service classrooms. Journal of

Research on Technology in Education, 36(2), 119-136.
Ferdig, R.E., & Trammell, K.D. (2004). Content delivery in the Blogosphere. THE

Journal, 31(7), 12, 16-17, 20.
Graesser, A. C., Person, N. K. & Hu, X. (2002). Improving Comprehension Through Discourse Processing. New Directions for Teaching and Learning, 89, 33-44. Harper, V. B. Jr. (2005). The New Student Teacher Channel. T.H.E. Journal, 33(3), 3032. Pew Internet & American Life Project Website (2006). Retrieved February 16, 2006 from http://www.pewinternet.org/PPF/r/113/report_display.asp

Evidence of Learning in Blogs

32

Richardson, W. (2003). Web logs in the English classroom: More than just chat. English

Journal, 93 (1), 39.
Wesley, D. (2002). A Critical Analysis on the Evolution of E-Learning. International

Journal on E-Learning, 1, 41-48.
Wikipedia: The Free Encyclopedia Website. Retrieved February 2, 2006 from http://en.wikipedia.org/wiki/Social_Software

Evidence of Learning in Blogs Appendices See attached Excel document titled “CaseStudyDataFinal.xls”

33