Maturation of Instant Messaging: Savings, Behaviors, Social Networks, and Beliefs*
Michael J. Muller, Mary Elizabeth Raven, Sandra Kogan, David R. Millen, and Kenneth Carey IBM Research / Collaborative User Experience and IBM Software Group One Rogers Street, Cambridge MA 02142 USA +1 617 693 4235 mullerm@acm.org ABSTRACT We describe the introduction of Lotus Sametime™, an instant messaging (IM) product, in an organization. This brief report analyzes the development of several aspects of user experience over a 24-month period: (1) savings in the use of other communication channels, (2) growth in messaging behaviors, (3) change in messaging partners, and (4) beliefs about the value of the product. Although IM is a simple “walk-up-and-use” product, users show subtle development in skills over an extended period. Keywords Instant messaging, Chat, Synchronous communication INTRODUCTION What happens when people begin to use a simple communication product? How quickly does their use make a measurable difference? How do their skills mature? We studied these questions during a 24-month introduction of Sametime, IBM’s instant messaging (IM) product, in an IBM organization. Sametime offers a suite of services, including synchronous meetings, application-sharing, and an application programmer interface that can be used by other services. This paper deals only with the Sametime Chat client, which provides a user-constructed list of frequent IM partners, the ability to exchange text chats with those partners, and awareness indicators of which of those partners is currently online. In partnership with the sponsoring organization, we conducted survey-based self-report research into people’s use of Sametime during a 24-month period. Individual users began using the product on different dates. We were thus able to administer the survey to 283 people in five cohorts of experience with the IM product: 1-3 months (n=10), 4-6 months (n=33), 6-12 months (n=74), 12-18 months (n=59), and more than 18 months (n=107). We note that we did not survey the same individuals on five different occasions. Rather, we surveyed different groups of individuals at five different levels of experience. This approach is has risks, because
*
To appear in Proceedings of CSCW’2002 (New Orleans LA USA, November 2002). New York: ACM (interactive poster).
different experience levels might have been linked to other, unmeasured differences among the groups. However, demographic data did not suggest group differences. RESULTS Savings in other Communication Channels One of the crucial claims of IM products is that they reduce the usage of other, non-IM communication channels. We began our analysis by testing this simple claim, grouping all 283 respondents into a single sample. Respondents used a seven-point scale to indicate whether their use of other communication channels had increased, decreased, or remained the same following their adoption of IM (not shown in Figure 1). Using the one-sample Z-test, we found reliable decreases in the use of each of the six channels that we asked about: Email (Z=-11.05, p<.0001), Voicemail (Z=14.67, p<.0001), Telephone (Z=-15.28, p<.0001), Teleconference (Z=-5.40, p<.0001), Pager (Z=-10.5, p<.0001), and Face-to-face meetings (Z=-10.97, p<.0001). These results support the hypothesis that IM use reduces the use of other communication channels.
4
Total number of Reasons to use IM Total Reasons to use IM
2 75 Managers Friends
Chat Partners
0 10 Persons/Day
Chat Behaviors
6
Messages/Chat
2 Savings No change Mean Savings in the use of Other Communication Channels 1-3 4-6 6-12 12-18 18+ Mean Savings
IM Experience (months)
Figure 1. Development of IM savings, behaviors, social networks, and beliefs over 24 months.
Figure 2. Reasons for using IM.
Trends in these savings across time appeared flat (one-way ANOVA tests at p<.05 significance level).1 Thus, the savings reported above appear to have occurred within the first three months, and they appear to have remained stable throughout the study period. For reasons of space, we have summarized these results with a single mean savings measure in Figure 1. Developments over Time Despite the lack of significant trends in savings over time, we found other developments over the study period. Chat Behaviors We asked about two chat behaviors: number of Person-chats per day, and number of Interchanges per chat session (Figure 1). Both measures increased with increasing IM experience, by one-way ANOVA: Person-chats/day, F(4,271)=3.66, p<.007 and Interchanges/session, F(4,272)=2.60, p<.04. Thus, despite the stability in the savings, we observed a gradual i n crease in sophistication of usage (similar results were reported by Isaacs et al. [2]). Chat Partners We also asked about chat partners – i.e., whom did people chat with? (In the language of Hersleb et al. [1], this could be rephrased as, Who is part of my critical mass for successful IM adoption?) These measures also showed changes over time for friends and for managers, as assessed by chi-square tests (Figure 1): Friends, χ2(4)=11.97, p<.02, and Managers, χ2(4)=26.72, p<.0001. Again, these results are evidence of gradual changes underlying the early and stable savings results. We did not find significant trends for partner categories of Team members, Tech support, Members of other departments, or a general Other category.
1
To save space, we omit the details of non-significant tests.
Reasons for Using IM Finally, we asked people to report (via checkboxes) on their reasons for using IM. We found significant trends over time in two of the suggested reasons: Scheduling an event, χ2(4)=12.46, p<.02, and Socializing, χ2(4)=13.22, p<.01. When we totaled all of the reasons given (a figure that could vary from 0-8 per respondent), we found an increasing trend over time, as measured by one-way ANOVA, F(4,278)=7.23, p<.02 (this total number of reasons is plotted in Figure 1). (For completeness, we note that we did not find significant trends for suggested reasons including Get a quick response, Avoid the telephone, Know who is available, Clarify a question, Shorten a meeting, or Other – a fill-in.) Again, despite the early and stable savings data, we find subtle changes in people’s beliefs about why they are using IM. Ethnographic research also suggests that people are finding new uses for existing IM features (these data will be detailed in a subsequent report). We note that the data about reasons for using IM tell a different story from the “outeraction” hypothesis of Nardi et al. [3]. They suggested that the principal use of IM was to make quick and lightweight arrangements for subsequent non-IM communications (telephone, face-to-face, meetings, etc.). Along with Isaacs et al. [2], we found that people report a more diverse set of reasons for using IM, and that the reason of Scheduling an event (the category closest to outeraction) ranked seventh out of the eight reasons cited (Figure 2). In our sample, Quick responses to questions, Avoidance of the telephone, Knowledge of who was available, and Clarifying a question were given at least twice as often as Scheduling an event, as reasons for using IM. CONCLUSION Although IM is a successful “walk-up-and-use” application, people show a gradual and subtle development of skills, social networking, and discovery of uses of the technology over time. We hope to clarify the choice of chat partners and the discovery of IM usefulness in future work. REFERENCES [1] Hersleb, J.D., Atkins, D.I., Boyer, D.G>, Handel, M., and Finholt, T.A. (2002). Introducing instant messaging and chat into the workplace. Proc. CHI 2002. Minneapolis MN USA: ACM. [2] Isaacs, E., Kamm, C., Schiano, D.J., Walendowski, A., and Whittaker, S. (2002). Characterizing instant messaging from recorded logs. CHI 2002 Extended Abstracts. Minneapolis MN USA: ACM. Nardi, B.A., Whittaker, S., & Bradner, E. (2000). Interaction and outeraction: Instant messaging in action. Proc. CSCW 2000. Philadelphia PA USA: ACM.
[3]