Virtual Worlds Infrastructure for Large Scale Collaboration Guy Garnett

Virtual Worlds: Infrastructure for Large Scale Collaboration Guy Garnett (UIUC) Roy Campbell (UIUC) Robert McGgrath (NCSA) A Project Summary We seek to empower scientists and engage the scientifically enquiring public in a wide range of collaborative interactive scientific observatories, data, and simulations made accessible and available through public and scientific networks. We propose to enable collaborative scientific environments based on open, extensible software infrastructure that leverages widespread low-cost technology for the creation of 3D persistent worlds drawn from online multiplayer games and adapted to meet the data requirements of scientific collaborative distributed computing. Given a large scale dynamic database associated with such projects as CLEANER (National Research Council), we will create a suite of tools that will enable the acquisition, analysis, visualization, and documentation phases of research to take on characteristics of an online multiplayer game environment. The characteristics include avatar (point of view) movement through data, direct manipulation of data, physics models that interpret the data as an environment for the avatar, integration of the environment with sensors and actuators, multidimensional visual representations of data (3 spatial dimensions plus the temporal dimension), and avatar-mediated co-presence of large numbers of collaborating researchers from physically widely distributed locations. One way to think of this is as an extensible browser for collaboratively exploring, interacting with, or simulating 3D or multidimensional datasets extracted from experiments, computations, or sensor monitoring. While a variety of software exists for handling various portions of this project, the intellectual challenges we will address include integration of these concepts into one interoperable and perceptually seamless whole; scalability to handle large numbers of simultaneous users and very large datasets; creation of an open source that has integrity and is extensible, provision of security and privacy needed for scientific research, and facilitating ease of use for scientists, educators, and others. The last point in particular needs some clarification. We intend the virtual worlds to include workshops and tools that empower scientists and users to customize their own worlds, to factor into the world their own measured or simulated data, to “browse” that world and its datasets via an intuitive avatar-based perspective, to interact with the data and its representations, and to share that world collaboratively with colleagues or visitors through the Internet. A.1.1 Intellectual Merit The easy to use, robust and scalable environment we will build for collaborative exploration of scientific data and models will have a transformative impact on science. Much as the web browser has enabled simple sharing of many kinds of data, especially 2D images, our mWorlds framework will enable easy, widespread sharing of 3D data over medium (consumer broadband) or high bandwidth networks and using desktop PCs as the front end. Beyond that, our framework—derived from commercial multiplayer games, though more robust, scalable, and extensible—will enable a higher level of collaboration and co-experience of the data, commentary, and modeling. It will enable network-based co-presence, through customizable avatars, and foster and support virtual organizations and communities. A.1.2 Broader Impact Like the impact of the web browser on nearly all aspects of life—scientific and cultural—our mWorlds framework will make possible a wide range of new experiences and will have an impact on areas as diverse as scientific computing, education, and entertainment. Indeed any group, institution, or community who wants to would be able to easily share data, collaborate, or co-create. Government entities will find support for environmental modeling, emergency response management, and other areas. Both government and enterprises will have decision support and team building support, and many opportunities for training. The provision of this basic infrastructure will enable new kinds of business. Indeed, companies as diverse as IBM and the Associated Press already have units working in Second Life, the non-extensible, proprietary counterpart to our proposal. Robust open standards for virtual world creation will enable the integration of scientific developments to improve public services, enhance public participation, and reap the benefits of public investments. B Project Description B.1 Background Recent advances in the computer game arena suggest the broad outlines of a paradigm shift in technology-mediated human interactions. Online social spaces such as SecondLife (Rosedale, et al) and shared web spaces such as MySpace (myspace.com) and YouTube (youtube.com) show a tremendous interest on the part of a very large and diverse group of people for one-to-one and many-to-many social interactions on a massive scale. Games such as World of Warcraft, with over six million monthly subscribers, show the attraction of adding social networking, grouping, communication and other features of online social spaces to the already popular role playing interactive game paradigm. Advances in technology that include 3-D graphical processing units, 3-D HDTV displays, inexpensive game consoles, multicore processors, high-bandwidth to the home, and cluster computing offer hardware support and new opportunities for these approaches. Our research goal is to advance and augment the capability of interactive media that will facilitate interactive, creative communication and collaboration on a massive scale and among groups that have not typically been well served by the web or by gaming commercial interests that currently drive these developments. The users who would benefit from our research include gamers, socializers, scientists, artists, entrepreneurs, businesses, academic and industry researchers, humanist scholars, policy makers, governmental and non-governmental actors, and in short, the entire human community. We propose to create an open-source suite of mutually supporting tools to create and develop this cyberinfrastructure. It will support the creation of persistent realistic or imaginary worlds, physical or non-physical, from virtual shopping malls to quantum physics simulations. It will facilitate local and distance collaboration, communication, and creative practice. It will scale from person-to-person to global scale connections. It will provide the most robust and diverse security tools. It will facilitate creativity in world, and of worlds, with a very low entry level, but no ceiling. It will be extensible. B.1.1 Scenario: A Virtual Observatory A global environmental observatory provides one inspiration for how we would like to interact with data intensive Cyber-Physical Systems through an immersive “virtual world”. We envision a standard toolkit for creating an environment resembling contemporary online games, in which scientists, students, or the general public can navigate among observation stations which present “views” from sensors and models. The interfaces will use high quality three dimensional graphics via consumer grade PC’s, delivered over networks. The interaction will be as complex and compelling as computer games, with the sense of presence enhanced by on-line communications via chat and telephony. Furthermore, in a key departure from most existing systems, we propose integrating tools for manipulating the visual interface and the virtual environment directly in the interface itself. This would enable a scientist to easily create new visual representations of both static and dynamic data, and to configure the interface to best represent the aspect of the data they are interested in. For example, NSF is developing several Earth Observatory projects, as scientists strive to understand the Earth as a large system of systems, including solid Earth, Oceans, Atmosphere, Biological processes, as well as human activities (Foser 2006). One initiative is the Collaborative Large-scale Engineering Analysis Network for Environmental Research (CLEANER) (National Research Council 2006), which is constructing virtual observatories to measure and understand hydrologic science and engineering. The CLEANER project is developing robust and reusable infrastructure that can be used for multiple projects. Users of the CLEANER infrastructure will need to be able to construct collaborative, interactive, visualizations of the complex data produced, at many temporal and spatial scales. For example, the Corpus Christi bay testbed has models of currents and chemistry, as well as data from a sensor array. Scientists need to be able to visualize the dynamic data about the three dimensional space of the bay at different scales. To exploit this observatory fully — making its data more widely useable, understandable, and accessible by scientists and students—we envision a toolkit that will enable communities to develop virtual worlds to represent a variety of views of the ocean and ocean processes. The worlds might present “realistic” views of the world, synthetic representations of data, and collaborative spaces for virtual organizations. The worlds might represent observation stations, control panels, class rooms, and museum displays. These views would be dynamic, 3D, persistent, interactive, and sharable by large communities. We envision a three dimensional view of the area (e.g., Corpus Christi Bay), presenting model data and measurements in a realistic or symbolic form. Flora, fauna, currents, and instruments could be represented by objects or avatars. Scientists or students can visit using standard commercial technology, e.g., game enabled home computers, perhaps represented as avatars, collaborating through inexpensive chat or voice channels. Alternative “worlds” would provide different views of the area, for different users and purposes. One “world” might be for school children, featuring realistic rendering, historic sequences, and educational scenarios (perhaps with helpful avatars to guide the learners). Science teams might construct other views that cover specific time periods, events or processes, perhaps at speeded up time, or different spatial resolutions. Some alternatives might allow authorized users to directly read and/or control sensors or other assets via avatars. Contemporary online games, such as Second Life (Rosedale, et al), World of Warcraft (worldofwarcraft.com), or SimCity (simcity.ea.com) suggest that this vision can be achieved. However, these systems fall short of what is needed because they are not extendible. B.1.2 Relevance to OCI The proposed work seeks to create the infrastructure that will be the foundation to enable communities, organizations, and individuals to create new interfaces for large scale, collaborative computational environments. These environments will advance many of the goals of the NSF Office of Cyber Infrastructure. The proposed technology is ideally suited to creating multiple low cost “interfaces” for collaboratories. Each “virtual world” may provide “views” of the data appropriate to the users and communities. For example, one virtual world for an environmental observatory might contain tours appropriate for the general public, another for school children, a third for decision makers, and yet another for discipline scientists. These environments would be built from the same toolkit, and will be able to share objects and code. Furthermore, the technology will be shared across disciplines, enabling the creating of common environments for large scale system science. This capability will meet NSF OCI’s goals to realize the full scientific potential of its large facilities and observatories, and will serve as a platform for the development of new, sophisticated applications for scientific research and education. The proposed infrastructure is designed to exploit the power of commercial technology, which will be both cost-effective and accessible to a broad range of users, including the general public and underrepresented groups. In order to reach these goals, it will be necessary to integrate existing open-source software and to develop a suite of open interfaces, free tools, and reference implementations. Together, these tools are the essential infrastructure that must be designed and built. This work requires substantial developments in fundamental computer science, including integration of open technology from a number of sources, to define an open, flexible architecture. These developments will require significant software engineering, the solution of fundamental computer science challenges, and experimentation with large scale networked systems. B.1.3 Relevant Results from Prior NSF Support NSF CNS 05-20182 (2005-2007) (K. Nahrstedt, R. Campbell) grant supports investigations of high-performance streaming protocols for multi-camera tele-immersive environments. In this work we are investigating a multi-stream coordination and adaptation point-to-point protocol framework for 3D multi-camera environments to understand what adaptive models are appropriate for these new environments when connecting to Internet infrastructure. Several important results have been presented in ACM and IEEE conferences. B.1.4 Previous and Related Work This project emerges from the confluence of several important trends which present an important opportunity to transform the Cyberinfrastructure for scientific collaboration and visualization. Contemporary entertainment technologies have driven commercial technology toward extremely high quality graphics and sound systems, which can exploit high bandwidth channels to homes and desktops. Online games have evolved into complex “synthetic worlds” (Castranova 2006), shared, persistent, immersive 3D environments in which people live out fantasy and reality. At the same time, new infrastructure standards and frameworks are being constructed, which enable virtual organizations of many kinds. The next generation of NSF virtual observatories and laboratories will use shared infrastructure to construct large scale, shared, persistent environments for science, engineering, and education. Clearly, virtual observatories share many characteristics and requirements of online games. Furthermore, scientists, engineers, and scholars should be able to use the full power of commercial graphics and multimedia technology at the lowest possible cost. Massively Multiplayer Online Games This work emerges from the technology of “Massively Multiplayer Online Games” (MMOG), such as EverQuest [Everquest], World or Warcraft [Williams, Ducheneaut, et al.], and Second Life [Rosedale, et al]. These Massively Multiplayer Online Games prove the viability and attraction of a persistent, shared, on-line environment, with thousands of users interacting within a three dimensional world, creating, using, and trading objects within complex social organizations. These persistent virtual worlds are inhabited by millions of people (and rudimentary artificial intelligences, AIs, as well), who participate over consumer networks through relatively sophisticated 3D interactive views from their own local machines. These synthetic worlds are the setting for what many see as a whole new way of living, in which people can create new cultures and opportunities digitally (Castranova 2006, Williams 2006, Taylor 2006, Yee 2006)]. These worlds are not simply fantasies or games. They host “massive flows of real human intercourse—information, commerce, war, politics, society, and culture.” (Castranova, p. 1) They are, in fact, just like real cities. Indeed, on one survey 20% of the respondents gave their online world as their primary residence (and many more answered that they would like to do so) (Castranova 2005). The “GDP” of these worlds is at least $100M (Castranova 2006), and a growing number of people earn their living in these environments. Essentially, people are migrating to this new territory for the usual reasons: they believe they will have a better life there. Many companies, including technology companies like IBM, a large number of universities, and other organizations such as the Associated Press have established outposts in these virtual worlds. This technology combines high quality audio and video available through broadband with cheap telephony and text messaging. Just as Web browsers have become the default interface, these environments will soon be nearly universal [26]. Relevance to Scientific Observatories Our reason for exploring the technology for use in on-line scientific observatories is multifold. First, the scientific community is actively seeking collaborative tools and social computing support for science. Using technology that has been successfully used to create active communities, albeit playing games, builds on known, well-documented approaches to introducing collaborative social computing. User interfaces are notoriously difficult to develop and build. Exploring the use of user interface metaphors learnt by users in games as interfaces into the scientific world would simplify adoption and build on known techniques. One example of these metaphors is the user interfaces used to control the location and motion that user’s employ to control their avatars in games. Game engines have adopted a few standardized approaches to integrating graphics, networks, databases, physical models (physics), networking, scripting, and game play. These approaches may be used in the scientific observatories in straightforward, obvious ways. In particular, the notion of physics in games allows us to define how scientist avatars could interact with data and its visualization and this could be used to represent many different aspects of the data model in a tangible manner. For example, thermal gradients can be modeled as slopes that avatars can climb and flows can be modeled as objects and avatars in motion floating in a current. Artificial intelligencies in games control the behavior of autonomous objects and can be used in scientific observatories to simulate the motion of plants, animals, and other phenomena . Limitations We view these games and game-like synthetic worlds as illustrating the directions for all future cyber-environments. However, these commercial synthetic worlds are each constrained in ways that prohibit the full range of activity and interactions we must support. In particular, none of them is scalable in the ways we envision, none of them have the high levels of data and transaction security that are necessary for critical applications, and none of them supports the input, display, and manipulation of large amounts of diverse, timecritical, real-world data, from databases, sensors, imaging systems, etc. Second Life is, perhaps, the closest commercial synthetic world that matches our requirements. However, there are limitations to full-scale scientific visualization in Second Life. For example, the user-defined graphics, avatar, and rendering are based on solid modeling whereas much scientific visualization is based on meshes. Fast interactive games often use tuned mesh renderers augmented with texture and bump maps, again needing some modification for scientific visualization. Second Life includes mechanisms to balance game play by sharing resources specifically to enhance collaborative play like cool down timers on in-world scripts and competition for overall system resources with other users. Virtualized collaborative scientific observatories will place different demands on the system, and these need to be factored into a realistic scientific synthetic world. Games often include cool down timers to limit spamming both inworld and out-world. However, this can interfere with appropriate scientific behavior, interaction, visualization, and simulation. Again, concerns of scientific realism must be factored into these kinds of algorithms. Distinct areas of the Second Life world, known as “sims”, are virtualized on a server without regard for resource allocation requirements and quality of display. Due to this virtualization, bottlenecks can occur when simulating, for example, large numbers of objects or when implementing objects requiring intensive computation. Second Life has good text communication tools but lacks higher bandwidth communication mechanisms like voice chat, high-definition video, or presentation sharing. For scientific purposes, these higherbandwidth multimedia communication schemes would need to be integrated into the system in a way that provides most flexible scientific usage. Visualization of datasets is not a concept in SecondLife. This facility should be combined with dynamic control of the visualization itself (similar to the experience of Google Earth.) Another significant problem with SecondLife is that their algorithm for scaling the world ties land size, land use, and the number of objects on the land to predetermined and invariant computing resources. Finally, the security and privacy of SecondLife is neither comprehensive nor robust and would limit its scientific use because of repeatability, accuracy, and interference. For example, recent viruses in SecondLife indicate data would be insecure and privacy violated. Our goal, therefore, is not to build a better game platform. For the near term that will probably be better handled by narrow purpose game-specific software—indeed a number of companies are already providing such software (multiverse [www.multiverse.net], offset engine [http://en.wikipedia.org/wiki/Offset_Engine], MS XNA [http://msdn2.microsoft.com/enus/xna/default.aspx], etc). Rather, our goal is to open the way for these collaborative communicative technologies to be used for many purposes, including commerce, education, science, and government. Other work, such as Croquet [Croquet Project, Lombardi], under development at the University of Wisconsin, Madison, takes a different approach and focuses on the user interface aspect and three dimensional modeling paradigm. They do not address issues such as security or scalability, nor do they have a design that readily admits massive sensor data inputs such as we propose. Overall, we propose a more robust underlying construction. The NSF and other organizations are collaborating to develop a robust national infrastructure, to support collaborative science and engineering, as well as education and commerce. Currently, this work is developing Web Portal based collaborative environments, which call upon standard, open mechanisms for securely sharing and executing software, storing and managing metadata, social and knowledge engineering, and other services. The virtual worlds that we envision go far beyond the portals of today, bringing immersive, 3D visualization from the laboratory to every desktop. But they can and should be built on the same shared infrastructure, using common standards. B.1.5 Expected Outcomes and Measures of Success This project is the initial effort in this area, so the primary goal is to establish a solid foundation for future work, by integrating off the shelf tools and standard infrastructure. This will take the form of specifications, proposed standards, and reference implementations of critical features. If successful, this project should lead to submissions to appropriate standards tracks, such as the Open Grid Forum. A key outcome will be the release of an integrated toolset for constructing immersive, collaborative virtual worlds for scientific collaboratories and others. The tools will integrate existing free and commercial tools with standard Cyberinfrastructure used by NSF virtual observatories. One measure of success will be incorporation into the NMI build and test facility, which will indicate mature, useful software. This project will seek to grow a broad user community that use and contribute to these tools. We plan to demonstrate and prototype with the CLEANER project, with feedback and evaluation by the CLEANER community. As the work progresses, similar collaborations will be possible with other virtual organizations and collaboratories. Success will be measured by the early adoption of the software, and by inclusion in standard toolsets and future proposals. B.2 Impacts B.2.1 Impact on Science and Engineering Scientists and Engineers have access to ever increasing information technology, networks, data stores, and computational models. These capabilities enable not only new scientific understandings, but also interdisciplinary system science. The NSF’s cyberinfrastructure capitalizes on these advances, using integrated computing, data, and networks to create global virtual observatories and experimental facilities. These facilities will foster and support virtual organizations, built using common components and services, but designed to meet the needs of communities with shared interests. These virtual organizations will need to create sophisticated collaborative environments for these observatories. Today’s Web portals are expensive to produce, and provide only some kinds of user experiences (text, 2D graphics, limited immersion). New approaches are needed to exploit emerging technologies of 3D graphics, high bandwidth networks, and immersive interfaces. Our approach is based on the observations of current commercial technology trends: 3-D graphical processing units, 3-D HDTV displays, inexpensive game consoles, multicore processors, high-bandwidth to the home, and cluster computing. These technologies present tremendous opportunities for creating flexible and diverse collaborations, new forms of understanding, and compelling educational experiences. The success of contemporary online games and virtual worlds offers a model for how to create persistent, shared, immersive environments. We would like to exploit and extend this successful technology and associated metaphors to create high-quality, low cost software accessible to a broad range of users, including the general public and underrepresented groups. In order to reach these goals, it will be necessary to integrate existing open-source software and to develop a suite of open interfaces, free tools, and reference implementations. Together, these tools are the essential infrastructure that must be designed and built. This work requires substantial developments in fundamental computer science, including integration of open technology from a number of sources, to define an open, flexible architecture. These developments will require significant software engineering, the solution of fundamental computer science challenges, and experimentation with large scale networked systems. B.2.2 Broader Impact The proposed technology will have broad impact beyond science and engineering. B.2.2.1 Impact on enterprises This technology has broad uses for many kinds of virtual organizations. The development of standard infrastructure will enable enterprises of all kinds to use as a platform for developing training, global collaboration, and services, and to create new ways of doing business, and, indeed, whole new businesses may emerge. Many companies, including technology companies like IBM, a large number of universities, and other organizations such as the Associated Press have established outposts in these virtual worlds. Development of robust, common infrastructure as envisioned here could enable a virtual “land-rush”, reminiscent of the transformation produced by the WWW. B.2.2.2 Impact on government “Virtual worlds” are well suited for many collaborative activities, including decision support, emergency management, and environmental monitoring. The public sector will benefit from the confluence of commercial developments (e.g., decision support environments) and scientific and technical assets (e.g., environmental observatories). Robust open standards will enable the integration of these developments to improve public services, enhance public participation, and reap the benefits of public investments. B.2.2.3 Impact on Education and Scholarly Communities The availability of a system such as we propose will be of tremendous use to a wide variety of scholarly and educational communities. We expect it to be adopted by most disciplines, creating virtual universities. These environments will be especially interesting for the arts (as a platform for new modes of experiential expression), the social sciences (for experiments in group behavior, economic behavior, etc), as well as sciences in which simulation could benefit from collaborative interaction (such as visualization of complex datasets, aggregation and visualization of data from sensors, geographical information systems, etc.). This project grows out of the Cultural Computing Program [www.culturalcomputing.uiuc.edu] at the Siebel Center for Computer Science, in partnership with the new media program in Art and Design, the School of Music, Dept of Dance, Department of Theater, Krannert Center, Krannert Art Museum, as well as with partners in the sciences and NCSA and Beckman. This activity has also attracted the interest of librarians and archivists, who are investigating methods for preserving this new cultural heritage [Lowood]. B.2.2.4 Fundamental social and Behavioral Research These environments are far more than fun and games. In fact, we already see many important and “serious” reasons to investigate and use these environments. These environments represent a new way of life for many people. We are collaborating with scholars at UIUC and other institutions who want to be able to study these environments and their use. This includes studies of social interaction and social networks [Williams 2006, “A (Brief)…”], economics [Castranova 2005], as well as technical developments in graphics, networking, and content management. These studies require the ability to measure and manipulate the system, and to conduct controlled experiments. The tools we create will also be useful for social science work: we are collaborating with a group at UIUC to use them as a common platform for the study of group interactions among disaster response teams (see supporting letter from Poole). B.2.2.5 Environments for “Everyday” Creativity The current generation of online games have demonstrated that users can and should create the “content” of these worlds, indeed, some argue that this is the only feasible way to build such complex environments (McCahill, 2004; Ondrejka, 2005; Rosedale, 2005). This creativity sets these environments apart from conventional media, even some so-called interactive media. To successfully provide this creative environment, the underlying system must be secure and robust, but also “infinitely” extensible and open. Clearly, it is difficult to create a system that is completely extensible, yet simple to use, and also difficult to break! For this reason, we must focus on modular, pluggable designs, with excellent toolkits for users of varying skills and backgrounds. B.3 Need for Cyberinfrastructure Development B.3.1 Approach This project ultimately requires integration of open technology from a number of sources, to define an open, flexible framework. • Pluggable framework to Integrate existing open source • Fill in missing pieces • Design interfaces, protocols, data representations • Create reference implementations • Tools for creating worlds and content Many of the technical challenges are similar or identical to those faced by conventional Web Portals. Consequently, this work will build upon previous and existing infrastructure, including middleware for security, data management, work flow, and collaboration. However, we will provide radically better user interfaces and interactions—more like a computer game than a Web site. For this reason, new software and interfaces are needed. B.3.2 Technical Challenges The system must include: • Persistent store capable of managing billions of objects, with millions of simultaneous users, propagating millions of updates per second • Distributed architecture capable of providing real time (interactive) views for millions of systems • Real security • Ability to plug in alternative implementation, new graphics, etc. • Tools to construct worlds and artifacts • Mechanisms to exchange objects between worlds. From our experience, we identify the following important requirements for our virtual worlds: 1) They should be open source and open, modular, and extensible in design. 2) They should allow for a wide range of operating conditions, from single user to massive communities of potentially simultaneous users. 3) They should allow experimentation from both developers and from a general public perspective. 4) They must support tracking and reporting facilities so that they can be used for experimentation. 5) They must provide clean experimental environments – i.e., not ones messed up by adverts, roving lunatics, or hackers. 6) They must offer reliability, security and privacy if experimental results are to be valid. They need a very robust security model. 7) They must accommodate a variety of talents at creating and manipulating content: the target user base is very broad, including scientists, business people, artists, teachers, and students of all ages. 8) They must support repeatable experiments. 9) They should have a framework for measurements. 10) They should allow easy insertion of different or new representation engines (such as graphics renderers) and new and extendable data bases, etc. 11) They must have a robust, reliable, repeatable structure for action sequences and scripts. 12) There should be excellent debugging tools and scripting tools. 13) As much as possible, the tools should be hosted within the environment and accommodate collaborative users. That is, while viewing and manipulating data or moving through representations, you should be able to create new representations or processes without restarting or leaving the system. B.4 Project Plan This project builds on our ongoing studies of contemporary game technology, collaborations with Cyberinfrastructure development projects, and collaboration with potential user communities. The overall goal is to push forward the development of an open set of standards and reference software to integrate free software from game technology with Cyberinfrastructure developed for NSF science and engineering, to produce a standard, shared infrastructure for many user communities to create and interoperate shared environments. The approach is to develop this framework in collaboration with one or more science communities, to provide both design guidance and frequent evaluation of the work. To start with we will work with the CLEANER group, a group in the social sciences (Poole, et al), and will consult and coordinate with IBM in the enterprise sector. This project also seeks to create a suite of tools for academic research by creating an open and flexible environment for experimentation. As much as possible, the tools should be usable and accessible from within a virtual world by simultaneous multiple users. This work requires substantial developments in fundamental computer science, including integration of open technology from a number of sources, to define an open, flexible architecture. This must include: • Persistent store capable of managing billions of objects, with millions of simultaneous users, propagating millions of updates per second • Distributed architecture capable of providing real time (interactive) views for millions of systems • Multilayer active security • Ability to plug in alternative implementation modules, new graphics, new physics, etc. • Tools to construct worlds and artifacts • Mechanisms to exchange objects between worlds. B.4.1 Approach to Solving Technical Challenges The difficulties of developing a platform for mWorlds from open source synthetic worlds and existing scientific software were described in a previous section. Below we address our approach to these problems. Our approach to creating a scalable system uses computer science distributed peer to peer processing augmented with access to the Teragrid. To avoid reinventing large amounts of scientific software and infrastructure, we leverage the Teragrid and the National CyberInfrastructure in our proposed system. We will develop appropriate conversion modules to stream and access software from scientific experiments and simulations into the mWorld environment, matching the scientific data formats to appropriate mWorld data representations that can be displayed with “game-like” realism. This will require us to authenticate mWorld users to the Grid and secure our mWorld interactions using Grid mechanisms. We will examine and develop modifications for scientific visualization that can be mapped to interactive game technology such as tuned mesh, texture and bump maps renderers. mWorlds will incorporate a multi-level, distributed system scheduler that integrates with the server clients used in the game world and the Teragrid. Scheduling algorithms, data staging, and other techniques will be developed and/or integrated to offer game play-like experiences specifically enhanced for collaborative science. Virtualized collaborative scientific observatories will increase the demands on both mWorlds and the Teragrid. The scheduling must interact with load-balancing, network quality of service, and large dataset sizes requiring an integration of techniques. To solve the problems of rendering worlds that have varying complexities of objects in differing densities that challenge easy scaling solutions, we will use peer to peer distributed technologies to exploit workstations and clustered and networked computing resources. Viruses and spamming are a concern of the modern world. Since mWorlds should, to be effective, reach into the public networks, our project must consider how to secure both the scientific and mWorld resources and data. We expect our particular form of computation to raise challenges in this area for which there are no ready solutions. We will design the security mechanisms to interoperate with Teragrid and the mWorlds client server interactions while providing appropriate scientific behavior, interaction, visualization, and simulation. mWorlds will include higher bandwidth communication mechanisms like voice chat, highdefinition video, and presentation sharing. These higher-bandwidth multimedia communication schemes will be integrated into the system using quality of service provisions, soft real-time scheduling, and load balancing including peer-to-peer distributed processing approaches. Visualization of datasets will be combined with dynamic control of the visualization itself and integrated into the rendering and user interfaces of mWorlds. We also plan to interface to sensor input and actuator output through streaming data and publish/subscribe mechanisms B.4.2 Proposed activities Year 1: Design, Initial Mockup In year 1, there will be a linked set of design, prototyping, and software evaluation. Task: complete evaluation of existing free software In current work, we are assessing existing free software (game engines, graphics, scripting, etc.). We will complete this assessment and select a portfolio of free software to reuse. Outcome: white paper comparison of existing software. Task: Define interactions with standard Cyberinfrastructure Given an understanding of the existing software tools and emerging Cyberinfrastructure, we will define interactions and data interfaces. For example, virtual worlds require persistent data, which should be built on standard naming, discovery, and data interchange standards. These interfaces must be designed. Outcome: design document defining interactions and interfaces. Task: complete science use cases and scenarios In current work, we are working with the CLEANER project (National Research Council 2006) to define scenarios and use cases for their virtual observatory. We will complete a white paper describing key scenarios and the implications for design. Outcome: white paper Task: mock up for review A key deliverable will be a mock up of a proposed system, presented to collaborators from science and technology. We will host or participate in several workshops to present and discuss our technology. Outcome: workshops, report Workshops: We have identified three potential workshops, which will be used to establish a community of interested uses and developers: • Workshop with games industry, “gaming the world.” We have budgeted this for year one. • Workshop with technologists, such as IBM and NSF CI providers. We will work with collaborators to sponsor this. • Workshop at science meetings, such as CLEANER all hands. Year 2: focus on content and content creation Task: set up working testbed based on the mock up Implement enough functionality to run a testbed, built on CI standards, using selected science scenarios, test data, etc. Outcome: a working prototype, suitable for experiments Task: create example content The testbed will be populated with example content, e.g., representations of the science data of interest, avatars, and so on. This task will require development of powerful, flexible, and easy to use editors and scripting language. Outcome: a working science scenario Task: Incorporate science content Implement interfaces and example cases of science content: sensor data, model output, model steering, collaborations. The initial targets will be data and models from the CLEANER project. Outcome: definition of interfaces and protocols, example implementations Task: Alpha release Alpha level release to community. Outcome: software integrated in NMI Test and Build, basic documentation and packaging completed, release to public. Year 3: Evaluation and hardening Task: Intensive evaluation studies Conduct user evaluations of the alpha, including domain scientists and others. In parallel, conduct technical review of the role and interoperation with Cyberinfrastructure. Outcome: Evaluation report, recommendations for revisions Task: Improve and Harden Software In light of workshop and evaluations, implement key improvements. Harden the software, e.g., additional testing, documentation, etc. Outcome: Beta level software B.4.3 Project Organization This project brings together a team from computer science, the games industry, Cyberinfrastructure development, and working scientists. PI Garnett, head of the Cultural Computing Initiative at UIUC, brings expertise on the serious uses of game technology, broad knowledge of the games industry. PI Campbell brings deep expertise in system design, software engineering, and computer technology trends. Senior Personnel McGrath brings extensive experience in Cyberinfrastructure development and deployment. Co-PIs will coordinate the project and lead the community building. McGrath will lead the software development efforts, working with students and other professionals. We will also coordinate with science partner, Minsker (see attached letter of support), PI of the CLEANER project, a flagship NSF observatory project. B.4.4 Cyberresources at NCSA B.4.5 Department of Computer Science, UIUC Campus The Department of Computer Science at UIUC will provide space and technical support services, budgeted per user. B.4.6 Potential Collaborators Minsker (see attached letter of support), PI of the CLEANER project, a flagship NSF observatory project Sandra Kearney, Director, IBM Emerging 3D Internet and Virtual Business. Marshall Scott Poole, Professor, UIUC department of Speech Communications, and Senior Research Scientist, NCSA. C References National Research Council, CLEANER and NSF's Environmental Observatories, National Academies Press, Washington , 2006. Castranova, Edward, Synthetic Worlds: The Business and Culture of Online Games, Chicago, university of Chicago, 2006. Chambers, Chris, Wu-chi Feng, Wu-cheng Feng, and Debanjan Saha. A Geographic Redirection Service for On-line Games. In ACM Multimedia 2003, 2003. Croquet Project. Croquet. 2006, http://www.opencroquet.org. Ducheneaut, Nicolas, Nicholas Yee, Eric Nickell, and Robert J. Moore. "Alone together?" Exploring the Social Dynamics of Massively Multiplayer Online Games. In CHI 2006, 2006, 407-416. Everquest, Sony Online Entertainment. 1999. Foundataion, Acceleration Studies, Metaverse Roadmap: Pathways to the 3D Web. Menlo Park, SRI International, Menlo Park, 2006. Lombardi, Julian and Mark P. McCahill. Enabling Social Dimensions of Learning Through a Persistent, Unified, Massively Multi-User, and Self-Organizing Virtual Environment. In The Second Conference on Creating, Connecting and Collaborating through Computing (C5’04), 2004. McCahill, Mark P. and Julian Lombardi. Design for an Extensible Croquet-Based Framework to Deliver a Persistent, Unified, Massively Multi-User, and Self-Organizing Virtual Environment. In Second Conference on Creating, Connecting, and Collaborating through Computing (C5 ’04), 2004. multiverse. Multiverse. 2006, http://multiverse.net. National Research Council, National Academies, “CLEANER and NSF’S Environmental Observatories. National Academies Press, Washington, D.C., 2006. NCSA. NCSA: Cyberinfrastructure for science and engineering. 2006, http://www.ncsa.uiuc.edu/AboutUs/mission.html. Ondrejka, Cory, Power by the People: User-Creation in Online Games, in Massively Multiplayer Game Development 2, t. Alexander, Editor. Charles River Media, Inc., Hinghan, MA, 2005, 57-84. Rosedale, Phillip and Cory Ondrejka. Enabling Player-Created Online Worlds with Grid Computing and Streaming. 2003, http://www.gamasutra.com/resource_guide/20030916/rosedale_01.shtml. Stephenson, Neal, Snow Crash, New York, Bantam Spectra, 1992. Taylor, T. L., Play Between Worlds: Exploring online Game Culture, Cambridge, The MIT Press, 2006. The Daedalus Project. The Daedalus Project: The Psychology of MMORPGS. 2006, http://www.nickyee.com/daedalus/. Wiliams, Dimitri, Nicholas Ducheneaut, Li Xiong, Yuanyuan Zhang, Nick Yee, and Eric nickell, From tree house to barracks: The social life of guilds in World of Warcraft (to appear). Games and Culture, 2006. Williams, Dimitri, A (Brief) Social History of Video Games, in Playing Computer Games: Motives, Responses, and Consequences, P. Vorderer and J. Bryant, Editors. Lawrence Erlbaum, Mahwah, NJ, 2006. Williams, Dimitri, On and off the 'net: Scales for social capital in an online era. Journal of ComputerMediated Communication, 11 (2): article 11., 2006. Williams, Dimitri, Virtual Cultivation: Online Worlds, Offline Perceptions. Journal of Communication, 56 (1):69-87, 2006. Williams, Dimitri, Williams, D. (2007, in press). The Impact of Time Online: Social Capital and Cyberbalkanization (in Press). CyberPsychology & Behavior., 2007. Yee, Nicholas, The Psychology of Massively multi-User Online Role-Playing Games: Motivations, Emotional Investment, Relationships and Problematic Usage (to appear), in Avatars at Work and Play: Collaboration and interaction in Shared Virtual Environments, R. Shroder and A. Axelsson, Editors. Springer-Verlag, London, 2006. Yee, Nick, The Demographics, Motivations and Derived Experiences of Users of Massively multiUser Online Graphical Environments. PRESENCE: Teleoperators and Virtual Environments, 15:309-329, 2006. Yee, Nick, The Labor of Fun: How Video Games Blur the Boundaries of Work and Play. Games and Culture, 1 (1):68-71, January 2006. Yee, Nick, Jeremey N. Bailenson, Mark Urbanek, Francis Chang, and Dan Merget, The Unbearable Likeness of Being Digital: The Persistence of Nonverbal Social Norms in Online Virtual Environments (in press). CyberPsychology and Behavior, 2006