Contents | Zoom in | Zoom out For navigation instructions please click here Search Issue | Next Page
Virtual World
Architectures
Interactive Television
Routing in the Cloud
Emerging Collectives
Contents | Zoom in | Zoom out For navigation instructions please click here Search Issue | Next Page
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
_________________________________
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
_____________________
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
S E P T E M B E R / O C T O B E R 2 011, V O L U M E 15 , N U M B E R 5
ALSO IN THIS ISSUE View from the Cloud
Access Control 72 Routers for the Cloud: Can the Internet
62 A User-Activity-Centric Framework Achieve 5-Nines Availability?
Andrei Agapi, Ken Birman, Robert Broberg, Chase Cotton,
for Access Control in Online Thilo Kielmann, Martin Millnert, Rick Payne, Robert Surton,
Social Networks and Robbert van Renesse
Jaehong Park, Ravi Sandhu, and Yuan Cheng
Standards
78 Inside the Identity Management Game
DEPARTMENTS Lucy Lynch
News & Trends Beyond Wires
7 Wi-Fi Making Big New Waves: “In-Room” 83 When the Shift Hits the (Television)
High-Speed Uses to Get Big Boost from Fan: A Growing Opportunity
Wireless Mainstay for Companion Devices
Greg Goth
Nitya Narasimhan
Web-Scale Workflow
66 Principles of Elastic Processes COLUMNS
Schahram Dustdar, Yike Guo, Benjamin Satzger,
and Hong-Linh Truong
From the Editors
4 Adversarial Machine Learning
J.D. Tygar
The Functional Web
87 Scala Web Frameworks: Looking Beyond Lift
www.computer.org/internet/ Dean Wampler
This publication is indexed by ISI (Institute for Scientific Information) in SciSearch, Research Alert,
the CompuMath Citation Index, and Current Contents/Engineering, Computing, and Technology.
Postmaster: Send undelivered copies and address changes to IEEE Internet Computing, IEEE Service
Practical Security
Center, 445 Hoes Ln., Piscataway, NJ 08855-1331. Periodicals postage paid at New York, NY, and at
additional mailing offices. Canadian GST #125634188. Canada Post Publications Mail Agreement
Number 40013885. Return undeliverable Canadian addresses to PO Box 122, Niagara Falls, ON L2E
95 Not Reinventing PKI until We Have
6S8. Printed in the USA. Circulation: IEEE Internet Computing (ISSN 1089-7801) is published bimonthly
by the IEEE Computer Society. IEEE headquarters: 3 Park Avenue, 17th Floor, New York, NY 10016-5997. Something Better
IEEE Computer Society headquarters: 1828 L St. N.W., Suite 1202, Washington, D.C. 20036-5104. IEEE
Computer Society Publications Office: 10662 Los Vaqueros Circle, PO Box 3014, Los Alamitos, Calif. Stephen Farrell
90720; (714) 821-8380; fax (714) 821-4010. Subscription rates: IEEE Computer Society members get
the lowest rates and choice of media option — US$48/1,300 for member/nonmember institutional
print + online. For information on other prices or to order, go to www.computer.org/subscribe. Back
issues: $20 for members, $173 for nonmembers. Reuse Rights and Reprint Permissions: Educational Peering
or personal use of this material is permitted without fee, provided such use: 1) is not made for profit;
2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does
not imply IEEE endorsement of any third-party products or services. Authors and their companies are 99 Emergent Collectives
permitted to post the accepted version of their IEEE-copyrighted material on their own Web servers
without permission, provided that the IEEE copyright notice and a full citation to the original work Charles Petrie
appear on the first screen of the posted copy. An accepted manuscript is a version which has been
revised by the author to incorporate review suggestions, but not the published version with copy-
editing, proofreading, and formatting added by IEEE. For more information, please go to: http://
_____________________
www.ieee.org/publications_standards/publications/rights/
__
Backspace
________
paperversionpolicy.html. Permission to reprint/republish this
material for commercial, advertising, or promotional purposes
or for creating new collective works for resale or redistribution
104 The Battle for Internet Openness
must be obtained from IEEE by writing to the IEEE Intellectual
Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854- Vinton G. Cerf
_________
4141 or pubs-permissions@ieee.org. Copyright © 2011 IEEE.
All rights reserved. Abstracting and Library Use:
Abstracting is permitted with credit to the source.
Libraries are permitted to photocopy for private use
of patrons, provided the per-copy fee indicated in the
code at the bottom of the first page is paid through
6 Advertiser Index
______ the Copyright Clearance Center, 222 Rosewood Drive, 14, 102 Calls for Papers
Danvers, MA 01923.
71 IEEE Computer Society Info
E N G I N E E R I N G A N D A P P LY I N G T H E I N T E R N E T
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Three-dimensional
About This Issue:
virtual worlds promise to
complement the Web with
3D models of virtual places
Asynchronous middleware
that are fanciful or that
is playing an increasingly
model and mirror the real
important role in distributed
world. To see how this will
and Web-based systems. This
come about, we must address
issue’s theme articles identify
current limitations engineering
some research and of virtual
worlds, deconstruct and
challenges that remain before
study their architectures,
this technology can fully and
consider how to evolve them
make good on its promises.
to realize their promises.
Cover by Randy Lyhus, www.randylyhus.com
VIRTUAL WORLD ARCHITECTURES
11 Guest Editor’s Introduction 46 Virtual and Real-World Ontology Services
Craig W. Thompson Joshua D. Eno and Craig W. Thompson
15 Extending Web Browsers with a Unity 53 Accuracy in 3D Virtual Worlds
3D-Based Virtual Worlds Viewer Applications: Interactive 3D Modeling
Neil Katz, Thomas Cook, and Robert Smart of the Refractory Linings of Copper
Smelters
22 Hypergrid: Architecture and Protocol Anthony J. Rigby, Kenneth Rigby, and Mark Melaney
for Virtual World Interoperability
Cristina Videira Lopes 56 I-Room: Augmenting Virtual Worlds
with Intelligent Systems
30 An Entity-Component Model Austin Tate
for Extensible Virtual Worlds
Toni Alatalo
38 Open Wonderland: An Extensible
Virtual World Architecture For more information on these or any other computing
topics, please visit the IEEE Computer Society Digital
Jonathan Kaplan and Nicole Yankelovich Library at www.computer.org/publications/dlib.
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
From the Editors
Adversarial Machine Learning
University of California, Berkeley
M
achine learning would seem to be a powerful apparent tricky spam email messages that my
technology for Internet computer secu- colleagues and I have collected (complete with
rity. If machines can learn when a system original spelling and punctuation):
is functioning normally and when it is under
attack, then we can build mechanisms that “what, is he coming home, and without poor
automatically and rapidly respond to emerging lydia?” she cried. “sure he will not leave
attacks. Such a system might be able to auto- London
matically screen out a wide variety of spam, “i am quite sorry, lizzy, that you should be
phishing, net work intr usions, malware, and forced to have that disagreeable man all to
other nasty Internet behavior. But the actual yourself.
deployment of machine learning in computer calvert dawson blockage card. coercion cho-
security has been less successful than we might reograph asparagine bonnet contrast bloop.
hope. What accounts for the difference? coextensive bodybuild bastion chalkboard
denominate clare churchgo compote act.
Tricking Machine Learning Systems childhood ardent brethren commercial com-
To understand the issues, let’s look more closely plain concerto depressor.
at what happens when we use machine learn- brocade crown bethought chimney. angelo
ing. In one popular model, supervised learn- asphyxiate brad abase decompression code-
ing, we train a system using labeled data — for break. crankcase big conjuncture chit conten-
example, in a spam email detector, we would tion acorn cpa bladderwort chick. cinematic
label a set of training email messages as spam agleam chemisorb brothel choir conformance
or ham (although it doesn’t sound very kosher, airfield.
“ham” is a term used to denote non-spam email).
The machine learning algorithm then produces What is going on here? The first two frag-
a classifier, which takes unlabeled email mes- ments are quotes from Jane Austen’s Pride and
sages as input, then classifies them as likely Prejudice. The second two messages are lists of
spam or ham. During training, a classifier is less-common words in English. These tricky
likely to learn that terms such as “Viagra” or spam messages poison the training set. When
“V1@gr@,” for example, are a strong indicator they’re labeled as spam and fed to a machine
of likely spam. learning algorithm, they dilute the quality
Good mach i ne lea r n i ng a lgor it h m s a re of spam detection. The algorithm could infer
designed to perform well even if they get some a rule that a benign term (such as “Lydia,”
random badly labeled input (such as a spam “London,” “brethren,” or “chimney”) is actually
message that’s accidentally mislabeled as ham). a marker for spam. When the classifier begins to
However, in the context of computer security, label its inputs, it will generate false positives:
this does not go far enough. Adversaries (in this ham that is incorrectly marked as spam. Large
case, spammers) might play dirty by creating numbers of false positives undermine users’
an adversarial training set: instead of sending confidence in the learning algorithm. In prac-
“normal” spam, they might send (Byzantine) tice, users find that their spam detectors seem
“tricky” spam designed to make the classifier tone-deaf and often misclassify email, requir-
misbehave. Here are some fragments from some ing them to constantly check their “likely spam”
4 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Adversarial Machine Learning
mailboxes to manually retrieve mis- Hardening Machine Learning the adversar y to control a much
classified ham. These examples highlight the fail- larger fraction of the input to mis-
Other types of attacks are also ings of classical machine learning. train the classifier.
possible. For example, in systems The good news is that a new science The search for adversarial machine
that continually retrain, an adver- of adversarial machine learning is learning algorithms is thrilling: it
sar y might tr y a “boiling-frog” emerging — the development of algo- combines the best work in robust
attack. (Legend has it that if you drop rithms that are effective even when statistics, machine lear ning, and
a frog in a boiling pot of water, it adversaries play dirty. computer security. One significant
will quickly jump out; but if you put My colleagues and I at UC tool security researchers use is the
a frog in lukewarm water and then Berkeley — as well as other research ability to look at attack scenarios
slowly raise the heat, the frog can- teams around the world — have from the adversary’s perspective (the
not detect the slow change and will been looking at these problems and black hat approach), and in that way,
ultimately be boiled.) Consider using developing new machine learning show the limits of computer security
machine learning to detect abnormal algorithms that are robust against techniques. In the field of adversar-
network traffic. In a boiling-frog adversarial input. One technique ial machine learning, this approach
attack, an adversary slowly intro- that we’ve used with great success is yields fundamental insights. Even
duces aberrant input, and the system Reject On Negative Impact (RONI). though a growing number of adver-
learns to tolerate it. Ultimately, the In RONI, we screen training input to sa r ia l mac h i ne lea r n i ng a lgo -
classifier learns to tolerate more and make sure that no single input sub- rithms are available, the black hat
more aberrant input, until the adver- stantially changes our classifier’s approach shows us that there are
sary can launch a full-scale attack behavior. This has a cost (we need a some t heoret ica l lim it s to t heir
without detection. larger training set), but it also forces effectiveness.
___________ Technical cosponsor:
________
EIC emeritus
CS Magazine Operations Committee
Editor in Chief
__________
IEEE Internet Computing
Associate Editors in Chief
________
____________
_______ CS Publications Board
Editorial Board
_________
_____________
_______ IEEE
________ Internet Computing
___________
Staff
______________
____________ __________
___________
__________ ___________
__________________
_________ ___________
IEEE Internet Computing
_______ ________________
____________ ___
___________
______________ ___________
___________ ______________
_________ _______________
__________
__________ ____________
______________ ___
___________ ____________
______
________ __________
___________ __
_______
_______ _______
__________ ____________________
SEPTEMBER/OCTOBER 2011 5
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
From the Editors
One powerful family of results target message “positive” because hat attack? It turns out that for an
t h at c ome f r om t he blac k h at the classifier will give it a positive important type of classifier, known
approach is called near-optimal eva- classification as spam. At the other as convex classifiers, we cannot stop
sion. We start by “thinking like a end, we find some message that’s it. A spammer’s binary search strat-
spammer.” Suppose we want to sell completely benign and that avoids egy is simply too strong. This shows
Viagra via unsolicited email. If we detection as spam. We call this our the boundaries of the underlying
try a direct approach, we’re certain “negat ive” instance (because t he theoretical limits of what is possible
to have our email automatically clas- classifier returns a negative result: it in adversarial machine learning. To
sified as spam. So, we’ll try to avoid is not spam). So now we have two get beyond them, we will either need
this by modifying our message. For extremes. We can perform a type of to make our systems more compli-
example, instead of using an email binary search — finding intermedi- cated (going beyond convex classi-
subject line such as “Cheap Online ate messages bet ween t hese t wo fiers) or use a fundamentally new
Pharmacy,” we can try a subject extremes. When we get two messages strategy that no longer depends as
line that promises instead a “Moder- that are close to each other — one much on machine learning.
ate Online Apothecary.” We assume classified as spam, the other classi-
that we have sufficient access to a
spam detector that we can pre-test
our messages to see whether they’re
fied as ham — we know we are near
the classifier’s boundary. We can
send the message that is classified
A lthough some of the questions
in this field have a theoretical
flavor, at the end of the day, this is
classified as spam. First, we identify as ham, and we say that it is “nearly not a theoretical field. We need real-
our positive target spam message optimal” but evades detection. world machine learning algorithms
hawking Viagra. We cannot send Now, we turn the tables again that perform well even in adver-
this message because it is certain to and resume the role of defender. We sar ial env ironments. A nd while
be identified as spam. We call our naturally ask: Can we stop this black various research groups around the
world are hard at work developing
powerful adversarial machine learn-
ing algorithms, more work is needed
before machine learning can fulfill
its full promise in improving our
Advertising Personnel cybersecurity algorithms. To find
Marian Anderson: Sr. Advertising Coordinator out more about the field and the
Email: manderson@computer.org
_______________ examples I mention, visit http://radlab.
Phone: +1 714 816 2139 | Fax: +1 714 821 4010
cs.berkeley.edu/wiki/SecML.
____________________
Sandy Brown: Sr. Business Development Mgr.
Email: sbrown@computer.org
_____________ Acknowledgments
Phone: +1 714 816 2144 | Fax: +1 714 821 4010 The work I mention is joint research with a
number of researchers listed at http://radlab.
IEEE Computer Society
10662 Los Vaqueros Circle cs.berkeley.edu/wiki/SecML. I would espe-
________________
Los Alamitos, CA 90720 USA cially like to acknowledge my collabora-
www.computer.org tors Marco Barreno, Anthony Joseph, Ling
Huang, Blaine Nelson, Benjamin Rubinstein,
Advertising Sales Representatives (Display)
and Satish Rao.
Western US/Pacific/Far East: Eric Kincaid
e.kincaid@computer.org
Email: ______________
Phone: +1 214 673 3742; Fax: +1 888 886 8599 J.D. Tygar is a professor at the University
of California, Berkeley, in the Electri-
Eastern US/Europe/Middle East: Ann & David Schissler cal Engineering and Computer Sciences
a.schissler@computer.org, d.schissler@computer.org
Email: ______________ _______________ Department and the School of Infor-
Phone: +1 508 394 4026; Fax: +1 508 394 4926
mation. His research focuses on com-
puter security. Contact him at tygar@
____
Advertising Sales Representatives (Classified Line/Jobs Board)
cs.berkeley.edu.
_________
Greg Barbash
Email: g.barbash@computer.org
______________
Phone: +1 914 944 0940 Selected CS articles and columns
are also available for free at http://
____
ComputingNow.computer.org.
__________________
6 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
News & Trends
Wi-Fi Making Big
New Waves
“In-Room” High-Speed Uses to Get
Big Boost from Wireless Mainstay
Greg Goth
T
echnology industry veterans might remember he’s pleased with the progress the TG has made
the contentious speculation that abounded at since its January 2009 inception.
the end of the 1990s over which short-range “I’m happy with the IEEE timeline,” Perahia
wireless technology would emerge supreme to says. “Perhaps for the first time in .11 history,
carry data over radio. Two of the most heavily a TG will actually finish on time. We’re saying
touted wireless technologies were Bluetooth and we’ll be ready for the sponsor ballot in Decem-
HomeRF. ber, and we’re right on track for that, and there’s
Eventually, of course, IEEE 802.11 technol- been no controversy or anything. And no sur-
ogy convincingly eclipsed both the presumed prises are lurking, hopefully. This has been the
frontrunners, to the point that “free Wi-Fi” is schedule from day one of the TG and has not
a selling point in locales as varied as coffee been modified.”
houses, hostelries, and even barber shops world- What might be most surprising, and hearten-
wide. 802.11 technology, whether it’s in the ing, to those involved in the work on 802.11ad
2.4-GHz b and g bands or the 2.4- and 5-GHz n is that it has proceeded in parallel with another
technology, is the de facto and assumed technol- 60-GHz 802.11 proposal from a consortium
ogy for wireless data networks everywhere. called the Wireless Gigabit Alliance (http://____
Several new updates to the Wi-Fi family are wirelessgigabitalliance.org). The two groups’
____________________
about to emerge, introducing extremely high proposals were similar, but not identical, upon
throughput rates and direct node-to-node data inception, bringing back memories of the most
transfer. From the wireless router sitting in the recent Wi-Fi imbroglio that surrounded 802.11n
living room to the flat-screen TV in the home standardization. 11n was the version of Wi-Fi
theater, Wi-Fi stands poised to become the pre- intended to convincingly supplant the data
ferred end-to-end technology for the uncabled rates supported by the older b and g bands (and
environment. In addition, the latest introduc- to some extent, the a band, which never gar-
tions might also serve as an object lesson to nered the market popularity of the other two).
other standards groups in bringing a technology Originally proposed in 2002, 802.11n didn’t
to market quickly and efficiently. receive final ratification until September 2009,
although the Wi-Fi Alliance began certify-
“We’re on Track” ing “pre-n” products based on the final pre-
Very high throughput Wi-Fi, capable of data ratification stable draft of the technology in June
transfer rates of up to 7 gigabits per second — 2007. Veterans of the 11n battle didn’t want to
or 10 times the rate currently available on the go through another such delay with the 60-GHz
fastest 802.11n networks — is emerging from standard.
the standards process, and products should be “I think what we learned from n is not to split
arriving by mid-2012, according to those work- up the chip vendors into two different proposal
ing on the technology. teams,” Perahia says. “When the chip vendors
Eldad Perahia, chairman of the IEEE 802.11ad split camps it gets really hard — it fundamen-
Task Group, charged with writing the standard tally fractures the silicon vendors, which frac-
for very high throughput Wi-Fi at 60 GHz, says tures the market.”
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 7
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
News & Trends
News in Brief
Even as concern and outcry from all Perahia was active in the 802.11n to act as a mediator between the
sides mounted, the US House Judi- work as well as ongoing work in WiGig and IEEE groups.
ciary Committee voted in July to 802.11ac, a 5-GHz technology that “Things came into line more or
recommend passage of H.R. 1981, improves on 802.11n. The 5-GHz and less organically,” she says. “We had
which includes a mandatory data- 60-GHz groups took different routes a liaison agreement with the WiGig
retention provision that requires to avoid fracturing the silicon ven- Alliance, and of course we’ve had
ISPs to stockpile customer infor- dors but arrived at the same spot — a a longstanding relationship with the
mation — including website visits more or less consensus approach. IEEE, and our position always was
and online postings — for a full year. “802.11ad used the WiGig Alli- [that] we’re going to certify the
The bill, which aims to combat child ance to facilitate the standard,” he 60-GHz tech that makes sense. So
pornography, has generated intense says. “There were two proposals, now here we are, and it’s a moot
opposition among organizations com- the WiGig Alliance’s and another, question. It seems like things are on
mitted to free speech and privacy but all the chip vendors were in the the same track now.”
rights. The Electronic Frontier WiGig proposal. In 802.11ac, we Grodzinsky says both the WiGig
Foundation (EFF) and 29 other civil went the route of specifying frame- A lliance, which published ver-
liberty and privacy groups sent a letter work development, then developing sion 1.1 of its 60-GHz specification
to the committee, condemning the bill the spec based on that; there were no in June, and the Wi-Fi Alliance plan
as a “direct assault” on Internet users’ proposals. In both ways, we avoided to begin certifying 60-GHz products
privacy. In a recent blog, the Ameri- what I thought was the crux of the by the middle of 2012, and products
can Civil Liberties Union noted matter in 11n — two camps in which should be available about the same
that if the bill becomes law, “Respect the chip vendors were split.” time.
for your anonymity online would be a “Usually, you see products avail-
thing of the past.” “A Hundred Groups” able when the programs are ready to
The bill’s text and current status Of course, as the Wi-Fi technology launch,” he says, “because they won’t
is at www.govtrack.us/congress/bill. and brand advances with a prolifera- launch if there’s no product.”
xpd?bill=h112-1981.
___________ tion of letter suffixes, and vendors
The EFF’s information page is at form adjunct consortia to augment What 60-GHz Wi-Fi Does
www.eff.org/deeplinks/2011/07/house standards creation and marketing According to Perahia, the IEEE and
-committee-approves-bill-mandating
________________________ efforts, the possibility for widespread WiGig 60-GHz specifications are
-internet.
______ confusion about which Wi-Fi tech- nearly identical, except for a few
nology does what presents itself. optional features in the IEEE tech-
The Open Cloud Initiative — “Between what’s going on at the nology; the most prominent of these
originally scheduled for a 2010 take- IEEE, the WiGig Alliance, and the is a device-to-device relay mecha-
off — was officially launched at July’s Wi-Fi Alliance, you’d think there are nism, which he terms a “minimalist
2011 Open Source Convention in a hundred groups going in different mesh.”
Portland, Oregon. The organization’s directions,” says Mark Grodzinsky, Grodzinsky says the addition of
goal is to create a legal framework marketing work group chairman for the optional features in the IEEE
for cloud computing providers and the WiGig Alliance, “but when you specif ication is unlikely to delay
users based on open cloud require- dig in and start looking at names, widespread market adoption because
ments as spelled out in the Open you’ll see a common set of people “the common denominator is when
Cloud Principles. The OCP man- and companies — in a lot of cases, you look at what the Wi-Fi Alliance
dates interoperability, open formats the exact same people.” is going to certif y, 802.11ad and
and interfaces, and free user move- Grodzinsky, for instance, has been WiGig are exactly identical, because
ment among systems. working on Wi-Fi specs since 2000, these other features that are differ-
More information is available at was chair of the 802.11n marketing ent are optional and not likely to be
www.opencloudinitiative.org. group at the Wi-Fi Alliance, and tested by the Wi-Fi Alliance.”
says “the editor of the WiGig spec The new Wi-Fi technology, because
To obtain help in cataloging a vast and happens to be the editor of the 11ad it operates at 60 GHz, is short-range.
important collection, Oxford Uni- spec, so there are a lot of us doing As envisioned, it will enable applica-
versity has launched a website that lets the same thing.” tions such as wireless docking and
armchair archeologists translate Kelly Davis-Felner, marketing direc- connection to displays, as well as
cont. on p. 9 tor for the Wi-Fi Alliance, says there was wireless backups, synchronization,
no need for the industry consortium and file transfers between computers
8 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Wi-Fi Making Big New Waves
News in Brief
and handheld devices. According to you’re looking for whole-home cov- cont. from p. 8
an introductor y white paper pub- erage, but if you have everybody and measure ancient Greek texts.
lished by the WiGig Alliance, the on the same band in a conference The Ancient Lives collection con-
technology features room, you’re going to get lower per- tains photographs of hundreds of
formance because there’s a lot more thousands of papyri containing lit-
support for data transmission rates noise. In 60 GHz, the beam is very erature and letters recovered in the
up to 7 gigabits per second; all narrow, and because we are able to early 20th Century from the Egyptian
devices based on the WiGig spec- do this beamsteering, you can have city of Oxyrhynchus — the “City of
ification will be capable of gigabit a bunch of people talking to each the Sharp-Nosed Fish.” Among the
data transfer rates; other, even in the same channel, and documents already translated from
support for low-power handheld you won’t have interference. this collection are masterpieces by
devices such as cell phones, as “If you wanted to replicate that the ancient Greek poet Sappho
well as high-performance devices in 2.4 and 5 GHz, you could, but a nd d r a m at i s t s M e n a n d e r a nd
such as computers; it includes you’d need 10 centimeters between Sophocles.
advanced power management; the antennas, so you run into size More information on the Ancient
native Wi-Fi support, and support limitations.” Lives project is at http://ancientlives.
for devices to transparently switch Perahia says testing the beam- org.
__
between 802.11 networks operat- forming technology will be paramount,
ing in any frequency band includ- “because without beamforming, you’re A new survey from the Pew Internet
ing 2.4 GHz, 5 GHz, and 60 GHz; talking about a foot of range. That’s & American Life Project shows
support for beamforming, maxi- where I’m hoping we got everything that one-third of US adults now
mizing signal strength, and enabling right; we’ll find that out in the test- own smart phones — and two-
robust communication at distances ing. It’s not just that you have a thirds of those owners sleep with the
beyond 10 meters; transmitter and receiver like in g phones next to their beds. Among the
advanced security using the Galois/ or n. This is beyond that, a whole demographic groups with the highest
Counter Mode of the Advanced handshaking that has to go on, and adoption levels are financially well-
Encryption Standard (AES) algo- exchange of information beyond the off and well-educated adults, non-
rithm; and normal testing of Layer 1 waveform.” whites, and people under 45 years
support for high-performance of age. The majority (87 percent)
wireless implementations of HDMI, Node-to-Node Wi-Fi access the Internet on the device, with
DisplayPort, USB, and PCIe. Two other Wi-Fi initiatives, Wi-Fi 68 percent of those surveyed doing
Direct (www.wi-fi.org/Wi-Fi_Direct. so daily.
The new technology’s keystone php) and 802.11s, which are intended
__ More information is available at
enabling feature is called beamform- to enable more node-to-node mesh- http://pewinternet.org/Reports/2011/
ing or beamsteering. Because radio like behavior, are also expected to Smartphones.aspx.
___________
signals at 60 GHz are extremely sen- hit the market soon. Davis-Felner
sitive to propagation loss, designers says the Wi-Fi Alliance, which intro- ISOC and the Internet Research
had to figure out a method by which duced the Wi-Fi Direct initiative, has Task Force (IRTF) have announced
signals could persist in instances already certified 219 products to the inaugur al winners of their
such as when someone walked between comply with the specification, but Applied Networking Research
two devices in the middle of a com- the market uptake has been slowed Prizes (ANRP) for work that
munications session. by a lack of native operating system directly improves products and ser-
Grodzinsky says that, in the same support in Windows, Android, a nd vices and advances Internet stan-
space that a 2.4- or 5-GHz device iOS and the commensurate dearth of dards. The ANRP winners were
can place two antennas, the 60-GHz applications running on them. Mattia Rossi, of the Swinburne
design allows an offset 16-antenna “I think the application support is University of Technology’s Centre
array. kind of relying on the OS support, for Advanced Internet Architectures,
“The more antennas you have, and I believe that will come,” she and Beichuan Zhang, of the Uni-
and when you can offset them by says. “It’s taking time because oper- versity of Arizona’s Computer Sci-
phase, you can start directing the ating systems don’t get updated every ence Department. The researchers
beams in specific locations,” he day. In the meantime, I think indi- presented their findings at the IRTF’s
say s. “2.4 - a nd 5-GH z Wi-Fi a r e vidual vendors are kind of stitching cont. on p. 10
omnidirectional. That’s great when together the apps, and the silicon
SEPTEMBER/OCTOBER 2011 9
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
News & Trends
News in Brief
cont. from p. 9 providers have an SDK or upgrade avis-Felner admits that the job
open meeting, held in July as part of
the IETF meeting in Quebec City.
that will bridge the gap: so Wi-Fi
Direct is going into products now,
D of promoting the ever-increasing
Wi-Fi technologies is becoming more
Rossi’s work focuses on reducing but its use in the market is nascent.” complex, but is also proud of the
Border Gateway Protocol traffic, She doesn’t think the 802.11s ubiquity it’s demonstrated.
while Zhang’s focuses on green traf- mesh standard — which garnered “I think the thing that has been
fic engineering. The awards will be 97 percent approval in sponsor bal- such a pleasant surprise is the way
given three times each year in con- loting in May — and Wi-Fi Direct the technology has continued to
junction with the IETF’s three annual will be vying for the same node- grow and expand. Smart energy is a
meetings. to-node uses. She predicts that the perfect example. It never occurred to
Information on the honored 802.11s mesh technology will emerge me that we would be putting Wi-Fi
researchers and the ANRP nomi- in applications such as smart energy on thermostats, but that’s exactly
nation process is available at http://
____ monitoring networks, and that Wi-Fi what we’re doing.”
InternetSociety.org/anrp.
_______________ Direct will be favored in intermittent
uses such as people sharing photos Greg Goth is a freelance technology writer
between Wi-Fi-enabled smart phones. based in Connecticut.
________________
___________________
10 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Guest Editor’s Introduction
Virtual World Architectures
E
arlier this year, in “Next-Generation of space, with fixed-size regions and a Craig W. Thompson
Virtual Worlds: Architecture, Sta- single physics model; others can accom- University of Arkansas
tus, and Directions,”1 I described modate portals that take a user from
the promise of 3D virtual worlds to one world through a door into another.
complement the Web with 3D models Some focus on cartoonish models and
of virtual places that are fanciful or support social interaction of small groups;
that model and mirror the real world. others are used for training or simula-
Marketplace evolution is one way to tions and can accommodate hundreds
wait and see if and how this will come of avatars per region.
about. Another approach is to identify As I noted in my previous article,
current limitations of virtual worlds, the real world is 3D, ver y high def,
deconstruct and study their architec- scalable, and diverse. If we wanted to
tures, and consider how to evolve them model it, we’d have to ask what kind of
to realize their promises. Here, I discuss database schema or object model could
nine articles that explore architectural be used to represent the world. With-
issues related to virtual world evolu- out going into detail, we could take
tion. Although there isn’t room in this the schema of a 3D virtual world as
special issue to run all the articles, a starting point. The kinds of entities
they all warrant introduction as inter- we’d need to model include locations at
esting examples of the state of the art a variety of scales, land use and struc-
in this field. tures, avatars, primitive and composite
objects, inventory items, assets, access
Virtual Worlds 101 authorizations for places and things,
Dozens of 3D virtual world implemen- and scripts.
tations currently exist. Most contain If we deconstruct the most widely
notions such as regions (land); avatars used virtual world, Second Life, we
that represent users who can walk, fly, would find that it’s architected as a cli-
chat, or speak; and objects that ava- ent viewer with servers that contain
tars can build, own, trade, or store in content or provide other services such
their inventor y. Some virtual worlds as avatar authentication. We might
are closed, in the sense that importing notice that virtual worlds are built on
or exporting content is difficult; oth- a suite of lower-level standards — for
ers are open. Some have a fixed notion instance, IRC for instant messaging and
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 11
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Guest Editor’s Introduction
Collada (Collaborative Design Activity; www.
____ Cristina Lopes explores these issues in “Hypergrid:
collada.org) or Second Life primitives (prims) Architecture and Protocol for Virtual World
for graphical content. Interoperability.”
It wouldn’t take long to notice that virtual At a high architectural level, we can dis-
worlds have various limitations. As similar as tinguish a virtual world platform from appli-
virtual worlds are to gaming platforms, they cations that are built on top of virtual worlds.
don’t meet all the requirements for building Architectural questions arise: What kinds of
certain kinds of games, especially fast-paced, applications can be built on virtual worlds?
first person shooter games. Today’s virtual Where is the dividing line between the virtual
world implementations don’t scale to a stadium world platform and the application? One way to
of avatars or the entire earth. Virtual world answer the question of what capabilities a vir-
implementations are heterogeneous, and most tual world platform should support is to view
don’t interoperate. Rapidly populating virtual this question as a red herring. Instead of a fixed
worlds by importing content from geographic virtual world platform, we’d like extensibil-
information systems or the CAD community ity mechanisms for augmenting virtual worlds
is still uncommon, and there isn’t yet a widely with additional capabilities. We can imagine
used way to mirror state change in the real virtual worlds with or without avatars, with
world directly into virtual worlds or to model different physics engines, with high- and low-
past, present, and possible futures in virtual fidelity sound, and so on. Toni Alatalo in “An
worlds. Entity-Component Model for Extensible Vir-
tual Worlds” and Jonathan Kaplan and Nicole
In this Issue Yankelovich in “Open Wonderland: An Exten-
So, what problems must we solve to make vir- sible Virtual World Architecture,” working in
tual world technology widely useful? two different virtual worlds (OpenSimulator
First, we’d need to make it as seamless for and Open Wonderland), have developed similar
any user anywhere to visit and leave one vir- component capability extension mechanisms
tual world for another as it is for us to come to accommodate the range of variation that we
and go to websites. Virtual worlds typically can predict will be needed in different virtual
use a client-side viewer that renders content worlds built for different purposes.
stored remotely on servers. It makes sense to Capabilities we’d want in a virtual world
integrate virtual world viewers into Web brows- could include search engines and ways to add
ers. Already, virtual world URLs can access a semantics to create “semantic worlds.” In “Vir-
virtual world location (for instance, the Second tual and Real-World Ontology Services,” Joshua
Life URL http://slurl.com/secondlife/University Eno and I explore how we can use virtual
of Arkansas/123/81/32/ accesses an x-y-z loca- world search engines to collect objects (and
tion on the University of Arkansas island). their labels) and then use those labels to build
In “Extending Web Browsers with a Unity taxonomies that match some large-scale ontol-
3D-Based Virtual Worlds Viewer,” Neil Katz, ogies, like WordNet and DBpedia. We observe
Thomas Cook, and Robert Smart describe an that virtual worlds don’t generally contain
architecture for plugging the Unity 3D viewer a semantic layer, that such a layer might be
into Web browsers. Their aim is to remove equally important in modeling the real world,
the roadblock of hav ing separate applica- and that a smart semantic world (analogous
tions for Web browsing and v ir t ual world to the Semantic Web) might result if we could
interaction. extend virtual worlds (that can mirror the real
Just as anyone can create a website, it makes world) with corresponding semantic types and
sense for anyone to create a virtual world. But rules. If virtual world architectures become exten-
it also makes sense that an end user’s avatar be sible (as in the Alatalo and Kaplan/Yankelovich
able to leave one virtual world and enter oth- articles), then an ontology service can store
ers. Thus, we need a solution to avatar inter- and retrieve semantics about avatars, objects,
operabilit y, so that an avatar can move and places for virtual or real-world applica-
between virtual worlds, and we need vari- tions that need that capability.
ous ways to federate virtual worlds so indi- Some virtual worlds such as Second Life
vidual worlds can come and go like websites do. make trade-offs in their modeling capabilities.
12 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
In Second Life, it is difficult to model very focuses on avatar interoperability; the Web 3D
large objects or very small ones, or to simulate Consortium (www.web3d.org) is developing 3D
small motor skills in workflows or object inte- standards; and the IEEE Metaverse Standards
riors. Second Life isn’t the platform to use for working group (www.metaversestandards.org)
performing remote surgery and doesn’t make is developing a glossary and a reference archi-
it easy to model stresses in bridges, heat flow, tecture for virtual worlds. Common APIs might
building plumbing and wiring diagrams, or make sense. In “Toward a Semantic Approach
similar simulation requirements. In “Accuracy to Virtual World Standards” (also to appear in
in 3D Virtual Worlds Applications: Interactive a future issue), David Burden considers virtual
3D Modeling of the Refractory Linings of Cop- world markup languages as another area that
per Smelters,” authors Anthony J. Rigby, Ken- could be standardized.
neth Rigby, and Mark Melaney identify and
discuss the requirement for accurate model- Future Directions
ing in some 3D world applications, like engi- Where is virtual world technology going, and
neering and CAD applications and military will virtual worlds fulfill their promise lead-
simulations. ing to pervasive use? Virtual world technol-
Two articles focus on applications built on ogy is no longer in its infancy, but it’s still
top of virtual world platforms. In “Connect- immature. A Gartner hype cycle graph shows
ing Virtual Worlds with the Real World for virtual world technology with inflated expec-
Learning a Foreign Language” (to appear in a tations in 2006, a disillusionment trough in
future issue of IEEE Internet Computing), María 2009, and the virtual world community cur-
Ibáñez, Carlos Kloos, Derick Leony, José García rently slowly climbing an enlightenment slope
Rueda, and David Maroto build an educational toward a productivity plateau. While Second
application on top of Open Wonderland that Life is still the dominant virtual world plat-
involves a mirror world where students inter- form, the open source OpenSimulator platform
act in the real world and also in a correspond- is solidly functional, as are several other vir-
ing model world, both representing an avenue tual world platforms such as Unity and Open
in Madrid. In “I-Room: Augmenting Virtual Wonderland. But there is not yet a clear front-
Worlds with Intelligent Systems,” Austin Tate runner architecture or implementation that
describes a suite of collaboration tools devel- meets the needs of the many potential virtual
oped at the University of Edinburgh that can worlds applications.
be used in civilian or military command cen- Early adopters in the broad education com-
ters to gather information, understand an munit y use vir tual worlds for classes and
evolving situation, and make decisions. Sev- meetings. There are workshops, conferences,
eral of the tools (to-do lists, planners, and so and journals that publish the occasional vir-
on) can be used independently of a virtual tual world paper — and a few venues directly
world. Interestingly, they can be tied into a focus on virtual worlds. But the academic-
virtual world (Second Life or OpenSimulator) industrial virtual world research community
so that, though geographically distant, the is splintered, heterogeneous, and distrib-
planners (that is, their avatars) can meet uted. The IEEE Metaverse Standards working
together, chat or talk, and see in-world rep- group provides one of the best current forums
resentations of shared collaborative content. for architects to meet to discuss virtual world
This virtual presence helps synchronize the directions.
team.
Although many areas of virtual world tech-
nology need further exploration, virtual worlds t seems clear that virtual worlds can go well
are evolving toward standardization. Rather
than a monolithic standard, the area is moving
I beyond being venues for social interaction to
also support serious applications involving
toward a suite of loosely coupled standards that teaching, training, and simulation. Especially,
help insure interoperability: Collada is recog- it seems likely that we’ll eventually have 3D
nized as the gold standard for graphical content; models of the real world and be able to use tech-
the IETF Virtual World Region Agent Protocol nologies such as RFID, Kinect, and smart phones
effort (VWRAP; _____________________ 2
http://tools.ietf.org/wg/vwrap) to constantly gather and update the models.
SEPTEMBER/OCTOBER 2011 13
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Guest Editor’s Introduction
It’s not yet clear what route we’ll take toward 2. J. Bell, M. Dinova, and D. Levine, “VWRAP for Vir-
a 3D Web or whether we’ll get there via the tual Worlds Interoperability,” IEEE Internet Computing,
efforts of a dominant player or virtual world vol. 14, no. 1, 2010, pp. 73–77.
platform or some other route coming out of
left field. However we get there, it seems that a Craig W. Thompson is the Charles Morgan chair in the
good understanding of virtual worlds’ software Department of Computer Science and Computer Engi-
architecture will help ensure that eventual neering at the University of Arkansas. His research
solutions will meet a broad array of community interests include artificial intelligence, databases, mid-
requirements. dleware architectures, virtual worlds, RFID, and per-
vasive computing. Thompson has a PhD in computer
science from the University of Texas at Austin. He’s an
References IEEE fellow. Contact him at cwt@uark.edu.
________
1. C. Thompson, “Next-Generation Virtual Worlds: Archi-
tecture, Status, and Directions,” IEEE Internet Comput- Selected CS articles and columns are also available
ing, vol. 15, no. 1, 2011, pp. 60–65. for free at http://ComputingNow.computer.org.
IEEE Internet Computing: Call for Papers
Submit a manuscript on ScholarOne at https://mc.manuscriptcentral.com:443/ic-cs
Programmatic Interfaces emerging technologies and best development practices that un-
derpin any modern programmatic Web interface. Sample topics
for Web Applications (July/August 2012) include
Final submissions due 1 November 2011 ■ best practices, patterns, and anti-patterns of a programmatic
Web interface design;
Please email the guest editors a brief description of the ■ benchmarking and evaluation of programmatic Web interface
article you plan to submit by 15 October 2011 scalability and performance in large-scale Web applications;
Guest Editors: Tomas Vitvar, Cesare Pautasso, and Steve ■ comparisons and empirical evaluation of various styles, pro-
Vinoski (ic4-2012@computer.org)
_______________ tocols, and descriptions for programmatic Web interfaces;
■ reports and lessons learned from developing programmatic
T
he rapid growth of programmatic Web service interfaces for Web interfaces for various application domains and sectors
Web applications (open Web APIs) has revolutionized online (such as social, e-commerce, video, geospatial, and so on); and
content integration and development practices. The increas- ■ end-to-end engineering of programmatic Web interfaces and
ing popularity of such Web interfaces raises questions of how their integration with existing back-end applications requir-
developers should design services and how they should maintain ing the development of novel dependable and scalable tech-
services’ good performance and scalability. Programmatic Web nology frameworks.
interfaces typically use REST style for communication, or REST-
ful services implemented with HTTP, while moving away from All submissions must be original manuscripts of fewer than
more traditional SOAP Web services. Although they can take 5,000 words, focused on Internet technologies and implementa-
advantage of already existing Web architecture, many APIs that tions. All manuscripts are subject to peer review on both technical
claim to be RESTful actually fail to do so. They overload the merit and relevance to IC’s international readership — primarily
meaning of HTTP methods, ignore standard response codes, or system and software design engineers. We do not accept white
do not well support hypermedia to represent relationships among papers, and we discourage strictly theoretical or mathemati-
application states. Moreover, developing a programmatic Web cal papers. To submit a manuscript, please log on to ScholarOne
interface requires a tight integration with already existing back- (https://mc.manuscriptcentral.com:443/ic-cs) to create or access
end applications and infrastructures, and sometimes requires a an account, which you can use to log on to IC’s Author Center and
new, highly dependable back-end technology. upload your submission.
This special issue seeks original articles on topics related to
www.computer.org/internet/author
14 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Extending Web Browsers
with a Unity 3D-Based
Virtual Worlds Viewer
Many virtual worlds are accessed via a rich client interface that must be
downloaded and installed into the user’s environment. For many users,
especially enterprise users, this large download and install represents a
significant obstacle to virtual world acceptance. The authors describe a
technical implementation that uses the Unity 3D browser plug-in as a way
to access a virtual world from within a Web browser. Using this familiar tool,
users can interact with the rich virtual environments provided by Second Life
and OpenSimulator.
T
hree-dimensional virtual worlds, need to communicate in real time. Vir- Neil Katz, Thomas Cook,
exemplified by Linden Labs’ Second tual worlds fill this gap. and Robert Smart
Life and the open source Open- Architecturally, virtual worlds are IBM
Simulator (OpenSim; www.opensimulator. often structured similarly to the World
org), let people collaborate and com-
__ Wide Web. Servers hold content (called
municate in ways not possible with regions or islands), and client applica-
today’s phone and videoconferenc- tions (viewers) let users browse (render)
ing systems.1 In these virtual worlds, the scenes. As on the Web, users can
users are represented by avatars, which follow a link (like a URL but containing
can walk, talk, and even fly, giving a region name and coordinates) at any
users a spatially familiar view that time to teleport to another region or
isn’t possible in other collaboration location. Unlike the Web, the protocol
systems. For example, users in vir- between the viewer and region servers
tual worlds can see and interact with is typically proprietary and not based
objects and other users, communicate on an open standard such as HTTP/
by voice or chat, and mimic real-world HTML. Also, unlike the Web, the proto-
interactions. Virtual worlds are useful col between the viewer and the server
for training and learning, as well as is stateful; the server must keep track
meetings and events.2,3 In today’s envi- of logged-in users and retain informa-
ronment, where travel is expensive and tion about the avatar’s position in the
involves high overhead, there is still a virtual world.
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 15
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Recreational users, including the game-playing their own virtual world and make it either pri-
community, have no problem downloading new vate or open to the public. These virtual worlds
applications. These users often have high-end can be linked together to create a grid of virtual
machines that can handle advanced graphics. worlds. They can sit behind corporate, school, or
But other classes of users, such as enterprise government firewalls in private grids, or they
users, might not have the same capabilities. can be connected to public grids. OpenSim
For example, in many enterprises, the aver- can use the Second Life client solution because
age machine is several years old, with a low- Linden Labs published the interface between the
end support for graphics rendering. The same is client and server. The open source community
true for many home users who might not have developed a server-side library to implement the
upgraded to machines with advanced graphics interface. Originally called libsecondlife, it was
that let them render virtual worlds with good renamed libopenmetaverse, or libomv for short.
performance and reliability. Also, users in both Today, OpenSim represents tens of thousands
environments might not install new applications of lines of open source code and can support a
such as a virtual world viewer: home users large grid structure. In addition, the Second Life
might find downloading, installing, and set- client can interoperably connect to OpenSim or
ting up new applications too complex, and Second Life grids. Finally, OpenSim has been
enterprise users might not be permitted to add moving toward a 3D application server model
software to their machines. Generally, both in which virtual world scripts process external
casual and enterprise users might want to use data from sources such as weather sensors, real-
virtual worlds for meetings and events, but world motion sensors, and vehicle positions,
they need to quickly connect and disconnect. which are ref lected in the vir tual world. At
Thus, for many virtual world scenarios, a full the same time, because the interface was well
client install is unnecessarily cumbersome — a defined, other open source efforts developed cli-
simpler solution is needed. ent viewers — for example, the Hippo viewer.
To address this problem, we extended a Web For expert users, a rich desktop client such
browser to seamlessly support not only Web as the Second Life client or the Hippo virtual
browsing but also connecting to virtual worlds world client are excellent solutions. The advent
such as Second Life and OpenSim. This exten- of OpenSim has helped improve libomv, which
sion lets a much broader class of users access has opened the door to client-side developers.
virtual worlds without requiring high-end Although the programming interface to imple-
machines or special viewers. Removing this ment such a client solution is available, in real-
roadblock to widespread adoption could remove ity no solutions have emerged as good-enough
a chicken-and-egg problem — as long as most for enterprise or casual users. Although some
users can’t access virtual worlds with just a researchers have attempted to use Linden Labs
browser, virtual worlds won’t become commer- protocols with the libomv open source code,
cially interesting. those efforts have been hampered by the need
for a low-impact game engine, which could run
Virtual World Viewer Issues inside a Web browser and effectively render 3D
One reason the Web was successful so quickly virtual world content.
was that anyone could create content on a
server accessible to the Internet that anyone Toward a Low-Impact Viewer
else could access, from anywhere in the world. Because IBM had a large user set that we needed
Only part of this equation was true for Second to connect to virtual worlds, we needed a light-
Life — any user anywhere could access it, but weight browser-based viewer. Over the course of
Linden Labs controlled all content on its grid of several years, we surveyed and experimented with
servers. OpenSim removed this constraint. a number of virtual world platforms. We studied
OpenSim is an open source server-based meetings in Second Life and also held large events
software project — anyone can download and with hundreds of participants both within IBM
install OpenSim and a companion viewer (for and at public events, such as the X10 Workshop
example, the Second Life viewer desktop client or on Extensible Virtual Worlds (http://vw.ddns.
the Hippo open source viewer [http://mjm-labs. uark.edu/X10/index.php?page=overview). As we
____________________________
com/viewer]). They can then (for free) create began to bring more corporate users into the
16 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Extending Web Browsers with a Unity 3D-Based Virtual Worlds Viewer
virtual world, we received feedback requesting compatibility problems. Unity has rock-solid support
a lightweight zero-install method of accessing for almost all hardware/software combinations, in
virtual worlds. both DirectX and OpenGL.
Overall, we found the Second Life and
OpenSim models of in-world content creation and In contrast, the Second Life website contains
scripting to be important in delivering a high- a lengthy list of graphics cards that aren’t
quality experience at an acceptable content- compatible (http://secondlife.com/support/system
creation cost point. We determined that our -requirements).
__________
solution for low-impact users needed to be Security within the enterprise is a big con-
either an extremely lightweight installable cern, with var ying levels of desktop lock-
game engine that could be compatible with down and firewall port constraints. IBM has
delivery of Second Life or OpenSim content, or progressive rules for employees, so we didn’t
it had to be browser-based. Web browsers sup- directly address these concerns when creating
port dynamic add-ons to add functionality. the viewer. In addition, both the Second Life
Users should be able to show up shortly before and OpenSim environments are accessed from
a meeting, perform a one-click browser install, within the IBM firewall.
and be in a conference or meeting quickly.
When we decided to create a low-impact Architectural Overview
viewer, the Unity 3D game engine was gaining The project team consisted of two core develop-
momentum in the marketplace. We combined a ment members and a handful of others drafted
Unity-based client with the rich virtual world for short periods to work on specific tasks such
content delivered by Second Life and OpenSim as graphic design. This is typical of projects
servers, leveraging the libomv APIs. It would conducted in IBM’s Chief Information Officer
take some experimentation and performance (CIO) Lab.
evaluation to ensure that the browser-based Us i n g s m a l l te a m s h ad t he fol low i n g
Unity 3D engine could deliver a good enough benefits:
experience for the casual user. Additionally,
we would have to determine a separate bidirec- clear lines of communication,
tional voice solution. reduction in process overhead,
The existing Second Life client, weighing easier division of tasks between members,
in at 25 Mbytes for Windows and 46 Mbytes and
for Mac, was a significant barrier to entry for production of a coherent design and code-
some users, particularly those on slower Inter- base without having to spend a lot of time
net connections. The time taken to download producing the architecture documentation
and run through the install process varies, but necessary for a large development team.
for users on a slow connection and old machine
the process can easily take 10 minutes or more. Throughout development the team produced
In contrast, the install size of the Unity plug- rapid iterations of the viewer, with new features
in is around 3 Mbytes and usually can install discussed, designed, and added in days or even
without a browser restart. So, although not a hours. This approach allowed the team to gain
zero install, it is as small as possible without immediate feedback from a group of hands on
resorting to using WebGL, which isn’t widely test users.
supported. Figure 1 shows the system’s overall archi-
Another problem area for enterprise users tecture. The main component is the virtual
where Unity fares well is support for old hard- spaces viewer, which runs on top of the Unity
ware and graphics drivers, as the Unity web- Web browser plug-i n; t he v iewer is com-
site notes (http://unity3D.com/unity/features/ patible with Mozilla Firefox and Microsoft
deployment):
________ Internet Explorer. The container webpage is
loaded from an application server that hosts the
Many potential players are using outdated graphics page’s dynamic HTML (DHTML) content and
hardware and drivers. Even many common computer the Unity 3D content archive. When the page
configurations are much less than ideal for games. finishes loading, the Unity 3D browser plug-
Unity has built-in fallbacks and workarounds for in requests the content archive and initializes it.
SEPTEMBER/OCTOBER 2011 17
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Server OpenSim/SecondLife Enterprise grid
LDAP
Virtual spaces IBM
Texture authentication
application user Authentication Grid
service Regions
server directory service services
HTTP Image conversion
page load jpeg2000 to png
Client UDP packets
Virtual spaces viewer webpage
XMLRPC-based containing world
DHTML page elements authentication state updates
Dojo toolkit Unity 3D browser plug-in
libopenmetaverse
Chat and IM
widget
JavaScript
callouts
Browser Object
HTTP
message manager
Dojo manager Texture Terrain Avatar
bridge
Map widget pub/sub manager manager manager
layer
PrimMesher
Contacts
Unity 3D engine
widget
Figure 1. Primary system components and how they communicate with existing virtual world server
components. The virtual spaces viewer is a Web browser add-on that’s compatible with Mozilla
Firefox and Microsoft Internet Explorer.
Once initialized, the plug-in invokes a Java- region is informed of the connecting user, the
Script method contained in the page, which dis- session is initiated, and a response returned
plays the login dialogue box to the user. containing avatar details and session informa-
This method of plug-in-to-browser two-way tion. The HTTP manager passes this information
communication handles most of the GUI I/O to the libomv DLL, which sets up the session on
traffic. Message commands are sent from the the client side and connects using UDP trans-
browser to the plug-in. A message bridge in the port to the region server.
plug-in routes the command to the correct com- After the client connects, the OpenSimulator
ponent. The components in the Unity 3D archive server sends the region state, including terrain,
are written in C# and executed by the plug- avatars, objects, and textures contained within,
ins built using the Mono virtual machine. The to the viewer for rendering.
inclusion of this virtual machine in the Unity
3D plug-in lets us use existing C# dynamic link Networking
libraries (DLLs). This ability, along with Unity One of the key design decisions was which vir-
3D’s other APIs and capabilities, provides a tual world server platforms the viewer would
major benefit over using something like WebGL support. The easy choice would have been to
as a rendering engine. support only the OpenSim platform; however,
When the user enters login details in the the Second Life Enterprise (SLE) platform and
browser, the details are passed via a JavaScript public Second Life are widely used inside IBM.
method to the plug-in. The message bridge then Supporting only OpenSim as a server plat-
routes the command to the HT TP manager, form would have been much more straightfor-
which makes an XML Remote Procedure Call ward because the team could modify and adapt
(XMLRPC) login request to the OpenSim (or the open source code to use any communica-
Second Life) authentication service. This ser- tions protocol. Because we chose to support
vice in turn contacts our internal user directory OpenSim, Second Life, and SLE, our only option
to authenticate. On the server side, an OpenSim was to use libomv, which left no possibility of
18 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Extending Web Browsers with a Unity 3D-Based Virtual Worlds Viewer
changing the protocol dictated by the closed
source SLE server.
Using libomv introduced an additional chal-
lenge: The library written in C# could be used
with Unity 3D, which includes the open source
Mono. However, Unity 3D applications deployed
as Web browser plug-ins run in a security sand-
box, which understandably prevents them from
accessing certain core classes of C# that allow Figure 2. Objects used in Second Life. In Second
access to the local machines’ storage. Life, primitive shapes such as cubes, cylinders,
This sandbox meant that components of the and spheres are modified and combined to make
chosen network library had to be rewritten to more complex shapes such as a chair.
use Unity 3D’s API so the rules of the security
sandbox could be adhered to. to understand these formulas so it could display
a scene it received from an OpenSim or Second
Describing 3D Objects Life server. To do this, we used the PrimMesher
A major challenge in developing a viewer for open source librar y. PrimMesher takes the
Second Life is recreating how it describes 3D description of each primitive shape sent over
objects. The Unity 3D engine, like most game the network and outputs structures containing
engines, uses a standard mesh description con- vertices, triangles, and UV maps describing how
taining the location of all points (vertexes) a texture should be positioned on the object.
that make up the object’s shape. Second Life, Using the Unity 3D API, PrimMesher can cre-
instead, uses the concept of primitives — basic ate a mesh object for each primitive shape and
shapes that can be modified and joined together place it in the scene.
to form more complex shapes. Textures are displayed on these primitive
Second Life was designed around the concept shapes, and pointers to the textures are included
of an entirely user-created 3D world, an ambi- in the data sent from the server. Each surface of
tious goal from the outset. In many games, such an object can display a different texture.
as World of Warcraft, taking part in a multi-
player 3D experience online requires installing a Graphical Performance
large game client that includes all the graphical Modern graphics cards are optimized to handle
content. These game clients can require several numerous polygons; however, a cost is associ-
gigabytes and take a long time to install. Linden ated with submitting each mesh to the graph-
Labs set out to create an expandable world with- ics card for processing. Submitting a few large
out limits; packaging all the content in the cli- polygonal mesh objects incurs less overhead
ent was never a viable option. To quickly load than submitting many small objects.
content from a server to a client, Linden Labs Unfortunately, OpenSim and Second Life
developed an efficient graphical representation scenes usually consist of thousands of small
called a prim (short for primitive object) with a primitive shapes. Even worse, each of these
fixed selection of primitive types such as cubes, primitives consists of several individual meshes,
spheres, and cones. The primitive shape could and a separate image texture can be displayed
then be modified in several ways, from simple on each surface.
scaling to complex twists and cuts. Each of To ensure a high frame rate, we optimized
these primitives could then be stored as a tex- the virtual spaces viewer in several ways.
tual object that included the base primitive type The first optimization, which might sound
and associated transformation modifiers. Using counterproductive, was to create two versions of
this representation, when a client connects to every object in the scene. The first version of each
the Second Life server, the server transmits a object was a fully detailed primitive assembled
compressed description of every object in the 3D from a separately textured mesh surface. The
scene, and the local Second Life client then uses second version was a single, less-detailed mesh
the prim description and the modifiers associ- combining all of the surface meshes with only
ated with it to recreate a 3D object (see Figure 2). one texture applied to the entire mesh. Unity
The virtual spaces viewer needed to be able 3D has a feature called layers, and objects can
SEPTEMBER/OCTOBER 2011 19
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Second Life, like many other online games,
uses the Vivox voice technology. The voice-over-
IP (VoIP) servers Vivox uses are fully Session
Initiation Protocol (SIP) compliant. The Second
Life client uses an embedded Vivox SIP client to
connect users to a shared voice channel using a
high-quality voice codec. Control messages con-
taining data such as an avatar’s world position
vary the volume of other speakers as well as ste-
reo information. Vivox produces a browser plug-
in that can be used to connect to the same servers
Figure 3. The number of objects displayed at any one time greatly and channels where other Second Life avatars
impacts the viewer’s performance. To increase performance, the are speaking. Connection instructions, volume,
viewer displays small objects only when the camera is near them. and positional information can be passed to the
Because checking each object’s size and distance for every frame plug-in using a JavaScript API.
rendered would be computationally expensive, we placed objects
onto logical layers ordered by size. Each layer displays objects up A Hybrid GUI Solution
to a fixed distance from the camera. The idea of running a Second Life or OpenSim
viewer client in a Web browser isn’t new and
be placed on one or more layers. Each scene in has been attempted using ActiveX wrappers
Unity 3D can contain one or more cameras that around the full Second Life client. This
can be dynamically positioned within the scene, approach at unifying virtual worlds and Web
and each of these cameras can be configured to browsers has drawn criticism because nothing
display zero or more layers. has been added to or removed from the original
Our second optimization was to use two experience, and these might as well have been
cameras in each scene displayed in the viewer. standalone desktop applications.
We configured the first to show the layer con- By their nature, 3D applications don’t handle
taining the high-resolution multisurfaced and display textual data well. Often in 3D deve-
meshes. We further configured this camera lopment, custom GUI elements are built to handle
to only show objects from zero to 40 meters the display of 2D information. Developing these
away. The second camera displayed a layer can be time consuming and ultimately redundant
containing only the low-resolution combined when displaying a 3D application in a browser.
mesh objects; it showed objects at a distance To exploit the browser’s strengths, we del-
of 40 meters or greater. Figure 3 illustrates this egated almost all text-handling capability to
technique. DHTML. This approach let us quickly develop
A similar optimization was to only put DHTML-based widgets to handle functions such
objects over a certain size in the layer contain- as text chat, contacts lists, and other 2D infor-
ing low-resolution objects. This reduced the mation displays.
number of meshes that had to be passed to the The Unity 3D plug-in provides scripting
graphics card for rendering. methods that let JavaScript functions be called
The optimization steps took a scene that in the host webpage and pass them informa-
previously ran at 20 frames per second and tion. In the same way, functions in the web-
increased that to around 100 fps. page can call script functions of the Unity 3D
plug-in. Figure 4 shows the resulting interface,
Adding Voice with DHTML-based widgets surrounding the 3D
Second Life’s introduction of voice to the 3D viewport.
environment was received with mixed reac- The viewer page’s portal-style layout also
tions. For business users, it was an essential allows for customization and expansion in the
tool and unquestionably a big addition to the form of new widgets or different arrangements.
platform. For some recreational users, however, A Web developer can easily edit the HTML to
it was an unwelcome interruption to the text- define a custom look and feel for particular
only conversational world and a step away from events or scenarios rather than needing a pro-
the fantasy world they liked to inhabit. grammer to change the 3D plug-in.
20 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Extending Web Browsers with a Unity 3D-Based Virtual Worlds Viewer
ne major constraint imposed on the view-
O er’s development was the need to support
both Second Life and OpenSim platforms. One
way to extend the architecture extension is to
abandon Second Life compatibility and tie the
viewer strictly to the OpenSim platform. This
step would open up several possibilities because
we could then change the server component and
protocol to specifically support the Unity view-
er’s needs. For example, one simple change is
to enable OpenSim to provide textures directly
in a PNG image format, which would remove
the need for the standalone texture conversion
service.
We field-tested our browser-based viewer
as the main interaction interface used by more Figure 4. A screenshot of a meeting taking place using the virtual
than 140 individuals from around the globe spaces viewer. The 3D view in the center contains the world
during an IBM Academy of Technology meet- view and movement controls, while the surrounding webpage
ing held in a virtual world in October 2010. components handle the display of contacts, map navigation, chat,
Compared to previous years when the Sec- IM, and voice features.
ond Life viewer was used, meeting attendees
reported fewer technical issues entering the
vir tual world meeting. As expected due to Neil Katz is an IBM distinguished engineer in the Chief
busy schedules, many users joined the meeting Information Officer’s Lab within the IBM CIO Office.
without prior testing of their ability to use the He’s responsible for the strategy and deployment of
low-impact viewer. Users were able to navi- emerging applications and technology to assist the
gate intuitively without prior training. None of IBM enterprise with better tools for enhanced collabo-
the attendees reverted to the high-resolution, ration. Katz has a BS in electrical engineering from the
standalone rich client virtual world viewer; all University of Florida and an MBA from Nova South-
continued to use the low-impact browser-based eastern University. He’s a member of IEEE. Contact him
solution. nkatz@us.ibm.com.
at ___________
A place still remains for the rich client
viewer as content builders and deep virtual Thomas Cook is a senior technical staff member at IBM
world users rely on the build functions and responsible for leading a team of designers and devel-
graphics depth for key 3D applications for vir- opers to create innovative solutions that help people
tual world collaboration and modeling. collaborate, connect, and share ideas. His work at
IBM has included mobile solutions, embedded sys-
References tems, game systems, virtual worlds, and operating
1. K. Bessière, J.B. Ellis, and W.A. Kellogg, “Acquiring a systems. Cook has a BS in computer science from
Professional ‘Second Life’: Problems and Prospects for Clarkson University. Contact him at tomcook@us.
________
the Use of Virtual Worlds in Business,” Proc. 27th Int’l ibm.com.
Conf. Extended Abstracts on Human Factors in Comput-
ing Systems (CHI 09), ACM Press, pp. 2883–2898. Robert Smart is an emerging technologies specialist at
2. R. Alther et al., “Virtual Spaces: Enabling Immer- IBM Hursley in the UK. As a member of the IBM CIO
sive Collaborative Enterprise, Part 2: Implementation Office Lab team, he’s responsible for architecting and
and Lessons Learned,” IBM developerWorks, 2009; developing technologies that allow IBM employees to
www.ibm.com/developer works/webser vices/librar y/ collaborate effectively. Smart has a BSc in computer
ws-virtualspaces2/index.html.
_________________ science from the University of Nottingham. Contact
3. R. Brunner et al., “Virtual Spaces: Enabling Immersive him at smartrob@uk.ibm.com.
_____________
Collaborative Enterprise, Part 1: Introduction to the
Opportunities and Technologies,” IBM developerWorks,
www.ibm.com/developer works/webser vices/librar y/ Selected CS articles and columns are also available
ws-virtualspaces/index.html.
_________________ for free at http://ComputingNow.computer.org.
SEPTEMBER/OCTOBER 2011 21
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Hypergrid: Architecture and
Protocol for Virtual World
Interoperability
Massive multiuser online (MMO) environments that simulate large virtual
spaces for many users have prompted the desire to create an even larger,
highly scalable environment in a federated manner. In a federation of virtual
environments, users should be able to visit different environments belonging
to different authorities while preserving their identity; they should also be able
to join a global, federated social network. The Hypergrid is an architecture
and protocol for securely decentralizing multiuser virtual environments.
It establishes an open federation of multiuser applications that can exchange
user agents and assets and can generally interoperate on several basic services.
M
Cristina Videira Lopes assive multiuser online (MMO) group) or jointly operating one single
University of California, Irvine games such as Linden Lab’s Sec- virtual world for the group (thereby
ond Life provide shared virtual losing control of their own share in
spaces, in which thousands of users that virtual world).
can interact with one another, with In this article, we present the Hyper-
virtual objects, and with artificial grid, an architecture and protocol for
intelligence (AI) agents. These environ- securely decentralizing multiuser vir-
ments require considerable server-side tual environments at all scales. The
infrastructure, controlled in each case Hypergrid establishes an open federa-
by a single organization. Centralized tion of multiuser applications that can
control of virtual worlds enables the exchange user agents and assets, and
development of walled-garden envi- can generally interoperate on several
ronments with high internal consis- basic services. It supports the teleport-
tency. However, several problems arise ing of user agents between worlds in
from centralization of authority. First, different administrative domains while
groups of individuals and organiza- preserving user identity, as well as the
tions wanting their own virtual worlds user’s 3D visual representation and con-
face the binary choice of either operat- nections to certain home-world ser-
ing separate walled gardens (thus mak- vices. We designed and implemented
ing them difficult to share across the the Hypergrid in the OpenSimulator
22 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Hypergrid: Architecture and Protocol for Virtual World Interoperability
Multiuser 3D Simulation and Gaming Environments
T he field of multiuser 3D simulation and gaming environ-
ments is divided into two architectural camps: peer-to-
peer (P2P) and client–server.
OpenWonderland).
___________ Open Wonderland is a virtual world cli-
ent implemented in Java that connects to Darkstar-based vir-
tual world servers, 8 also written in Java. Like the Hypergrid,
In P2P systems, the program that the user drives is both the Open Wonderland supports a federation of virtual worlds.
simulator and the user interface. An additional network layer However, that federation has the following architectural differ-
lets several peers join in one logical simulation, and physical ences. First, the client itself keeps the user agent information;
simulation of different parts of the scene occurs in the different the client is its own authority and keeps that state through-
peers. Examples of P2P multiuser virtual environments include out the session. Second, Open Wonderland relies entirely on
MiMaze,1 High-Level Architecture standards, 2 TeCo3D, 3 Cro- the Java programming language for dynamically loading code
quet,4,5 Miramar,6 and Unity 3D Basic (http://unity3d.com). P2P as the user moves from one world to another. These are
virtual environments are naturally federated, in the sense that interesting variations that simplify the interoperability archi-
each user-driven peer represents exactly one user and has full tecture at the expense of narrowing down the implementa-
authority over the user agent’s state and over parts of that vir- tion technologies and tightly coupling the servers with the
tual world. clients.
Massive multiuser virtual worlds follow a client–server
architecture. Their internal architectures vary considerably, References
but they all share one authoritative server side, to which 1. L. Gautier and C. Diot, “Design and Evaluation of MiMaze, A Multiplayer
interactive rendering clients connect. Besides the well-known Game on the Internet,” Proc. IEEE Int’l Conf. Multimedia Computing and
commercial massive multiuser online (MMO) games such as Systems (ICMCS 98), IEEE CS Press, 1998, pp. 233–236.
Second Life, Eve Online, and World of Warcraft, examples 2. F. Kuhl, R. Weatherly, and J. Dahmann, Creating Computer Simulation Systems:
of publicly documented server-side systems and prototypes An Introduction to the High-Level Architecture, Prentice Hall, 1999.
include RING,7 Project Darkstar (now RedDwarf), 8 Meru,9 and 3. M. Mauve, “TeCo3D — A 3D Telecooperation Application Based on VRML
OpenSimulator. and Java,” Proc. Multimedia Computing and Networking (MMCN 99), SPIE
In many ways, client–server architectures do well where 3654, Int’l Soc. for Optics and Photonics, 1999, pp. 240–251.
P2P architectures do poorly. First, client–server architectures 4. D.A. Smith et al., Croquet User Manual, tech. report, Open Croquet, 2005;
naturally support persistent, sharable virtual environments www.opencroquet.org.
that exist beyond the user agents that visit them. Second, they 5. D.P. Reed, “Designing Croquet’s TeaTime: A Real-Time, Temporal Environ-
provide many more options for scalability because the server ment for Active Object Cooperation,” Proc. 20th Ann. ACM SIGPLAN Conf.
side can be fueled with many high-end servers and appropriate Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 05),
bandwidth for acceptable quality of service. ACM Press, 2005, p. 7.
In other ways, client–server architectures do poorly where 6. J.D. Miller and C. Pickering, “From One to Many: Transforming Miramar
P2P architectures do well: client–server systems, such as the into a Collaboration Space,” Proc. 5th Int’l Conf. Creating, Connecting and Col-
Web, promote walled-garden environments, some of which laborating through Computing, IEEE CS Press, 2007, pp. 109–116.
end up dominating specific application areas. As people and 7. T.A. Funkhouser, “RING: A Client-Server System for Multiuser Virtual
organizations see value in interconnecting, additional pieces of Environments,” Proc. Symp. Interactive 3D Graphics (I3D 95), ACM Press,
architecture become necessary to enable those walled gardens 1995, pp. 85–ff.
to interoperate. This has been happening on the Web for a 8. J. Waldo, “Scaling in Games and Virtual Worlds,” Comm. ACM, vol. 51,
while. The Hypergrid is another step in that direction. no. 8, 2008, pp. 38–44.
The closest system to the spirit of the Hypergrid is Open 9. D. Horn et al., “Scaling Virtual Worlds with a Physical Metaphor,” IEEE
Wonderland (http://code.google.com/p/openwonderland/wiki/ Pervasive Computing, vol. 8, no. 3, 2009, pp. 50–54.
projec t ( ht t p://open si mu lator.org). A sec- are accessible via Second Life viewers as the
ond, independent implementation is now avail- user-driven clients. However, the Hypergrid
able in the SimianGrid (http://code.google.com/p/ can also support arbitrary Web-based multiuser
openmetaverse/wiki/SimianGrid). The SimianGrid
______________________ applications — a critical capability, as an ever-
is an alternative back end to OpenSimulator based growing number of Web-based viewers for
on PHP and Apache. The Hypergrid is already these virtual worlds are being developed. (The
deployed in several OpenSimulator-based virtual “Multiuser 3D Simulation and Gaming Environ-
worlds. ments” sidebar describes two main types of 3D
Here, we focus on the design of the Hyper- simulation and gaming architectures: peer-to-
grid for worlds based in OpenSimulator that peer (P2P) and client–server.)
SEPTEMBER/OCTOBER 2011 23
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Popular configurations include small grids with
User’s viewer client one or a few simulator servers, all directly con-
nected to a MySQL server on the same LAN,
and grids with multiple simulators connected to
Apache-server-based resource services over the
Internet.
Internet
Protocols
Login S S S S S S S …
service Here, we describe the major protocols in Open-
Simulator between the viewer, the login service,
and the simulator services when both the login
Virtual world
and simulators are all within the same adminis-
LAN,VPN, Internet
trative trust domain.
So that the viewer software could be reused
User without changing it, these protocols were heav-
accounts Assets Inventory Avatar ily influenced by how Second Life is engineered.
storage storage storage storage …
Although the protocols described here target spe-
cific commercial virtual worlds, they’re important
Figure 1. Main architectural components of an OpenSimulator- for three reasons. First, they embody a profound
based virtual world. These worlds can be as small as one single generalization of the well-known user agent con-
simulator (S) or as large as thousands of simulators that share cept on the Web. Second, they show how to man-
persistent resources. User-driven clients first authenticate with age user agent transfers in a distributed system.
the world’s login service, and then exchange data with specific Third, they’re the basis for the Hypergrid proto-
simulators. (VPN: virtual private network.) cols described later, which simply add security
safeguards for when the interacting components
belong to different administrative trust domains.
OpenSimulator
The OpenSimulator project began in early 2007 Login. The login protocol involves the user’s
as an open source server side to the Second Life viewer client, the login service, and a simulator:
client. A simulator is the basic unit of virtual
space containing one or more regions, which 1. The viewer contacts the login service on an
are 3D spaces of 256 m 256 m . Simula- HTTP-based (or HTTPS) connection, sending
tors can be interconnected to form larger, con- the user’s credentials (username and pass-
tinuous spaces that share persistent resources, word) and desired virtual place (simulator).
known as grids. In this article we treat “grid” as 2. The login service verifies the user’s creden-
synonymous with “virtual world.” tials. If they’re valid, the login process gen-
erates a pair of session IDs. (This is a minor
Architectural Style and Components detail of Linden Lab’s viewer; in other applica-
OpenSimulator worlds follow a client–server tions, only one session ID would be necessary.)
architecture similar to that of the Web: user- 3. The login service creates a user agent, which
driven clients merely render the application includes the session IDs and information
state, which remains on the server side. Figure 1 about the user’s 3D representation.
depicts the overall client–server architecture of 4. The login service logs the user’s session in
OpenSimulator-based virtual worlds. the grid using a persistent presence resource.
Logically, a grid comprises one or more 5. The login service sends this user agent to the
simulator services, a common login service, and simulator that runs the desired virtual place.
a collection of common resources such as assets 6. The simulator verifies the user’s presence
and inventory. Users access the virtual world with the given session IDs. If verification
through a client (or virtual world viewer). is successful, the simulator stores the user
In OpenSimulator’s software architecture, the agent and prepares for initial viewer contact.
connectors to all resource services are plug-ins. 7. The login service sends the login reply to
This allows for developing various concrete mid- the viewer, including the session IDs and the
dleware grid services to support the simulators. desired simulator’s IP end point.
24 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Hypergrid: Architecture and Protocol for Virtual World Interoperability
8. The viewer contacts the simulator. The simu- Gecko/20101203 Firefox/3.6.13. The Session
lator then verifies the existence of the valid Initiation Protocol (SIP) uses the term “user
user agent, and the simulation proceeds agent” to denote the user-driven client software’s
from there on. Internet end points.1
Here, we’ve generalized the concept of user
Once the user logs in, he or she can access and agent to include information not only about the
interact with the resources of the virtual world. software that users drive and their Internet end
The user can also move around to different points but also about the users themselves: the
parts of that virtual world through a process service end points used by a particular user,
called teleporting. identifiers of assets related to that user’s 3D rep-
resentation, authorization tokens, and so on.
Intragrid agent transfer. The intragrid agent Because these environments provide simu-
transfer (teleport) protocol involves the viewer, lation of virtual spaces, of which the user’s
the current simulator, and the target simulator representation is a part, portions of the user’s
(that is, the simulator to which the user wants to state could change as that user visits different
go next). We assume all server-side components simulators. For example, the user could carry
are in the same administrative trust domain: a script that stores the names of all users that this
script encounters; such a script is part of the
1. The viewer notifies the current simulator user agent, and it’s executed by each simula-
about the desired virtual place where the tor that the user visits. That script’s state (users’
user wants to go next. names) is accumulated as the user moves around
2. The current simulator sends a copy of the and the script migrates from one simulator to
user agent to the target simulator running another. Transfers of user agents between simu-
the desired virtual place. It also sends an lators ensure the preservation of the server-side
opaque callback address for later use. state related to the respective users throughout
3. The target simulator stores the user agent the session’s duration.
and prepares for initial viewer contact.
Preparation includes creating authorization The Hypergrid
tokens for the user agent to use while visit- The Hypergrid’s goal is to provide a relatively
ing that simulator. seamless user experience as users visit dif-
4. The current simulator sends information ferent v ir tual env ironments, while ensur-
to the viewer about the target simulator, ing the integrity of all parts. The Hypergrid
including the target simulator’s IP end point. achieves seamlessness through a single sign-on
Although the mechanism is quite different, (SSO) mechanism that preser ves user iden-
the nature of this step is similar to HTTP’s tity throughout the session’s duration, and by
redirect return code. making certain user services available to the
5. The viewer contacts the target simulator, virtual worlds that the user visits. The result
which verifies the existence of a valid pres- is an open but secure federation of virtual
ence for the user. environments.
6. The target simulator invokes the callback
to the original simulator, signaling that the Architectural Components
viewer has made contact. The Hypergrid consists of a collection of Web
services provided by the virtual environments
The current simulator discards its copy of the to the rest of the world in addition to their inter-
user agent, and the hand-off is complete. nal services. Figure 2 illustrates the Hypergrid’s
architecture.
User Agents
The term “user agent” is most notably used to The gatekeeper service. Users can enter a vir-
identify clients that access Web servers. For tual world via two main entry points: the regu-
example, the HTTP protocol includes a User- lar login service, which requires a local account
Agent request header that identifies the soft- in the virtual world, and the gatekeeper service,
ware used to issue the Web request: Mozilla/5.0 which is the entry point for users with accounts
(Windows;U;WindowsNT6.1;en-US;rv:1.9.2.13) elsewhere. All user agents from users of other
SEPTEMBER/OCTOBER 2011 25
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Login S S S S S S S … Gatekeeper Once users acquire an identity with their
service service home world via a login procedure, they can visit
other worlds. A visit to another world requires
LAN,VPN, Internet
sending user agent information to the target
world — more specifically, to its gatekeeper. For
Virtual world
security reasons, the only authority that can
User Assets Inventory Avatar send user agents to other virtual worlds is the
accounts storage storage storage … home world’s user agents service.
storage To illustrate the need for this component,
consider the following scenario. User 1 is visit-
ing some foreign world Y and wishes to move
Instant-
Assets Inventory messaging
User agents to another world Z. World Y has a copy of the
service service … service user agent, so it can simply send it directly to
service
world Z. However, such a direct exchange could
compromise the user agent’s integrity. A rogue
Figure 2. Main architectural components of the Hypergrid,
world Y could add malicious data to the user
including additional Web services that virtual worlds can provide
agent, undermining possible trust relations
to become part of the open Hypergrid Federation. The green
between the user’s home world and world Z.
boxes are internal to each grid, whereas the tan boxes are the
To avoid such vulnerabilities, the Hyper-
Hypergrid’s components.
grid establishes the existence of the user agents
service — the authoritative driver of all user
worlds enter through the gatekeeper; one of agents pertaining to each world’s local users.
the gatekeeper’s responsibilities is to authen- One of this service’s main responsibilities is to
ticate such user agents. Attempts at sending regenerate valid user agents every time users
user agents directly to the world’s simulators move between worlds. A second main respon-
will fail because the simulators expect the sibility is to keep track of all user agents and
gatekeeper to have authenticated those agents. their locations.
The authentication procedure is the basis for
the SSO mechanism, which is the core of the Additional user support services. Besides securely
Hypergrid. preserving user identity across virtual environ-
Additionally, the gatekeeper can filter user ments, the Hypergrid also provides federated
agents on the basis of access control rules and access to certain services that support a better
policy decisions regarding incoming data. user experience (see Figure 2). We describe one
of these services here.
The user agents service. As explained earlier, In these virtual worlds’ rich visual environ-
the virtual environments considered here use ments, the user’s 3D representation (avatar) is
a client–server architecture in which the client important, and its preservation across worlds
simply renders information kept by the server might be desirable. There are several differ-
side. This has some important consequences for ent ways to represent the avatar, but it always
user identity, and how it is managed throughout includes assets stored in the user’s home world.
the sessions. As such, preserving the avatar upon agent
The most important consequence is that all transfers requires providing access to those
authority resides not on user-driven compo- assets by the world that the user is visiting.
nents but on servers, and this includes informa- Serving assets on the Hypergrid isn’t the
tion pertaining to user identity: users acquire same as serving assets within one world, because
identities by logging in to identity services asset exchanges between worlds involve differ-
on the Internet. Those identity services could ent administrative and trust domains, and hence
be part of the virtual worlds, or they could be require additional filters and safeguards. Hyper-
stand-alone identity services. OpenSimulator grid asset servers should perform authorization
worlds include user accounts, and thus can be of requests and could perform metadata adjust-
identity providers. We call the system with ments. For example, the current implementation
which a user acquires his or her identity the of the Hypergrid asset service in OpenSimulator
home world for that user. adds universal resource identifiers to create
26 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Hypergrid: Architecture and Protocol for Virtual World Interoperability
information pertaining to exported assets. Thus, Hypergrid agent transfers. Here, we describe a
if John Smith created an asset in world A, the teleport protocol involving the viewer, the cur-
exported information would include a URL for rent simulator, the user agents service of the
user John Smith’s profile. user, the grid’s gatekeeper service where the
Besides assets, the Hypergrid enables the user wants to go next, and the target simula-
open-ended collection of user support services tor. There are three trust domains: the viewer
related to the user’s resources, social network, and the user agents service, the gatekeeper and
and communication. Social networking in the the target simulator in its grid, and the current
Hypergrid is a global, federated facility: users simulator. This protocol is based on the teleport
can have friends in other grids and can com- protocol described in the “Intragrid agent trans-
municate with them. Therefore, each world can fer” section:
expose services that support those global social
connections in a manner that shares similarities 1. (same, except the target virtual place is in
with Diaspora (https://joindiaspora.com). another grid and is identified by that grid’s
gatekeeper address)
Single Sign-On 2. The current simulator sends a snapshot of the
The Hypergrid SSO mechanism lets users log user agent to the user agents service of that
in only once to their home world and securely user, along with information about where
use their identities to visit other worlds in the the user wants to go next. It also sends an
federation without being prompted for cre- opaque callback address for later use.
dentials or confirmation. The Hypergrid SSO 2.1 T he gatekeeper and user agents ser v ice
mechanism is based on the protocols described interactions ensue. The security precautions
earlier but extends them to deal with compo- regarding these interactions are as follows.
nents in different administrative trust domains. First, the user agents service generates a
In these protocols, parts in bold denote the unique service key for the desired grid, adds
new protocol elements that the Hypergrid has it to the user agent data, and launches the
added. agent at the desired location’s gatekeeper
service. The unique service key consists of
SSO login. For the sake of simplicity, the pro- the destination’s gatekeeper URL, to which a
tocol explained here assumes users always log unique random token is added (for instance,
into their home world. In OpenSimulator, the http://hg.osgrid.org/?cap=9876543210). Sec-
Hypergrid login procedure is more general, let- ond, the user agents service updates the user’s
ting users log in directly to any grid. The sim- traveling data with the new destination and
plification made here doesn’t change in any way service key (for example, a user agent with
the main security safeguards on agent transfers session ID 1, IP address 70.45.12.64, going
that the Hypergrid adds. to hg.osgrid.org, with service key http://
Here, the user agents service, the login ser- hg.osgrid.org/?cap=9876543210). Third, the
vice, and the initial simulator are all within user agents service might filter data from
the same administrative trust domain. The the user agent it received from the departing
sequence of events is essentially identical to simulator, and then launch the agent at the
the one described earlier in the “Login” section; destination gatekeeper. The destination gate-
the main difference is the collection and stor- keeper service performs verification against
age of additional information for the user agent: fake agents (impersonations). The data used
that is, in step 3, the login service creates a user for this is the provided service token and the
agent comprising the session IDs, information reported user agents service URL.
about the user’s 3D representation, and a col- 2.2 If all verifications succeed, the gatekeeper
lection of URLs representing the user’s services, logs the user session in its grid using a per-
including the user agents service; and, in the sistent presence resource, and launches the
latter part of step 4, the login service sends the user agent at the desired local simulator.
user agent’s information to the user agents 3. (same)
service; this information includes the user’s 4. (same)
client IP address, as reported by the initial login 5. The viewer contacts the target simulator,
request’s TCP stack. which verifies the existence of a valid presence
SEPTEMBER/OCTOBER 2011 27
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
for the user. Additionally, the target simula- servers should be granted only to authorized
tor contacts the user’s reported user agents asset consumers and not to anonymous clients
service URL for verification of the user’s cli- on the Internet. The only authorized consumers
ent IP address. This prevents other kinds of of those resources are the worlds that the home
impersonations. users visit at any point in time. For example, if
user U of home world H visits virtual world Z,
Finally, steps 6 and 7 are the same as in the then Z might need to download U’s avatar assets
“Intragrid agent transfer” section. from H to construct an accurate 3D representa-
tion of the user. Similarly, if, while visiting Z,
Security U gives an item to another user, Z must broker
Virtual worlds, especially those built on client- that transfer, which requires access to the item’s
ser ver architectures, operate within broad assets stored in H.
margins of mutual trust. To a large extent, We’re currently adding this authorization
this trust is determined by current technologi- mechanism to OpenSimulator’s Hy pergr id-
cal limitations about what can be protected. facing asset server. It works in the following
Nevertheless, neither users nor virtual worlds manner. According to the protocol explained in
should be allowed to go beyond those mar- the “Hypergrid agent transfers” section, every
gins of trust. Hypergrid security must ensure time the user agents service sends a user agent
the integrity, availability, and confidentiality to a new virtual world, it issues a unique ser-
of resources intended to be integral, available, vice key. The target world uses that key as an
and confidential. Two particularly important authorization token to access resources of the
types of resources that need protection are the user’s home world. Subsequent requests to
user agents themselves and the virtual worlds’ the resources of the user’s home world must
assets. include that key. Unauthorized requests will be
denied access to the asset resources.
User Agent Integrity The Hypergrid asset servers can establish
Whatever abuse might occur in one world more restrictive policies on top of this autho-
should be limited to that world only, and rization mechanism. For example, they could
shouldn’t compromise the integrity of the user’s deny access to certain types of assets.
agents sent to other virtual worlds. This is the
main purpose of the user agents service in the
Hypergrid architecture. lthough we designed and implemented the
Hypergrid security relies primarily on reliable
user agent authentication throughout the federa-
A Hypergrid for OpenSimulator-based virtual
worlds and the dedicated clients currently used
tion of virtual worlds. If impersonations were to to interact with those worlds, we ultimately
occur, the Hypergrid wouldn’t function. Imper- would like it to be an architecture and proto-
sonations could occur if rogue virtual worlds col for federating virtual environments on the
visited by users could send rogue user agents to Web itself. Consider the architecture in Figure 2.
other worlds and then control those user agents If we substitute Web servers for the simula-
as if they were representing the users. tors (S), the components in green represent the
Fortunately, the verifications that the gate- server-side architecture of many multiuser
keeper and the target simulator make against Web applications. As such, it’s straightfor-
the user agents service ensure that imperson- ward to add the (tan) Hy pergrid compo-
ations won’t occur. nents, making those applications ready to be
federated.
Confidentiality of Assets The reason behind this goal is simple: the
Whatever abuse a user might perform on the Web has the critical mass of users, and interest
world’s data should be limited to the exposed has increased in adding 3D immersion to Web
data only, and shouldn’t compromise the confi- applications. Promising emerging technologies
dentiality of assets that aren’t exposed. for adding interactive 3D elements to regular
Hypergrid-facing asset servers open another Web applications include Flash; Unity 3D; the
door to the world’s assets that must be carefully combination of JavaScript, WebGL, and Web-
secured. Access to assets via Hypergrid asset Sockets; and server-side streaming. The future
28 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Hypergrid: Architecture and Protocol for Virtual World Interoperability
of virtual worlds will likely include various 2. R.T. Fielding and R.N. Taylor, “Principled Design of the
viewers that run on Web browsers — not only Modern Web Architecture,” ACM Trans. Internet Tech.,
the popular 2.5D Flash applications, but well vol. 2, no. 2, 2002, pp. 115–150.
beyond. Therefore, the Hypergrid takes the Web
design principles and the server-side of Web Cristina Videira Lopes is an associate professor with the
applications as design invariants, 2 carefully Department of Informatics in the School of Infor-
staying away from optimizations and simplifi- mation and Computer Sciences at the University of
cations that would compromise applicability to California, Irvine. Her research interests include infor-
the Web in general. mation retrieval for aspect-oriented programming;
software engineering for large-scale systems; ubiqui-
Acknowledgments tous computing, including lightweight software acous-
Melanie Thielker contributed invaluable input to the tic modems that can be played and decoded in small
Hypergrid’s design, especially its security. The Open- portable devices such as cell phones; and massive
Simulator community has made the Hypergrid a reality; multiuser online (MMO) virtual worlds and their appli-
their feedback and enthusiasm is what matured the Hyper- cations beyond gaming. She’s a core contributor to
grid from an experiment to a viable interoperability archi- the OpenSimulator project, a server-side virtual world
tecture. This work is partially supported by NSF grant platform. Lopes has a PhD from Northeastern Univer-
IIS-0808783. sity. She’s a senior member of IEEE.
References
1. J. Rosenberg et al., “SIP: Session Initiation Protocol,” Selected CS articles and columns are also available
IETF Internet draft, work in progress, June 2002. for free at http://ComputingNow.computer.org.
NEW {EssentialSets} Available:
ESSENTIAL INDUSTRIAL Edited by TC AE Elisardo Antelo, these EssentialSets
IMPLEMENTATIONS OF surveys the industrial design of floating-point
FLOATING-POINT UNITS units during the last decade. This EssentialSet is
DURING THE LAST DECADE: broken into two volumes, sold separately.
VOLUMES 1 & 2
Order Online: computer.org/store.
SEPTEMBER/OCTOBER 2011 29
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
An Entity-Component Model
for Extensible Virtual Worlds
The open source realXtend project has developed a freely available open
source virtual world platform that lets anyone create 3D applications.
RealXtend is fully implemented in the new Tundra SDK and in an add-on
for the OpenSimulator server. The framework treats fundamental elements
of virtual worlds (such as support for avatars) as an add-in functionality, so
the overall architecture can accommodate a wider range of virtual worlds.
Attribute values are automatically synchronized among the participants in a
networked environment. A core API provides basic functionality for GUIs,
controller input, audio, and means for 3D scene manipulation for application
code.
S
Toni Alatalo ince 2007, the realXtend project doc/ blob/ma ster/acm _ mu lt i med ia/
____________________________
Playsign and realXtend Association has developed a freely available overview.rst.) The Tundra SDK, which is
_________
open source virtual world plat- built entirely using the entity-component
form that lets anyone create their own model, is a true platform that doesn’t
3D applications. RealXtend began as get in the way of application develop-
a collaboration between several small ers; they can create anything, from
companies coordinating to develop a a medical simulator for teachers to
common technology base that they action-packed networked games —
then applied in different application and always with a custom interface
fields, including virtual worlds, video- that exactly fits the application’s pur-
games, and educational applications. pose. We treat seemingly fundamen-
The realXtend Association was founded tal elements of virtual worlds (such as
in early 2011 to coordinate further, support for avatars) as an add-in func-
open development. tionality, so the overall architecture
In this article, we describe the real- can accommodate a wider range of vir-
Xtend project and particularly focus tual worlds.
on its entity-component-action (ECA) To demonstrate the feasibility of our
architecture, which provides a general generic scene-modeling approach, we
extensibility mechanism for building use Tundra to develop a growing collection
3D virtual worlds. (For a generic intro- of example scenes in a directory avail-
duction to the platform and the mod- able on GitHub (https://github.com/
ules, see https://github.com/realXtend/ realXtend/naali/blob/tundra/bin/scenes).
___________________________
30 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
An Entity-Component Model for Extensible Virtual Worlds
This article presents two example scenes to via a URL reference, and the Naali GUI supports
illustrate how the ECA model works in practice. drag and drop of 3D models from webpages
In the first example, we implement a Second like the Google 3D Warehouse to the 3D virtual
Life-style avatar that runs on both the server world scene. In realXtend, a virtual world can
and the clients. The second example is a presen- be snapped together from existing components
tation application that lets a presenter control like Lego bricks and viewed instantly.
the view for the other participants as the pre- Another architectural goal of realXtend
sentation proceeds. is flexible editing of virtual worlds — that is,
Our approach demonstrates how vir tual users can edit locally and publish their creations
world architectures can be simple and practical, later. In contrast, all edits and additions in Sec-
yet powerful and truly extensible. ond Life happen on remote servers, and the cli-
ent application is no more than an interface to
RealXtend Architectural Goals server-side functionality. Naali/Tundra is com-
Similar to several other 3D virtual world plat- pletely stand-alone, without the complexity of
forms, the realXtend project has taken a client– setting up a separate server for local editing
server approach. A browser-like client called a as with OpenSimulator (http://an.org/opensim/
viewer renders content, enabling end users to see usbkey). This is similar to how end users can
_____
and manipulate a 3D window into a virtual world author an HTML webpage locally by just editing
where the content itself is stored and shared on the HTML, CSS, and JavaScript sources before
a (typically remote) server. The realXtend proj- publishing them simply by copying the files
ect has developed an open source viewer called over to a Web server. Tundra can similarly open
Naali (the Finnish word for the arctic fox), which scenes from local files to show the 3D view,
references the project’s Finnish origins and the which streamlines object and scene creation so
open source Firefox Web browser because Naali that changes to texture images, 3D models, and
aims for similar widespread availability as a scripts update immediately in the final form
browser for virtual worlds. The Naali viewer can without any uploads to a virtual world system.
connect to Second Life, OpenSimulator, or real- Our project’s f inal architectural goal is
Xtend’s own Tundra server and can run on Win- extensibility — the ability to dynamically add
dows, Linux, Mac, and some mobile platforms. or remove functionality to a virtual world plat-
One architectural goal of the realXtend project form to meet specific applications’ needs. The
has been to build entirely on open standards and approach is similar to Web browsers, which also
open source software to remove the roadblock download both data and executable code from
of proprietary software and pave the way for servers so that applications can implement cus-
widely used 3D virtual worlds. To this end, Naali tom behavior in the client. This makes realXtend
and Tundra use HTTP, Collada (Collaborative a generic platform; you can use the same viewer
Design Activity), Extensible Messaging and executable to connect to any server, when the
Presence Protocol (XMPP), and open source scene and associated custom JavaScript code is
software such as OGRE 3D, Qt, OpenSimulator, downloaded from the Web and executed locally
and Blender. We can see an immediate benefit to implement a specific behavior.
in that realXtend supports 3D geometry in the
typical polygon mesh format, so existing game Extensible Scene Architecture
characters, CAD models, and building models The extensible scene model is independent of
can be used by exporting them from packages any particular virtual world platform imple-
such as 3ds Max, Maya, and Blender. RealXtend mentation. A scene is defined by its entities;
has had this capability since the initial proto- nothing is hardcoded about the scenes at the
type. Second Life (a widely used but proprietary platform level. This differs essentially from the
3D virtual world), on the other hand, has been current OpenSimulator paradigm when using
limited to its own special representation using the Second Life protocol, where the model is
primitive graphical objects (prims) and still only largely predefined and hardcoded into the plat-
has meshes in beta testing as of summer 2011. form. In Second Life, a certain kind of land
RealXtend also allows reuse of existing mod- (a height-map-based terrain with altitude-based
els and scripts from Web libraries. Any model texturing) always exists, and the sea, sky, and
asset in realXtend can be included in a scene sun are always there as well. And each client
SEPTEMBER/OCTOBER 2011 31
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Client Entity-actions: Server Avatars Aren’t Part of the Platform
move [dir], stop Avatars are graphical representations of the
user within the virtual world. It might seem at
first that the concept of an avatar is integral to
3D virtual worlds. Second Life’s avatar proto-
col is hardcoded into the platform. Yet, many
Movement and
animation state virtual worlds, simulation platforms, and games
Reads input Creates the AVs
Applies animations sync with ECs Physics don’t have a single character as the locus of
Movement code control. For instance, map applications or astro-
nomical simulations are about efficient naviga-
Placeable tion and time control of the whole space, not
InputMapper AnimationController RigidBody about moving your presence around, and real-
time strategy games involve controlling several
AvatarAppearance units, similar to board games like chess. Thus,
we argue that avatars shouldn’t be part of the
base platform because many simulations don’t
Figure 1. Avatar architecture. This example uses a client (green) require them. Of course, a generic platform
and a server (brown). The filled boxes represent entity-component- must still allow the implementation of an avatar
actions on the client, server, or shared by both. The arrows add-in functionality.
represent network messages made as entity-action calls from Here, we describe a proof of concept imple-
the client side to the server. mentation of avatars as add-ins using the real-
Xtend ECA model. Application XML and usage
connection is always assigned to a single ava- information are available at https://github.com/
tar to which the user’s controls are mapped.1 We realXtend/naali/tree/tundra/bin/scenes/Avatar.
_________________________________
argue that there is no need to embed assump- We split avatar functionality into two parts
tions about the world’s features in the base plat- (see Figure 1). The first part governs the visual
form and protocols. appearance and related functionality to mod-
Our Naali viewer uses the ECA model as a ify the look and clothing as well as the use of
basis for constructing extensible scenes. We animation for communication. The second part
adapted this model from contemporary game- gives every user connection a single entity as
engine architectures.2 Entities are unique iden- the point of focus and control. The default inputs
tities, with no data or typing. They aggregate from arrow keys and the mouse are mapped to
components, which can be of any type and store move and rotate the avatar. For this discussion,
arbitrary data. Applications built using Naali although we cover the basics of avatar appear-
can add their own components so they have the ance, we focus on the latter control functionality.
data they need for their own functionality. The To give every new client connection a des-
code that handles the data exists in preinstalled ignated avatar, we implement the server-side
custom modules or in scripts loaded at run- functionality in JavaScript (see Figure 2). Upon
time as a part of the application data. To get a a new connection, this script creates a new ava-
matching server counterpart where the scene is tar entity and attaches these components to it:
entirely built with entity components, we added EC_Mesh for the visible 3D model and an asso-
the Tundra server module to the Naali codebase ciated skeleton for animations; EC_Placeable
and a new protocol without application-level for the entity to be positioned in the 3D scene;
assumptions. Tundra consists of both viewer EC_AnimationController to change and syn-
and server executables. chronize the animation states; and EC_Script to
The Tundra platform provides basic func- implement a single avatar’s functionality. Differ-
tionality for all ECAs: persistence, network ent parts of the same script are executed on the
synchronization among all the participants via client, where it adds two additional components:
a server, and a user interface for manipulating a new camera that follows the avatar and a key-
components and their attributes, and eventually binding to toggle between camera modes.
will support security. In addition, Tundra intro- A second script for an individual avatar
duces the concept of entity actions, a simple (simpleavatar.js) adds additional components:
form of remote procedure call. AvatarAppearance for the customizable looks;
32 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
An Entity-Component Model for Extensible Virtual Worlds
function serverHandleUserConnected(connectionID, userconnection) {
var avatarEntity = scene.CreateEntity(scene.NextFreeId(),
["EC_Script", "EC_Placeable", "EC_AnimationController"]);
avatarEntity.Name = "Avatar" + connectionID;
avatarEntity.Description = userconnection.GetProperty("username");
avatarEntity.script.ref = "simpleavatar.js";
// Set random starting position for avatar
var transform = avatarEntity.placeable.transform;
transform.pos.x = (Math.random() - 0.5) * avatar_area_size + avatar_area_x;
transform.pos.y = (Math.random() - 0.5) * avatar_area_size + avatar_area_y;
transform.pos.z = avatar_area_z;
avatarEntity.placeable.transform = transform;
}
Figure 2. JavaScript source code. The avatarapplication.js code creates a new avatar entity and attaches several
components to it.
RigidBody for physics; and on the client side, earlier realXtend prototype that didn’t have the
an InputMapper for user input. Entity actions ECA model, but it is reused in this demo as is.
ensure the avatar moves according to the user A more generic and customizable appearance
controls. These actions are commands that can system could be implemented with the ECAs,
be invoked on an entity and executed either but that’s outside the scope of this example.
locally in the same client or remotely on the The division of work between the clients and
server, or on all connected peers. For example, the server we describe here isn’t the only possi-
the local code sends the action “move(forward)” ble configuration. With Tundra SDK, we use
to be executed on the server when the user the same core code and A PI for the ser ver
presses the up-arrow on the client. The built-in and the clients, making it simple to reconfig-
EC_InputMapper component provides trigger- ure what is executed where. This model of cli-
ing actions based on input, so the avatar code ents only sending commands and the server
needs to register only the mappings it wants. doing all the movement is identical to that of
The server maintains a velocity vector for the the Second Life protocol. It is suitable when
avatar and applies physics for it. Using ECA trust and physics are centralized on a server.
attributes, the resulting position in the trans- A drawback is that user control responsiveness
form attribute of the component Placeable is can suffer from network lag. We can already
automatically synchronized with the generic use the physics module on the client end too,
mechanism, so the avatar moves on all clients. which can allow movement code to run locally
The server also sets the animation state to either as well.
“stand” or “walk” based on whether the avatar Along with the ability to run custom code
is moving. All participants run common ani- in the client, it’s easy to extend avatar-related
mation update code to play back t he walk functionality. For example, in one project
animation while moving, calculating the cor- for schools, we made it possible for avatars to
rect speed from the velocity data from the phys- carry objects around as a simple means for 3D
ics on the server. scene editing. Another possibility is to further
These two parts are enough to implement augment the client with more data that’s syn-
basic avatar functionality using the ECA model. chronized for animations — for instance, the
This proof of concept implementation includes full skeleton for motion capture or machine-
369 lines of JavaScript code in two files. The vision-based mapping of the real body to the
visual appearance comes from a preexisting avatar pose. Our open source Chesapeake Bay
AvatarAppearance component, which reads an watershed demo scene includes minigames with
XML description with references to the base customized game character controls, includ-
meshes used and individual morphing values ing flying as an osprey with the ability to dive
that the user sets in an editor. Implemented in to catch fish. We implemented these using the
C++, it uses the realXtend avatar model from an human-avatar functionality as a starting point,
SEPTEMBER/OCTOBER 2011 33
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
and consider a situation in which we added
multiple views for the presentation, such as a
slide and outline view, or where we animate
the presentation content, add voice and text
chat components to let users communicate with
other viewers, or add annotations to the presen-
tation. For simplicity, however, we only demon-
strate a basic application here.
Regardless of the presentation view, the
presenter typically needs the same controls. In
Second Life, avatar controls are fixed, and to
control a presentation, users might need to cre-
ate a presentation sequence object with mouse
Figure 3. Example shared presentation. Two Naali clients stand click controls to press virtual buttons. In real-
nearby and view the presentation stage of the TOY system, an Xtend, custom controls in the client can directly
open source learning environment for the Future School of Finland change the shared scene state.
project. The one on the left just added a webpage to the stage and For the implementation in realXtend ECA,
is currently carrying the object. the simplest way to get a shared, synchronized
view of the presentation slides is to use a static
camera that shows a single webpage view. It
then modified them according to the different then suffices for the server to change the cur-
animal characteristics. rent page on that object for everyone to see
it. We could implement this in a 2D GUI, but
A Collaborative Presentation Tool we do it in the 3D scene here to illustrate its
To demonstrate an entirely different use of the extensibility.
ECA framework, we consider an application To implement this application, we add a new
that, in its simplest form, implements collab- nonspatial entity called Presentation, an appli-
orative presentations in which one user controls cation that’s globally available in the scene.
sequencing through a collection (of webpages (The Tundra chat application is implemented
or PowerPoint slides) while others watch. The in a similar fashion.) To display webpages, we
presentation tool lets the presenter control the need a few basic components: EC_Placeable to
position in the prepared material, for example, have something in the scene, EC_Mesh to have
to select the currently visible slide in a slide- geometry (such as a plane) on which to show
show. In a local setting where everyone is in the the slides, and WebView to render HTML from
same physical space, it’s simply about choosing URLs. For our custom functionality, we add two
what to show via the overhead projector. In a additional components: EC_DynamicComponent
remote distributed setting, there must be some for custom data and EC_Script to implement
system to get a shared view over the network. the user interface for presentation controls.
A shared, collaborative view of a set of 2D As data, we need a list of URLs and an index
webpages could be implemented without real- number for the current position. This custom
Xtend technology by using regular Web brows- data becomes part of the scene data and is
ers with HTML, JavaScript, and some backend automatically stored and synchronized among
server logic. Our goal here is to illustrate the the participants. The EC_Script component is
use of the ECA model and automatic attribute a reference to JavaScript or Python code that
synchronization for developing custom func- implements the logic.
tionality. In a minimal implementation of shared We have two options for handling the user
collaborative presentations, we can use ECA input: either handle input events and modify
without using avatars or geography. the state correspondingly directly in the client
Alternatively, because it’s easy to do, we code, or send remote actions like in the avatar
could add those components back in to build example. Here, we use remote actions again so
shared presentations such as the one in Figure 3, we can use the server as a security broker and
where different avatars see the presentation to get a similar design to compare with the ava-
from different viewpoints. We could go further tar example.
34 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
An Entity-Component Model for Extensible Virtual Worlds
The client-side code maps right-arrow and talks with some Sirikata developers, we con-
spacebar keys to SetPresentationPos(index+1) cluded that they aimed to keep the base level
and so forth. The server can then check if the clean from high-level functionality, but that
caller has permission to do that action — for capabilities such as attribute synchronization
example, in presentation mode, only the des- would be desirable in application-level support
ignated presenter can change the shared view. scripts.
Then, if the presentation material is left in the The Naali ECA model borrows the idea of
scene for later use, control can be freed for using aggregation and not inheritance from the
anyone. game-engine literature. 2 Automatically syn-
The index attribute is synchronized for all chronizing attribute data and using the same
participants so the outline GUI can update JavaScript code on both the client and server side
accordingly. To add an outline view, we can add is inspired by a gaming-oriented virtual world
a 2D panel with thumbnails of all the slides and platform called Syntensity (www.syntensity.
highlight the current slide. For free browsing, com). The difference is that the entities in Syn-
__
clicking on a thumbnail can open a new win- tensity exists only on the scripting level, and
dow with that slide, while the main presentation basic functionality such as object movement is
view remains. hardcoded in the Sauerbraten/Cube2 first-person
Thus, we have a simple, complete shared shooter platform.
presentation application implemented on top In Naali, all functionality is now imple-
of a generic ECA model virtual world plat- mented with the ECAs, so the same tools work
form architecture. Source code of this model’s for graphical editing, persistence, network sync
implementation is available at https://github.
___________ identically for all data, and the like. The document-
com/realXtend/naali/tree/tundra/bin/scenes/ oriented approach of having representing
SlideShow, with the additional feature that it
_______ worlds externally as files has precedent in 3D
automatically creates the presentation when a file format standards such as VRML, X3D, and
premade slideshow (such as a PowerPoint file) is Collada. Unlike those, the realXtend files don’t
added to the scene. directly include 3D geometry, but they describe
a scene using URL references to external assets,
Comparing Virtual such as meshes in the Collada format. Essen-
World Architectures tially, these files describing scenes are a mech-
Simulations have long demonstrated that ava- anism for application-specific custom data,
tars and geography aren’t always required. For which is automatically synchronized over the
example, the open source Celestia universe Internet. They have script references that imple-
simulator (www.shatters.net/celestia) lets users ment the applications’ functionality, similar to
view 100,000 stars but doesn’t have any hard- the way HTML documents contain JavaScript
coded land or sky. Nor are we the first to pro- references. This isn’t specified in the file format;
pose a generic component model for virtual instead, it’s how the bundled JavaScript compo-
world architectures. For example, the NPSNET-V nent works.
system is a minimal microkernel on which arbi-
trary code can be added at runtime using a Java Status of realXtend Implementations
Virtual Machine.3 A contemporary example is Two generations of realXtend technology are
the meru architecture from the Sirikata proj- currently available. The original prototype, a
ect, where a space server only knows the object General Public License (GPL) licensed fork of the
locations. Separate object hosts, either running Second Life viewer, has become mostly irrel-
on the same server or any client/peer, can run evant as the newer Naali viewer has matured.
arbitrary code to implement the objects in the We built it from scratch, and it’s available under
federated world.4,5 Messaging is used exclu- the Apache 2 license and is the modular and
sively for all object interactions.6 extensible platform. Taiga (which combines
The idea with the ECA mechanism in Naali, OpenSimulator and the realXtend add-on for it)
rather, is to lessen the need to invent particular is a continuation and refinement of the original
protocols for all networked application behavior server project (BSD license). The latest addition
when, for many simple cases, using automatically to the new generation, Tundra, completes the
synchronized attributes suffices. In preliminary Naali code base with server functionality and
SEPTEMBER/OCTOBER 2011 35
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
a new protocol built purely on the ECA design. functionality on public servers. It also serves
It has the same API on the server and clients, as an example of how a generic EC approach to
resulting in a powerful toolkit for networked virtual worlds functionality can be simple, yet
application development. All the functionality practical.
is configured by the applications, but the plat-
form has the building blocks, such as playback he generic EC architecture was proposed to
of 3D skeletal animations and physics colli-
sions in the efficient C++ libraries — Ogre3D for
T the OpenSimulator core and accepted as the
plan of record in December 2009.7 We’ve only
graphics and Bullet for physics. In this way, the begun to experiment with the actual refactor-
JavaScript-driven logic can still perform well. ing of OpenSimulator scene code to be built
The Naali viewer has matured and been with EC. However, EC can be utilized with the
deployed to customers by some of the develop- Naali client communicating with the Open-
ment companies. It’s a straightforward modu- Simulator servers running the realXtend add-
lar C++ application with optional Python and on (modrex) in a limited fashion. These servers
JavaScript support. The Qt object metadata still assume the hardcoded Second Life model,
system is utilized to expose the C++ internals but developers using Naali can add additional
automatically. This covers all modules, includ- arbitrary client-side functionality and have the
ing the renderer and user interface as well as all data automatically stored and synchronized
the ECs. The QtScript library provides this for over the Internet via OpenSimulator. Entity
JavaScript support, and PythonQt does the same actions are currently not implemented in this
for Python. There is also a QtLua so that Lua OpenSimulator realXtend add-on.
support can be added. Thanks to the Ogre3D The realXtend platform doesn’t yet solve all
graphics engine, Naali runs on various plat- problems related to virtual world architectures.
forms, such as the N900 mobile phone with Naali doesn’t address scaling at all, nor does
OpenGL ES, and on powerful PCs with multiple it support federated content from several pos-
video outputs with the built-in CAVE render- sible untrusted sources. We started by provid-
ing support. An experimental WebNaali client, ing power at a small scale to let end users easily
written in JavaScript to run in a Web browser, develop rich interactive applications. Another
does EC synchronization over WebSockets and important missing element in our current EC
rendering with WebGL. synchronization architecture is security, such
The generic ECA architecture is imple- as a permission system. Support for permissions
mented in Naali and is in use throughout in the was just implemented that is similar to Synten-
Tundra SDK, which complements the original sity where the server can control if and when
Naali code base with a server module (http://
____ clients are allowed to modify entity attributes.
realxtend.blogspot.com/2010/11/tundra-project. In the future, we look forward to continu-
html). This configuration enables Naali to run
___ ing collaboration with communities such as
stand-alone for local content authoring or for OpenSimulator and Sirikata to address trust
single-user applications, but it can also be used and scalability issues. OpenSimulator is already
as a server instead of using OpenSimulator. used to host large grids by numerous people,
Tundra doesn’t use LLUDP; instead, all basic and the Sirikata architecture seems promis-
functionality is achieved with the generic EC ing for the long run.4,5 Also, Intel Research has
synchronization. recently demonstrated how multiple servers can
For the transport, we use a new protocol be used to host a single scene for thousands
called kNet, which can run on top of either UDP of interacting users using OpenSimulator.8 We
or TCP (http://bitbucket.org/clb/knet). kNet is will see whether that design can either be easily
similar to eNet, but it performed better in tests ported to the Tundra server or better utilized for
with regard to flow control. The Tundra server realXtend as is by using OpenSimulator.
lacks many Second Life specific features of the Applications implement functionality against
more advanced OpenSimulator, such as running the Naali/Tundra core API. It’s role is simi-
untrusted user-authored scripts and combining lar to the W3C Document Object Model (DOM)
multiple regions to form a large grid. However, standard in HTML browsers. We’re currently
Tundra is already useful for both local author- freezing the API 1.0 version so that applica-
ing and deploying applications with custom tions developed now will continue to work in
36 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
An Entity-Component Model for Extensible Virtual Worlds
upcoming releases. We have documented this Educational & Serious Games (CGAMES), 2010; http://
___
work at www.realxtend.org/doxygen. This API sing.stanford.edu/pubs/cgames10.pdf.
______________________
will be reviewed for ease of development, for 7. A. Frisby, “[Opensim-dev] Refactoring SceneObjectGroup —
challenges in scalability and security, and with Introducing Components,” 11 Dec. 2009; http://lists.berlios.
regard to interoperability and standardization. de/pipermail/opensim-dev/2009-December/008098.
________________________________
We hope our approach is taken into consider- html.
___
ation both in future OpenSimulator development 8. D. Lake, M. Bowman, and H. Liu, “Distributed Scene
and upcoming standardization processes — Graph to Enable Thousands of Interacting Users in a
for example, if the IETF Virtual World Region Virtual Environment,” Proc. 3rd Int’l Workshop Mas-
Agent Protocol (V WRAP) or IEEE Metaverse sively Multiuser Virtual Environments, ACM Press,
standardization efforts choose to address in- 2010; www.pap.vs.uni-due.de/MMVE10/papers/mmve2010_
world scene functionality. We’ll continue to submission_7.pdf.
__________
develop the realXtend platform and applications
on top of it. Anyone is free to use it for their Toni Alatalo is the CTO of Playsign and the current lead
needs, and motivated developers are invited to architect of the open source realXtend Association.
participate in the effort, which is mainly coor- His research interests include agile game development
dinated online. and playful information systems. Alatalo has studied
and worked at the Department of Information Process-
Acknowledgments ing Sciences at the University of Oulu. Contact him at
Work on this new version of the realXtend platform was toni@playsign.net.
___________
initially led by Ryan McDougall, who was working as
the principal architect in the beginning of the effort. The Selected CS articles and columns are also available
Tundra server and protocol is designed by Jukka Jylänki for free at http://ComputingNow.computer.org.
at Ludocraft Oy, where most of the core development has
occurred. I was initially responsible for the scripting API
development and later for coordinating the overall open
source development. My work for the realXtend Associa-
tion is now sponsored by the Center for Internet Excellence
(CIE) at the University of Oulu.
References stay connected.
1. J. Bell, M. Dinova, and D. Levine, “VWRAP for Vir-
tual Worlds Interoperability,” IEEE Internet Computing,
vol. 14, no. 1, 2010, pp. 73–77.
2. M. West, “Evolve Your Hierarchy: Refactoring Game
Entities with Components,” 5 Jan. 2007; ht t p:// ____
cowboyprogramming.com/2007/01/05/evolve-your-
heirachy.
_____
3. A. Kapolka, D. McGregor, and M. Capps, “A Unified
Component Framework for Dynamically Extensible
Virtual Environments,” Proc. 4th Int’l Conf. Collabora-
tive Virtual Environments (CVE 02), ACM Press, 2002,
pp. 64–71.
Keep up with the latest IEEE Computer Society
4. D. Horn et al., “Scaling Virtual Worlds with a Physi-
publications and activities wherever you are.
cal Metaphor,” IEEE Pervasive Computing, vol. 8, no. 3,
2009, pp. 50–54. TM
| @ComputerSociety
| @ComputingNow
5. D. Horn et al., To Infinity and Not Beyond: Scaling Com-
| facebook.com/IEEEComputerSociety
munication in Virtual Worlds with Meru, tech. report | facebook.com/ComputingNow
CSTR 2010-01 5/11/09, Stanford Univ., 2010; http://hci.
| IEEE Computer Society
stanford.edu/cstr/reports/2010-01.pdf.
______________________ | Computing Now
6. B. Chandra et al., “Emerson: Scr ipting for Feder-
ated Virtual Worlds,” Proc. 15th Int’l Conf. Computer
Games: AI, Animation, Mobile, Interactive Multimedia,
SEPTEMBER/OCTOBER 2011 37
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Open Wonderland:
An Extensible Virtual
World Architecture
Open Wonderland is a toolkit for building 3D virtual worlds. The system
architecture, based entirely on open standards, is highly modular and designed
with a focus on extensibility. In this article, the authors articulate design goals
related to collaboration, extensibility, and federation and describe the Open
Wonderland architecture, including the design of the server, the client, the
communications layer, and the extensibility mechanisms. They also discuss the
trade-offs made in implementing the architecture.
V
Jonathan Kaplan irtual world technology is on the OpenSimulator (http://opensimulator.org),
and Nicole Yankelovich verge of a phase change from an represent a new genre of virtual world
Open Wonderland Foundation interesting experiment to a large- technology that has the potential for
scale phenomena. Although today’s large-scale deployment in which orga-
most popular virtual worlds such as nizations will host their own virtual
Second Life (http://secondlife.com) and worlds that will be federated together
Active Worlds (http://activeworlds.com) into an enhanced 3D Web. Open Wonder-
have fallen short of expectations for land follows a large body of work on
collaboration and education, we predict collaborative virtual environments,
that in the future, most Internet sites starting with research systems from
will engage visitors with 3D experi- the early 1990s such as Diamond Park1
ences. We base this belief on factors and the Distributed Interactive Virtual
such as broadband pervasiveness, the Environment (DIVE).2
advent of voice over IP (VoIP) for home The Open Wonderland architecture
users, and the popularity of massively defines a common foundation for build-
multiplayer online games, which dem- ing a diverse ecosystem of such worlds,
onstrate the power of real-time collab- each with different features and capa-
oration in 3D environments. bilities. The Open Wonderland project,
Open Wonde r l a nd ( ht t p://
_____ which began at Sun Microsystems in
openwonderland.org), an open source
_______________ 2007 as Project Wonderland, has been
toolkit for creating 3D virtual worlds, completely community-driven since
along with a few other systems such as January 2010. Although the initial moti-
Open Croquet (http://opencroquet.org) and vation for creating the toolkit was to
38 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Open Wonderland: An Extensible Virtual World Architecture
support business collaboration, the project’s mis-
sion quickly broadened to encompass education,
training, simulation, and visualization. Here,
we examine Open Wonderland’s architecture
and design.
Design Goals
In designing the Open Wonderland architecture, Virtual piano MIT TEALSim physics Hospital privacy screen
we had three main goals: enabling collabora-
tion with a focus on synchronous interaction,
providing an extensible toolkit based on open
standards, and putting in place the infrastruc-
ture for federation to enable the 3D Web.
Collaboration
Our goal with regard to collaboration was to Animated code editor CMU Alice integration Marble rollercoaster
enable all the types of synchronous collabora-
tion possible with Web-based tools while add- Figure 1. Example extensions. By building an extensible toolkit
ing the benefits inherent to 3D interaction. rather than a fixed-feature environment, we aimed to let
In particular, we wanted to support informal developers quickly build highly customized worlds with task-specific
collaboration. Many of the same features that applications.
support formal collaboration, such as immer-
sive audio, also apply to informal interaction. data, or otherwise disrupt the real work taking
One important advantage of a 3D space is that place in the virtual world.
it provides an intuitive way to organize multiple,
simultaneous conversations, something not pos- Extensibility
sible with current audio- or video-conferencing Our goals for collaboration led us to focus
technology. the technical design on extensibility. While
Immersive audio coupled with the visual 3D we could identify certain features — such as
context also enhances collaboration by pro- audio — that were relevant to all collaborations,
viding a strong sense of other people’s pres- making the environment useful for real work
ence in the virtual world.3 As we know from required that it be customizable for different
our research,4 audio is perhaps the single most tasks. Each use case we looked at benefited from
important factor in successful remote collabora- new interactive applications, visualizations, and
tion. Given this, we aimed to create an architec- integration with different data (see Figure 1).
ture that treats high-fidelity, immersive audio By building an extensible toolkit rather than
as a core toolkit component. a fixed-feature environment, we aimed to let
We made it a design priority to support real developers quickly build highly customized
work activities with both legacy applications worlds with task-specific applications.
and collaboration-aware applications designed To enable this broad range of extensions,
specifically for multiple users. If an application we focused on a modular architecture based on
is in the world, it is shared, unless a user speci- open source Java components. We structured
fies otherwise. To make sharing as seamless as the project with a small set of core services that
possible, we wanted to enable users to drag- manage the 3D world, including authentica-
and-drop content and automatically launch the tion, networking, content management, and cli-
correct application to display that type of data ent rendering. Beyond these core services, we
in the world. implemented most of the features in modules.
Lastly, we wanted to provide enterprise- Our extensive use of modules to implement
class security and authentication. For business core features — including avatars, audio, and
and education applications, users must know shared applications — meant that we needed
people’s identity. It is also important to secure a comprehensive set of extension points. We
objects in the world so that unauthorized users knew we would require extension at many dif-
can’t change important documents, delete crucial ferent levels, from adding new menus in the
SEPTEMBER/OCTOBER 2011 39
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Wonderland client
Rendering MT game JMonkeyEngine
Web administration
Input/events Collision Physics
Core
services Avatars Cell HUD
DnD Security Audio
Networking Communications
HTTP HTTP
Darkstar App data SIP/RTP(UDP)
(TCP) (TCP)
RESTful Web service APIs
Web server
Shared Voice
app server bridge
Web-based management Darkstar
server
Module Single Asset Service
manager sign-on storage manager Control channels (TCP)
Service nodes
Figure 2. Open Wonderland network diagram. We show communication between the system
components. The Wonderland client communicates via HTTP with the Web server. Using a number
of task-specific protocols, the client communicates with other services including the game server and
voice bridge.
client to pluggable authentication mechanisms with different purposes, features, and code. Cli-
in the server to integrating new services such ent browsers will let users easily move between
as Extensible Messaging and Presence Protocol servers, downloading both content (3D artwork)
(XMPP; http://xmpp.org) chat servers. and behavior (mobile Java code). Unlike the
Our final extensibility goal was to enable Web, these worlds’ focus will be on synchro-
integration with external data. We started by nous communication, and as such, they’ll need
choosing a set of well-supported open stan- richer, more extensible programming interfaces
dards, including Collada (Collaborative Design and network protocols, which can handle 3D
Activity; http://collada.org) for graphics and visualization, rich presence information, real-
the Session Initiation Protocol (SIP; http://ietf. time application sharing, and full multimedia
org/rfc/rfc3261.txt) for audio. We also wanted collaboration.
to make sure that developers could integrate
data from other sources — for instance, from Wonderland Architecture
open Web services to proprietary databases. Fig u re 2 shows Open Wonderla nd’s va r i-
It was especially important that developers ous components and how they communicate.
be able to use existing Java libraries to access Wonderland uses a client–server model to create
these services. collaborative virtual worlds. In practical terms, a
world is a virtual space with its own 3D coordi-
Federation nate system that clients can connect to in order
Our long-term goal for the Wonderland toolkit to collaborate. Wonderland is written entirely
is to enable a new type of 3D Web. We imag- in the Java programming language. The cli-
ine a set of loosely connected servers — like ent provides a browser that turns these shared
the World Wide Web — each presenting worlds services into a 3D view of the environment.
40 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Open Wonderland: An Extensible Virtual World Architecture
This includes rendering graphics, downloading at http://reddwarfser ver.org). Darkstar pro-
and caching content, responding to user inter- vides a server platform specifically designed
actions, and reacting to server messages. for online games, including “serious games”
The client and server communicate using such as the Wonderland environment. Unlike
several networking protocols optimized for dif- a Web server, it is optimized for low latency
ferent data types: rather than high throughput. The Darkstar
server divides all actions into short tasks that
Web services for authentication, download- it executes within a transaction. It immediately
ing code, and world assets such as 3D mod- writes out the results to an internal database,
els and textures; guaranteeing that no state is lost even during
custom TCP-based protocols for communi- server crashes. Wonderland uses the Darkstar
cating world data such as object properties server to track the frequently updated state of
and position; live objects in the world. This includes prop-
SIP and RTP for audio; and erties such as the location for each object and
multimedia streaming protocols for video, avatar. Darkstar also provides an abstract com-
application sharing, and screen sharing. munication mechanism, allowing a client to
send simple messages to the server and the
Using multiple communications channels allows server to send messages to any subset of clients
each protocol to be optimized for the type connected to that same server.
of data being sent between the client and the JVoiceBridge (http://tinyurl.com/jvoicebridge)
server. is a pure Java audio-mixing application that
provides server-side mixing of high-fidelity,
Server Components immersive audio. It runs as a separate Wonder-
The Wonderland server is based on a set of four land server that mixes SIP audio for multiple
cooperating services. Each service is a separate users, based on where in the virtual space they
Java application with its own networking and are. Objects in the world, such as microphones
storage mechanisms. Designing these as sepa- and cones of silence, can also affect audio.
rate services enables increased flexibility and JVoiceBridge communicates directly with the
scalability: typically, we deploy all ser vices Darkstar server over a private channel to keep
on a single machine, but Wonderland admin- all the audio in sync with the world’s state as
istrators can spread services across multiple users move around or are added and removed.
machines to increase scalability. The shared application server (SAS) is the
The Web administration server is the main final standard server component. The SAS runs
coordination point for the various services. on Linux or Solaris systems to allow server-
This server is based on the open source Glass- hosted application sharing (see Figure 3). In this
fish Java EE Application server (http://glassfish. model, an unmodified X Windows application,
java.net). The core Wonderland features such
______ such as Firefox or Open Office, runs inside a
as authentication and asset management are custom X Windows server. This server broad-
implemented as Java EE Web services. The Web casts application updates in the form of images
server acts as a central management console, to each Wonderland client with an avatar in the
providing Web-based management of all ser- application’s range. Clients reconstruct these
vices in the system, regardless of which server images into a local view of the application that
they are running on. Another important service users can see and interact with. These legacy
is a token-based single sign-on mechanism. applications are designed for a single user, so
After users authenticate to the Web server, a control-passing system ensures that only one
they receive a token that they can give to other user makes changes to the application at a time.
services. Those services then use the token to This is necessary only for legacy applications.
authenticate the client when it connects over Multiuser collaboration-aware applications
different channels. written specifically for Wonderland run locally
The Darkstar server is based on the Project on each client and send change events through
Darkstar technology, also developed at Sun. the Darkstar server, allowing multiple users to
(Project Darkstar has subsequently become a interact simultaneously while using minimal
community project known as RedDwarf Server bandwidth.
SEPTEMBER/OCTOBER 2011 41
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
2D shared XII applications and rendering framework but is limited to
NetBeans Firefox working on a single thread at a time. MT Game
is a subproject of Open Wonderland that adds
multiprocessor capabilities to JMonkeyEngine
by breaking computation into separate process-
ing and rendering phases.
The core services layer provides the features
that the Wonderland modules use. These services
include the position of objects in the 3D world, the
ability to move objects, and collision detection.
Extended core services, such as the ability to load
models, calculate real physics, and enforce secu-
rity, are layered on top of the core as modules.
Communication
Sticky notes Whiteboard Audio recorder
The Wonderland client’s communications layer is
2D and 3D collaboration-aware apps
implemented in a combination of built-in Wonder-
land features and module extensions. The built-in
Figure 3. Sharing applications. The Open Wonderland platform features support authenticating to the Web server
supports both legacy 2D X11 applications and 2D and 3D Java and communicating with the Darkstar server.
applications written specifically for multiple users. Other communications, such as audio and shared
application channels, are specified in modules.
This demonstrates the toolkit’s ability to support
App cell new network protocols entirely in modules.
Cell tree
Room cell
Wonderland Extensibility
The Open Wonderland toolkit provides the
WorldRoot
cell
framework for building a collaborative 3D envi-
Avatar cell ronment, but extensions create the world the
user sees. To enable this extensibility, we cre-
ated a core modular architecture with several
well-defined extension points. We also designed
Bed cell
mechanisms for integrating with external data.
Extension Points
The Wonderland toolkit provides developers
with a number of standard extension points and
patterns. New object types are the most common
type of extension. An object in the 3D world
Figure 4. A world divided into cells. A cell is a volume of 3D space. is referred to in the Wonderland code as a cell
Any cell can contain other cells to form a cell tree. (because the word “object” is already used in
most programming languages). A cell is simply
Client Design a volume of 3D space, and any cell can contain
The Wonderland client is a single application other cells to form a cell tree.
that acts as a browser for connecting to differ- Each cell in Wonderland is an independent
ent Wonderland servers. As with the server, the Java object that can have both client and server
client provides several core services based on behavior. Examples of client behavior include
existing open source components. rendering a 2D or 3D object, reacting to user
The client’s rendering layer consists of input, or sending and receiving messages from
two separate projects. JMonkeyEngine (http://
____ the server. Examples of server behavior include
jmonkeyengine.com) is a popular rendering storing persistent properties, receiving mes-
framework for writing OpenGL-based applica- sages from clients, and sending messages to
tions in Java. It provides the basic scene graph groups of clients. Figure 4 shows a Wonderland
42 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Open Wonderland: An Extensible Virtual World Architecture
world represented as a cell tree. Note how each plug-ins, connections, and Web applications.
object in the world, including the room, the 2D A module is a specially formatted Java archive
application, and the avatars, are all variations (JAR) file. In addition to the standard JAR attri-
of the basic cell. Cells have a well-defined life butes, a module contains metadata including
cycle that includes the ability to save them as the module name, version number, and depen-
XML for long-term storage. dencies on other modules.
Another important extension point is a The bulk of a Wonderland module is in the
capability, or a feature that can be dynamically data. We divide module data up by type, with
added to any cell. Example capabilities include each type represented as a top-level direc-
a placemark, which adds an item to users’ tory within the module. The module system
placemark menu so they can jump to a par- handles each type using a deployer that is in
ticular cell, and a clickable link, which opens charge of unpacking the data and making it
a Web browser to a particular page whenever available to the correct subsystem. Example
a user clicks on an object. When building a deployers in the Wonderland core include
world, users can add capabilities to any cell to artwork, which is unpacked into a directory
augment its functionality. A capability has the in the Web server where clients can down-
same life cycle as a cell and is almost identi- load it; client code, which is also made avail-
cal except that each instance of a capability is able to clients via the Web server; server code,
associated with a particular cell. which is installed in the Darkstar ser ver;
Both cells and capabilities relate to items and Web administration modules, which are
that have a particular location in the world. deployed to the Web server using standard
Developers can add other extensions that aren’t Java EE mechanisms. The set of deployers in
spatial in nature via plug-ins, which are avail- the module system itself is even extensible;
able to users no matter where they are in the Developers can use a new deployer contained
world. Thus, they’re useful for features such as in a module to deploy custom content in other
text chat and inventory that must always be modules.
available. Like cells, plug-ins can have func-
tionality in both the client and server, so the Design Trade-Offs
client plug-in can send messages that the server The Wonderland architecture has been in use
plug-in must process. The server plug-in can for close to four years, having undergone two
also save its state in persistent storage. complete rewrites in that time. Here we discuss
Plug-ins might also use custom connec- some of the major design decisions we made and
tions. A connection is a particular data chan- the advantages and disadvantages we found for
nel between any number of clients to the server. each approach.
The connection’s type defines the format of the
data the plug-in will send over the channel. Simulation Model
Custom connections are useful for adding new Wonderland is based on a hybrid computa-
data channels for features such as text chat or tion model between the client and server. In
administrator tools. Developers can also employ this model, the server maintains objects’ states
custom connections to connect to special- primarily by reacting to client requests. The
purpose applications other than the Wonderland server doesn’t handle objects’ graphical states
client to form a bridge. but rather their properties, such as name or
The last major extension point is the ability to position. The client does most of the work in
add custom Web applications. This lets develop- rendering the object on the screen as well as
ers add functionality to the Web administration responding to user input and property changes
user interface or entire new Web services. These the server sends.
extensions are provided as standard Java EE This approach falls somewhere in between
applications that are deployed to the Wonder- comparable systems; OpenSimulator performs
land Web server. more computation — including physics — on
the server and shares fine-grained state with
Module System the client (see http://opensimulator.org/wiki/
Wonderland modules are the mechanism for pack- OpenSim:Introduction_and_Definitions). Open
_____________________________
aging extensions, including objects, capabilities, Croquet, on the other hand, uses a peer-to-peer
SEPTEMBER/OCTOBER 2011 43
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
model in which most computation is replicated developing features in the core, interfaces are
bet ween ser vers (see w w w.opencobalt.org/ designed in the core and implemented in modules.
about/synchronization-architecture). Wonder-
__________________________ Figuring out which module implements which
land is flexible in that developers can employ feature can be difficult. Furthermore, because
e it he r mode l a s ne e de d; h ig h ly i nte r ac- Wonderland administrators can add, update, and
tive tasks can be simulated on the client with remove modules individually, module depen-
the u nde r sta nd i ng t hat s y nc h ron i zat ion dencies and versions become a management
might not be perfect between different users. challenge.
Tasks with stronger synchronization require- Despite this complexity, a modular architec-
ments can run on the server, with the trade- ture lets us build an ecosystem of extensions
off of higher latency and therefore less frequent around the Wonderland toolkit. We provide a
updates. Module Warehouse where developers can share
their modules with others, and we host module
Scalability and Interactivity repositories so they can share code.
In many cases, we’ve found the need to choose
between scalability and interactivity. The basic
trade-off is simple: a world that’s more inter- he Open Wonderland toolkit is in active use
active changes more frequently, requiring more
bandwidth and computation to keep all the cli-
T all over the world for projects in education,
collaboration, and simulation. Our main focus
ents up to date. A world that changes less fre- is on improving the current version’s collabora-
quently, or is static (as in many videogames), tion features, stability, and scalability.
can support more users with less communi- One key area of f uture development is
cation required per user. This same decision increased server federation — that is, the ability
applies to almost every feature of the environ- to connect multiple servers. We’ve developed
ment. For example, using more graphically rich our client as a browser, enabling a single cli-
avatars provides a better sense of presence but ent to connect to many servers with different
requires more resources from the video card, features. We’d like to enhance this ability — for
limiting the number of avatars that a Wonder- example, to let a client connect to multiple serv-
land world can display. ers simultaneously — to simulate large, continu-
For our Wonderland collaboration use case, ous environments. Another extension would
we targeted small work groups of fewer than be to cluster servers so that a group of servers
20 people, putting more emphasis on interactivity share common resources such as authentica-
than on large numbers of users. This target was tion scope, content repositories, and presence
based on research related to meeting behavior information.
in which we found that the typical meeting had As we start expanding support for multiple
between two and 16 participants.4 The current servers and data types, we must also think
version of Wonderland supports up to 50 users about interoperability. As a first pass, many
in a single space, allowing room for multiple groups are working together to define com-
simultaneous groups to interact in the same mon artwork formats and presence mecha-
space. Larger groups must be divided into multi- nisms that different virtual worlds could use.
ple spaces. Different trade-offs might be made Eventually, as with the Web, we expect to see
in a world designed for giving large presenta- large-scale standardization of virtual environ-
tions, with much less interactivity but scaling ments. This will require standardization not
to many more users. only of content but also of behavior, so that
a user can access interactive, collaborative
Modularity and Complexity virtual spaces that work the same no matter
The last major trade-off is between a modular which browser they use. Although predicting
architecture and software complexity. We’ve what this standard model will look like is dif-
already described many of a modular architec- ficult, the Open Wonderland architecture can
ture’s advantages, including extensibility and be a starting point for this standardization
manageability. Some downsides exist as well. effort.
Developing in a modular fashion introduces Open Wonderland is a highly exten-
much more fragmentation to the code. Rather than sible toolkit for building vir tual worlds.
44 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Open Wonderland: An Extensible Virtual World Architecture
In its current form, we can deploy it to sup- Jonathan Kaplan is an architect for the Open Wonderland
port a wide range of collaboration use cases. Foundation and the CTO of WonderBuilders. He is the
Due to our focus on extensibility, it is also original software architect of the Wonderland plat-
an ideal platform for experimentation and form, a project he cofounded at Sun Microsystems Lab-
research into new virtual world features and oratories. Kaplan has an MSE in computer science from
applications. the University of Pennsylvania. He is the coauthor of
J2EE Design Patterns (O’Reilly and Associates, 2003).
jonathankap@gmail.com.
Contact him at _______________
References
1. D.B. Anderson et al., “Building Multi-User Interactive Nicole Yankelovich is the executive director of the Open
Multimedia Environments at MERL,” IEEE Multimedia, Wonderland Foundation and CEO of WonderBuilders.
vol. 2, no. 4, 1995, pp. 77–82. She cofounded the Wonderland project in 2007 as prin-
2. W. Broll, “Interacting in Distributed Collaborative Vir- cipal investigator of the Collaborative Environments
tual Environments,” Proc. Virtual Reality Ann. Int’l research program at Sun Microsystems Laboratories.
Symp., 1995, pp. 148–155. She’s also a visiting scientist at the Massachusetts
3. J. Andreano et al., “Auditory Cues Increase the Hippo- Institute of Technology Center for Educational Com-
campal Response to Unimodal Virtual Reality,” puting Initiatives. Yankelovich holds seven patents
CyberPsychology & Behavior, vol. 12, no. 3, 2009, and has published in the areas of collaborative envi-
pp. 309–313. ronments, speech applications, and hypertext. Contact
4. N. Yankelovich et al., “Meeting Central: Making Dis- her at nicole@openwonderland.org.
_________________
tributed Meetings More Effective,” Proc. ACM Conf.
Computer Supported Cooperative Work (CSCW 04), Selected CS articles and columns are also available
ACM Press, 2004, pp. 419–442. for free at http://ComputingNow.computer.org.
A new publication model that will
provide subscribers with features
and benefits that cannot be found in
traditional print such as:
Available Transactions Titles by 2012:
For more information about OnlinePlus™,
please visit http://www.computer.org/onlineplus.
SEPTEMBER/OCTOBER 2011 45
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Virtual and Real-World
Ontology Services
Both augmented-reality and virtual world applications must model semantic
knowledge about real- or virtual world objects. The current generation of
virtual world platforms provides limited facilities for representing this kind
of knowledge, but a next generation will provide the means to tie semantic
information to general or application-specific ontology services. This article
motivates the need for ontology services, outlines several approaches for
associating ontology concepts with objects and locations, and discusses how
to populate common-sense ontologies using data harvested from real and
virtual worlds.
V
Joshua D. Eno and irtual worlds such as Second Life interact with the environment or other
Craig W. Thompson (http://secondlife.com) and Open- users.
University of Arkansas Simulator (http://opensimulator. Augmenting objects with additional
org) provide a 3D landscape in which
__ semantics is similar to the idea of the
user-controlled avatars traverse a shared, Semantic Web, which Tim Berners-Lee,
multiplayer world, visiting places, creat- James Hendler, and Ora Lassila pro-
ing objects, and selling land and goods posed in 2001 as an extension of the
to others. Although several architectural World Wide Web to augment webpages
variants exist among virtual worlds, the with semantic information that intelli-
notion of representing a 3D space con- gent agents could understand and use.1
taining terrain, animate, and inanimate They envisioned that the Semantic Web
objects is common. These 3D models can would require knowledge representa-
represent fantasy places or can model tion, ontologies, and agents. Around
real-world locations. the same time, the MIT Auto-ID Labs
Whereas many applications of vir- coined the term Internet of Things
tual worlds (socializing, training, meet- (IoT) to describe the notion of the
ings, and education) involve humans real world populated with intelligent
using the vir tual world platform objects with semantic attributes that
directly, gaming and simulation appli- can interact with people or autonomous
cations must augment virtual world agents.2
objects with information from remote Here, we recognize that we can explore
data sources. Often, these applications IoT ideas using vir tual worlds and
include computational agents that focus on how virtual world ontologies
46 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual and Real-World Ontology Services
can borrow ideas from both the Semantic Web www.yelp.com/yelpmobile, or www.google.com/
and the IoT but can also give back — providing a mobile/goggles).
__________
way to model the real world using virtual world From a computational viewpoint, 3D vir-
data structures and a way to attach semantics tual world platforms represent objects that
to those data structures — to help develop a have explicit identity, a graphical representa-
unified vision we could call a semantic world tion, optional text labels, and behavior models
or smart world in which objects and avatars are (often represented by scripts that are triggered
associated with knowledge. by events). In the future, a corresponding real-
world computational model could incorporate
Semantics and the Virtual World analogous data structures, which lets us con-
In our own work, we’ve explored how to use sider the real world as one more type of virtual
virtual worlds to model the real world in health- world.
care and retail applications and have developed Current mainstream 3D virtual worlds pro-
a collection of prototype smart-world applica- vide minimal support for semantically label-
tions (http://vw.ddns.uark.edu), including ing objects. Beyond rendering, the Second Life
platform itself doesn’t distinguish functionally
an application that tracks virtual world whether an object is a door or a castle. It provides
medical supplies in a supply chain with a text labels for objects but no further descriptions,
remote database, recording a track history of and these labels aren’t tied to semantic concepts.
objects’ and avatars’ past locations (related Similarly, the real world doesn’t provide labels —
search applications can locate objects — for humans can look at an object and recognize it
instance, find a wheelchair not in use); as a chair, as can computers if the object is at
a mirror-world application in which we use a a known location, has an RFID tag, or is other-
real-time location service to track real-world wise digitally identifiable. Humans can then
RFID-tagged apparel items and then display bring to bear other information they know about
the same items moving in a virtual world to that object — about its superclasses, its parts,
create a retail store command post; its function, how to repair it, how to operate it,
an annotation ser v ice for allowing any and so on. Some of this is general common-
avatar to annotate unlabeled virtual world sense knowledge and some might be application-
objects with descriptions or other attributes specific, such as the cost of a particular can of
(price, calorie count, washing instructions, a okra at a particular market on a particular day.
link to a repair manual, and so on); In this article, we call this kind of knowledge an
a recommendation service that compares an ontology, by which we just mean a data struc-
avatar’s profile and recent locations to rec- ture for recording various kinds of information
ommend similar locations or avatars; and including identity, type, supertype, parts, API,
a protocol for service discovery such that, and an open-ended collection of attributes and
when APIs are associated with virtual world scripts. We can then describe the semantics of
objects (such as a thermostat or a baby an entity in the world by referencing informa-
monitor), lets users discover, download, and tion from this ontology, allowing computational
remotely control those objects.3 agents to interact with and reason about the
world more effectively.
In each case, we observe the same architec- 3D virtual world platforms don’t currently
tural design pattern: virtual world objects are support an ontology capability (beyond plain-
augmented with domain-relevant information text labels), so applications must provide it. But
that’s then utilized by application-specific logic — a new generation of virtual world platforms
for example, to make a refrigerator smarter so it is beginning to provide extensibility mecha-
knows the food expiration dates or a bus route nisms4,5 that can, among other things, provide
smarter so riders can see when the next bus ontology services.
will come. A similar design pattern exists for
augmented-reality applications, which provide Ontology Services
information about nearby locations based on and Knowledge Sources
geospatial coordinates or pictures of landmarks If we had a more consistent semantic labeling
(see www.acrossair.com/apps_nearesttube.htm, for objects, then we could label some things
SEPTEMBER/OCTOBER 2011 47
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
chair and other things table and begin to Augmented-reality applications already dem-
associate functional specifications to enable onstrate using a smart phone to view an area,
computers to reason about them. If humans can and labeling nearby buildings.
look at an object and recognize it as a chair, A problem in providing an ontology layer is
it would be useful if computers, including, for how to populate the ontology and associate con-
instance, our smart phones, could also do so cepts with entities. One option is to build fully
using the same conceptual categories. automated recognizers to identify and cate-
Let’s reason about a virtual or real-world gorize objects. In important special cases, as
architecture with a semantic ontology capability: when all objects are labeled with RFID tags or
barcodes that smart phones can read, machines
Not all real- or virtual world applications can use the labels. More generally, humans use
need an ontology layer, so such a capability image recognition, but more work in image
could be structured as an optional plug-in understanding will be needed for machines to
service or services. generally recognize objects. Additionally, a
The same ontology content could be use- fully automated system will need to be able to
ful for modeling a virtual world, but also recognize when the ontology must expand to
the real world; an ontology service can be incorporate new types or relationships discov-
agnostic to which world it’s modeling. ered from the environment.
Similarly, an application might not be aware A second option is for content creators to
if it’s operating in the real world or a virtual manually associate concepts with entities by
one. We can build applications and test them labeling the objects they create. This approach
in virtual worlds before we install them in is already available in Second Life and Open-
the real world. Simulator, but in practice creators label only
General-purpose ontologies are a useful 20 percent of top-level objects, and these labels
starting point and in many cases are suffi- are inconsistent. A second, manual method
cient, but because the kinds of metadata and (which we prototyped) is to provide an annota-
relationships between concepts are open- tion service that lets any user label any object
ended, specific applications often require with a semantic label, name, description, rec-
application-specific ontologies that augment ommendation, or other property. This approach
or replace general ontologies. uses crowd sourcing to populate ontologies, but
Finally, if the same ontology is useful for is still a manual process.
both real and virtual worlds, perhaps we can A third option is to use existing ontologies
build it using data from both real- and vir- and link these open datasets together.6 For some
tual world data sources. classes of objects, building a partial ontology
can be automated based on existing databases.
An ontology ser vice must provide well- The WordNet ontology provides a word/concept-
structured, standard interfaces that can accom- level ontology and can represent taxonomic and
modate multiple sources and uses. Intelligent compositional (ISA and HASPART) relationships.7
agents have difficulty using semantic informa- DBpedia har vests ontology templates from
tion when ontology information sources aren’t Wikipedia, representing roughly 300 ontology
structured to be readily available, and the inter- classes in RDF (including places, people, orga-
faces for accessing information or controlling nizations, species, vehicles, devices, and works),
objects aren’t standardized. Although central- linked to more than 3.5 million things (see
ized services would standardize sources and http://dbpedia.org).
interfaces, no single source will likely be suf- Another source of explicit, existing ontolo-
ficient, so many service providers would offer gies is retailer databases containing SKU des-
private ontology services. As an example, a ignations for thousands of item types — for
retailer’s ontology could provide pricing infor- example, different kinds of chairs and tables —
mation while product specifications could come that include corresponding price and other
from a manufacturer-provided ontology. attributes, such as product descriptions. More
Ontologies could be organized or indexed by generally, the Linked Data project seeks to con-
various means: type-subtype, location in a 3D nect a wide range of open datasets (see ____http://
world, temporally, by context, or by other means. linkeddata.org/home). For objects with interfaces,
______________
48 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual and Real-World Ontology Services
we can further associate API specifications and Although we’re just now entering a time
provide a consistent way for humans or pro- when we can gather dense datasets from the
grams to call the APIs — so we could use our real world, we can already do so using 3D vir-
smart phones to recognize a nearby object, tual worlds. To gather descriptive data from
download its API, and then generate an inter- virtual worlds, we created a system that harvests
face that humans or programs could use to data from OpenSimulator and Second Life.10 In
query or control the object (a more universal our system, avatarbots (program- rather than
remote). Several protocols that we could asso- human-controlled avatars) navigate the virtual
ciate with objects to make them into smart(er) world, storing metadata about the locations and
objects are explored elsewhere.3 objects they encounter. (We could take a similar
A fourth option is to mine data from the real approach with other virtual worlds, but many
or virtual world to create or expand ontologies, are small and data-sparse).
which we can then add to the broader linked
data community. In the real world, RFID, GPS, Experiments
image, and sensor data are commonly collected Next, we describe initial steps for exploring
to help model particular real-world applications how to use the partial, sparse text labeling
such as supply chain, battle management, or in 3D virtual worlds to improve and simplify
mapping services, like Google Earth. Especially all four approaches for providing ontolo-
interesting is recognition of daily living activi- gies and semantic associations between onto-
ties (such as setting the table) based on object logical concepts and virtual world entities.
usage (GPS or RFID traces).8 For the first approach, we use probabilistic
We can also mine similar information from models to help populate currently unlabeled
unstructured text on the Web.9 Smart phones virtual world entity attributes. To improve user-
open the door wide to harvesting this sort of provided semantic information, we provide lists
information from the real world with humans of likely concepts for newly created objects
acting as search spiders. With GPS, they can (such as auto-complete for ontologies). We test
collect and record where a person has been, the feasibility of linking existing ontologies to
their communications (voice and email), and, if objects based on unstructured object names and
RFID readers are added to cell phones, traces of explore methods for expanding existing ontolo-
all the objects a person passes. A community gies with data-derived relationships.
of humans could harvest a model of locations
and movements, potentially providing a fairly Determining Location Context
dense model of the world. If graphical and other Humans rely on context to guide our ability
models were associated with the tags, a virtual to reason about the world; in the same way,
earth model could be populated and updated in context can improve computers’ ability to rec-
this way. ognize and interact with objects in real and
In practice, a combination of all four virtual worlds. A computer agent (such as an
approaches provides increasingly accurate and avatarbot that harvests virtual world content)
useful ontologies and entity associations. Auto- might recognize that solid objects are obstacles
matically associating attributes with entities is to avoid. However, if the agent knows that the
more feasible when contextual information can object is a door, it could reason that it can be
narrow down the possible concepts to improve opened rather than avoided and might further
accuracy. User-provided tags will be more con- recognize that a door on a residential property
sistent and useful if a means exists for associ- separates public from private areas, whereas a
ating existing ontology concepts with entities. front door on a commercial property is a public
Existing ontologies will benefit from additional entrance.
details derived from virtual or real-world data, In virtual worlds, context can be explicit in
and data-derived ontologies will be more accu- cases where property owners have labeled the
rate and useful if they’re based on a scaffold of property as residential or shopping, but rela-
existing ontologies. All these approaches rely tively few owners do so. However, because the
on a combination of existing ontologies, data- difference between a house and a store is obvi-
derived ontological relationships, and user- ous to humans, even in a virtual world, users
provided contextual information. expect other avatars to respect their privacy
SEPTEMBER/OCTOBER 2011 49
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
0.5
0.4 SVM Actual
Compounding this is the problem of multiple
0.3 meanings for the same term. A white house
0.2 could be a brightly painted home, or it might
0.1 be a specific government building. One way
0 to minimize these issues is to suggest likely
Arts
Adult
Linden
Gaming
Shopping
Stage
Other
Hangout
Business
Residential
Newcomer
Education
Park
semantic labels for an object. If the likely labels
have multiple senses in the ontology, the user
can further select the sense of the word that’s
most appropriate for the object.
Figure 1. Support vector machine (SVM) Context is again important in providing
classification results. The classifier performed relevant suggestions. We developed a proto-
60 percent better than simple probability type annotation system that focuses on resi-
weighting, although it still over-represented dential locations. The system can suggest a
the Residential and Shopping categories. set of likely terms based both on the location
type and on other objects found near that loca-
tion. The system collects the object names that
and stay out of their residential homes. To act already exist at a location, finds other locations
correctly in such cases, an agent must use the in the system with the same objects, then sug-
same kind of contextual clues that humans gests names that best match the existing set of
do, rather than relying on explicit labels. To objects.
enable our crawler agents to avoid being intru- The annotation system ranks suggested
sive in residential areas, we developed a clas- terms based on a relevance score that incor-
sifier that uses location and object metadata to porates the existing terms’ importance and
classify locations, even if they aren’t explicitly the frequency with which the suggested terms
labeled. occur with the existing terms. The rank score
Using the subset of labeled locations, we for a suggested term is the sum of the condi-
trained a classification model to classify loca- tional probability that each existing term will
tions based on their text and region. We used appear, given that the suggested term is pres-
a support vector machine (SVM) classifier,11 ent. This probability is smoothed by a factor in
which performs well compared to other algo- the denominator to account for low-frequency
rithms such as naïve Bayesian and k-nearest terms, as discussed elsewhere.13 The prob-
neighbor for text classification with large fea- abilities are weighted by the existing terms’
ture spaces.12 Figure 1 shows a comparison of inverse-document frequency (idf ), so that co-
the SVM classifications with labeled locations’ locations with rare terms receive greater weight
true classifications. than co-locations with common terms. Finally,
The average classifier accuracy using five- we normalize the sum by the sum of the idf val-
fold validation was 58.2 percent across 13 parcel ues. The resulting score is similar to the com-
Second Life classifications, a result that was mon TF-IDF score used in information retrieval,
60 percent better than the best naïve classifier except reversed to provide terms for the loca-
based just on the underlying parcel type prob- tion rather than relevant locations for the query
abilities. Once a location’s general purpose is terms:
known, the likelihood of finding certain objects
sti
changes. In a fully automated system, an object ∑ i st +
idfi
recognition system can use the new contex- rst = count ,
tual probabilities to improve object classifica- Ei idfi
tion, providing a way to disambiguate a bed (for
flowers) in a park from a bed (for sleeping) in a where r st is the rank score of suggested
residence. term st, sti is the number of locations with
co-occurrences of st and existing term i, stcount
Labeling Suggestions for Users is the total number of locations with st, and
One difficulty in working with user-assigned is a smoothing factor for low-frequency terms.
names and descriptions is that different users The inverse-document frequency (idf ) is a
might use different terms for the same object type. commonly used means for giving rare terms
50 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual and Real-World Ontology Services
Table 1. Existing ontology coverage.
in the collection more weight than common Ontology coverage Full terms (%) Stemmed (%)
terms: DBpedia percent of Second 58.0 59.5
Life terms
WordNet percent of Second 30.3 33.7
Life terms
where |L| is the total number of locations and DBpedia percent of total 0.3 0.3
icount is the total number of locations with term i. WordNet percent of total 7.9 8.7
The label suggestions generally conform to
common-sense terms that we’d expect to find
together, particularly for locations that have words increased coverage by eliminating misses
distinctive objects. For example, the system caused by plurals, but in some cases might have
suggests the terms {hamper, towels, sink, created less-accurate matching. By examin-
faucet} when given a location with the terms ing the terms DBpedia matched that WordNet
{house, bathroom, sink}. In contrast, when missed, we found that many newer terms or
we give it a location with the terms {kitchen, informal words existed in the DBpedia dataset,
oven, dishwasher}, it returns the suggestions which is constructed using Wikipedia entries.
{microwave, freezer, backsplash, utensils}. Some terms, such as “YouTube,” are unsurpris-
In locations with more ambiguous terms {chair, ing, but others, such as “media,” were unex-
door, table}, the suggestions are less focused, pected. However, some of those matches might
resulting in generally common matches. Because have been matching brand names to abbrevia-
the system works on existing plaintext terms, tions or foreign-language terms that weren’t
it still has difficulty differentiating between actually related. Another factor working in
different word senses, but we could adapt and DBpedia’s favor is its larger size. DBpedia has
improve this approach as semantic labels become 7.5 million titles in its index, covering 3.5 million
available. things (the index contains multiple terms that
map to a single thing in some cases). WordNet is
Linking Existing Ontologies smaller, with fewer than 150,000 index entries, so
Although the existing term-suggestion service while it had fewer overall matches, it had a
is helpful in creating a more homogenous set higher hit rate as a percentage of its size. Another
of terms for the set of objects commonly found factor that might indicate a greater utility to
in similar locations, it still relies on plaintext the WordNet matches is that only 1.67 million
terms rather than ontological concepts. How- of the DBpedia instances are classified in a con-
ever, we’re experimenting with tools to associ- sistent ontology.
ate concepts from WordNet and DBpedia with
virtual world entities. Expanding and Creating Ontologies
One question to ask is how these existing Although high-quality ontologies are created
ontologies cover the objects found in 3D vir- for a range of purposes, individual applications
tual worlds. Some terms, such as specific brand might find that general-purpose ontologies lack
names, have no analog in the real world and needed details. Additionally, ontologies often
hence won’t appear in ontologies based on the focus on taxonomic data that provide “is-a” rela-
real world. Likewise, some concepts and terms tionships but not necessarily functional or “has-
are specific to virtual world lexicons, such as parts” relationships. Researchers have already
prims, which are primitive objects used to build found that observing use patterns can reveal
3D models in Second Life. To quantify the cov- functional semantic relationships,8 and simi-
erage of the virtual world by two large and lar approaches with objects found in images can
widely used ontologies, we matched the 38,000 derive has-part relationships.13 Using an approach
terms found in the residential location dataset similar to that used to suggest likely description
with concept terms in both the WordNet and terms for content creators, we developed tools to
DBpedia ontologies. discover common relationships between objects
To account for variations in word forms, we found in the virtual world. For this tool, we rely
analyzed the matches for both raw (or full) and on the conditional probability P(p|c) that a par-
stemmed terms (see Table 1). Stemming the ent term will be found, given that some child
SEPTEMBER/OCTOBER 2011 51
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
term is found to identify child terms that are 3. A. Eguchi and C. Thompson, “Towards a Semantic
strongly associated with a parent term. World: Smart Objects in a Virtual World,” Web Vir-
The results for creating or expanding onto- tual Reality and Three Dimensional Worlds Workshop,
logies varied based on the parent–child relation- Proc. Int’l Assoc. for the Development of the Info. Soc.
ships’ specificity. For example, the probability (IADIS) Multiconf. Computer Science and Information
that a kitchen will be present is highest if the Systems, IADIS Press, 2010, pp. 488–493.
terms cooktop, cabinets, microwave, fridge, 4. T. Alatalo, “An Entity-Component Model for Extensible
or oven are found. For the specific kitchen Virtual Worlds,” IEEE Internet Computing, vol. 15, no. 5,
instance, the system’s precision is high, with 2011, pp. 30–37.
90 percent of the top 20 objects having a real- 5. J. Kaplan and N. Yankelovich, “Open Wonderland: An
world relationship to kitchens. For parent terms Extensible Virtual World Architecture,” IEEE Internet
with less-distinct component parts, the accuracy Computing, vol. 15, no. 5, 2011, pp. 38–45.
is mixed, but still provides useful information 6. T. Berners-Lee, “Linked Data — Design Issues,” 27 July
in terms of expanding the ontology probabilisti- 2006; www.w3.org/DesignIssues/LinkedData.html.
cally. We might be able to improve the results 7. C. Fellbaum, WordNet: An Electronic Lexical Database,
for more general terms by incorporating proxi- MIT Press, 1998.
mity more directly in the scoring function. This 8. M. Philipose et al., “Inferring Activities from Interac-
approach’s primary advantage is that it can tions with Objects,” IEEE Pervasive Computing, vol. 3,
expand existing ontologies with relationships no. 4, 2004, pp. 10–17.
that are too specific for general-purpose ontolo- 9. M. Perkowitz et al., “Mining Models of Human Activi-
gies, or with functional relationships that are ties from the Web,” Proc. 13th Int’l Conf. World Wide
difficult to derive from taxonomic ontologies. Web, ACM Press, 2004, pp. 573–582.
10. J. Eno, S. Gauch, and C. Thompson, “Searching for the
Metaverse,” Proc. ACM Symp. Virtual Reality Software
ot all virtual world applications will need an
N ontology service. Still, such a service makes
sense as a data structure that virtual worlds
and Technology, ACM Press, 2009, pp. 223–226.
11. T. Joachims, “Making Large-Scale SVM Learning
Practical,” Advances in Kernel Methods — Support Vec-
can use to represent declarative content. Many tor Learning, C.B.B. Schölkopf and A. Smola, eds., MIT
ontology services will require application- Press, 1999, pp. 169–184.
specific content, so, for virtual world architectures, 12. T. Joachims, “Text Categorization with Support Vec-
the ability to add ontology plug-ins also makes tor Machines: Learning with Many Relevant Features,”
sense. At the same time, applications can find Proc. European Conf. Machine Learning, Springer,
value in general-purpose semantic information. 1998, pp. 137–142.
We’ve seen that only 20 percent of Second 13. B. Russel et al., “LabelMe: A Database and Web-Based
Life objects are labeled by their creator. Even Tool for Image Annotation,” Int’l J. Computer Vision,
so, a harvester that collects virtual world data vol. 77, nos. 1–3, 2008, pp. 157–173.
can create a database that a classification sys-
tem can then mine for semantic information. Joshua D. Eno is a postdoctoral researcher at the University
Even though the information is from a virtual of Arkansas working on virtual world architectures
world, for many information types, the vir- and ontologies. His interests include middleware archi-
tual world provides data that models common- tectures, 3D virtual worlds, data mining, and health-
sense aspects of the real world — so kitchens care informatics. Eno has a PhD in computer science
have stoves and refrigerators (and, with a lower jeno@
from the University of Arkansas. Contact him at ____
probability, microwaves and can openers). The uark.edu.
_____
resulting common-sense model can be useful in
real or virtual worlds. Craig W. Thompson is the Charles Morgan chair in the
Computer Science and Computer Engineering Depart-
References ment at the University of Arkansas. His research inter-
1. T. Berners-Lee, J. Hendler, and O. Lassila, “The Semantic ests include artificial intelligence, databases, middleware
Web,” Scientific Am., vol. 284, May 2001, pp. 34–43. architectures, RFID, virtual worlds, and pervasive com-
2. N. Gershenfeld, R. Krikorian, and D. Cohen, “The Inter- puting. Thompson has a PhD in computer science from
net of Things,” Scientific Am., vol. 291, no. 4, 2004, the University of Texas at Austin. He’s an IEEE fellow.
pp. 76–81. Contact him at cwt@uark.edu.
________
52 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
Accuracy in 3D Virtual
Worlds Applications
Interactive 3D Modeling of the
Refractory Linings of Copper Smelters
This article highlights the need for accurate modeling in some virtual world
applications, especially in engineering, manufacturing, and certain military
applications. For example, virtual worlds can enable teams of engineers,
managers, and customers to collaboratively view a copper smelter during
design and deployment. This article specifically looks at how a virtual world
can help in the design and maintenance of a copper smelter model and its
refractory lining for copper production.
S
ome of the most popular 3D vir- The optimization of these copper Anthony J. Rigby,
tual world engines, such as Second smelting furnaces is primarily achieved Kenneth Rigby,
Life and OpenSimulator, are very by specific zoning of the vessels’ heat- and Mark Melaney
effective for socializing and meetings resistant lining. The most severe oper- MellaniuM
but fall short for serious applications ating areas come in contact with high
that involve accurate rendering. Con- temperature fluid oxide slags and must
sider the problem of engineers design- be protected with a well-engineered
ing, maintaining, and deploying the design and relatively high-cost prod-
refractory linings for smelters used in ucts. Using virtual environments can
copper production. The longevity of illustrate the complexity of the required
the refractory lining in these 40-foot × configuration much more efficiently
15-foot anode vessels is a crucial pro- than a set of 2D prints and extracted
duction issue. The smelters must oper- details.
ate continuously for four to five years.1
During brief shutdowns of the con- The Need for Virtual
verting furnaces due to plant main- World Modeling
tenance, refractory installation crews Using 3D modeling, designers can
must be able to rapidly patch the anode effectively illustrate the refractor y
vessels and ready them to resume installation, design, and lining con-
production. cepts required to optimize the copper
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 53
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
smelter’s desired performance. The use of a applications (for example, collaboration using
3D-engineered model is highly instructive in Google docs), and data (such as Wikipedia
detailing some of the more sophisticated aspects entries). It supports file sharing, URL shar-
of the lining design. Refractory engineers can ing, text chat, and file drop boxes.
design these vessels with a dimensional accuracy The VoIP works with no driver configura-
in AutoCAD 3D, with a tolerance of +/– 0.5 mm. tion. Noise suppression and echo cancellation
Using AutoCAD, they can render the model as operate with built-in laptop speakers and a
a mesh to engineering dimension specifications microphone.
and import it into a texturing software appli- The world has secure areas with an invitation
cation, which can color-code and realistically feature that lets you vouch for other users.
apply a surface to it. Users behind virtually any firewalls (includ-
Allowing teams of engineers with differ- ing HTTP proxies) can access Web.alive.
ent backgrounds to “walk around” inside the Presenters are automatically granted addi-
smelter helps them examine different elements. tional capabilities to help communicate with
AutoCAD doesn’t support this kind of team and manage their audience.
review, so to achieve collaborative design and
monitoring, we considered 3D virtual worlds. Web.alive uses the Unreal game engine to
However, we found that most popular plat- render accurate architectural and engineer-
forms failed to provide accurate renderings. For ing virtual environments. Unreal (as used in
instance, the base representation in Second Life the America’s Army recruitment project; ____http://
consists of primitive graphics objects (called en.wikipedia.org/wiki/America%27s_ Army)
__________________________________
prims), so importing AutoCA D 3D graphics provides a powerful combination of an accurate
mesh files wasn’t possible (although Second physics application (Karma), particle system
Life is reportedly integrating mesh import via editor, and vehicle physics for any engineering
the standardized Collada [Collaborative Design or military use. It isn’t yet available in virtual
Activity] format, which provides additional world platforms such as Second Life. How-
rendering accuracy). In addition, we can’t ever, Unreal version 2.5, in combination with
model certain features in these 3D worlds with DIRECTX 8.0 and 3D graphical acceleration
AutoCAD-comparable accuracy. video cards, enables us to render engineered
To meet the need for an accurate 3D virtual objects with AutoCAD dimensional accuracy
world engine that could be used in engineering, and verisimilitude, including rendering high-
manufacturing, and military applications, we polygon static meshes, photo-realistic textures,
chose Avaya’s Web.alive (http://avayalive.com), and 2D graphics that aren’t subject to debilitat-
which lets us import complex, high-polygon 3D ing pixelation on close inspection. The UnrealEd
models for deployment in a multiparticipant level editor is integrated with the rendering
environment. engine and, along with an extensible C++ core,
provides an UnrealScript high-level scripting
Web.alive and Unreal interface as well as visual editing of avatars
Web.alive was developed primarily for virtual and surface textures within the virtual world.
conferencing and collaboration engagement. It’s MellaniuM’s bridge between CAD and Unreal
based on a browser that embeds the Unreal gam- lets us import CAD designs into Unreal.
ing engine and DiamondWare 3D spatial voice UnrealEd is a real-time design tool, opti-
over IP (VoIP). Web.alive was designed to accu- mized for building 3D environments. It’s inte-
rately display engineering applications, enabling grated with Unreal’s rendering engine, offering a
teams (currently up to 25 avatars) to collaborate WYSIWYG camera view and immediate display
on a design or monitor an engineering system. of lighting, texture placement, and geometry
Web.alive offers the following features: operations. UnrealEd also provides single-click
playability; designers can launch the viewer and
Users can drag and drop documents and walk around their created environment in real
images to make presentations, collaboration, time, even during the design process.
and training easy. Any Web content can be After 3D model creation, designers can apply
rendered in the world, allowing access to photo-realistic textures up to 2,048 2,048 pixels
media (such as streaming video from YouTube), to surfaces to enhance objects’ perceived detail.
54 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Accuracy in 3D Virtual Worlds Applications
This capability, combined with detailed texture
mapping, yields photorealistic surfaces that can
display intricate engineered details. Because
Unreal can handle up to 60,000 polygons in one
modeled item, and there is an indefinite limit
to the assembled unit’s size, even with a fully
textured and lit surface, the engine can handle
enormous spaces suitable for generating immer-
sive engineering scenarios.
Modeling Copper Smelters
Web.alive, in conjunction with the Unreal gam-
ing engine, provides a 3D virtual world that
supports the accuracy required in engineering,
manufacturing, and other complex, real-world
Figure 1. Web.alive rendering of a copper smelter. The 3D
applications. For the copper smelter model we dis-
environment displays an accurate model of the anode vessel,
cussed earlier, we imported computer-generated,
showing the smelter and converter furnaces, including the design
actual-scale furnace models into the 3D virtual
of the vessel’s refractory lining.
world application to provide accurate and realis-
tic surface features and lighting. Figures 1 and 2
show a rendering of the copper smelter after we
import the AutoCAD mesh into Web.alive.
hen we import these models into the 3D
W engine framework, we can create content-
rich environments that enable teams to inter-
actively develop or later monitor and maintain
complex equipment. In the near future, we plan
to release similar environments to illustrate
the more complex smelter designs required for
aluminum and nickel metal primary produc-
tion and the innovative incineration of domes-
tic waste. All these applications demand a high
level of engineering complexity, and real-time
collaboration within these environments will
result in rapid assimilation of the know-how
deemed necessar y for extended campaign
performance.
Figure 2. Interior of a copper anode vessel, rendered in Web.alive.
Reference
This interior consists of thick reinforced tuyere areas, the access
1. A.J. Rigby, “Controlling the Process Parameter Affect-
door, the porous plug placement, the skimming mouth, and the
ing the Refractory Requirements for Peirce-Smith Con-
slag line refractory.
verters and Anode Vessels,” TMS 2005 Converting and
Fire Refining, A.G. Ross, T. Warner, and K. Scholey, eds.,
Wiley, 2005, pp. 213–222. Kenneth Rigby is the CEO of MellaniuM. He has 35 years
experience in the British aerospace industry. Contact
Anthony J. Rigby is the marketing manager at MellaniuM, him at info@mellanium.co.uk.
_____________
a content creation company specifically geared to HD
environments generated in the Unreal engine used in Mark Melaney is the CTO and CIO of MellaniuM. He has
Avaya’s web.alive browser embedded virtual world 15 years of experience in AutoCAD, 3D Studio Max,
platform. He has 20 years experience in furnace and Unreal 2.5. Contact him at info@mellanium.
__________
joe.rigby@sympatico.ca.
design. Contact him at ______________ co.uk.
___
SEPTEMBER/OCTOBER 2011 55
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
I-Room: Augmenting
Virtual Worlds with
Intelligent Systems
An I-Room is a virtual world “intelligent room” that can support collaborative
meetings and activities, especially when these involve sense-making about a
current situation, planning, considering options, and decision making. The
combination of a virtual worlds meeting space and intelligent systems to
support planning and decision making in an I-Room provides a readily
understandable framework and generic architecture for a wide range of
potential collaborative applications and uses.
M
Austin Tate i l i t a r y c om m a nd p o s t s a nd systems to support planning, collab-
Artificial Intelligence Applications civilian emergency operations orative option generation, plan critiqu-
Institute, University of Edinburgh centers provide a nexus where ing, and adaptive plan execution in
a team of decision makers can come very dynamic situations. Over the past
together to gather information, under- few years, we’ve linked these technolo-
stand a situation, and make decisions gies together with a virtual interactive
in crisis situations, during disasters, meeting space to provide an I-Room — a
and when an organization or region is virtual space for intelligent interaction.
under threat. But often, these decision Applications to date include emergency
makers are distributed and can’t be response operations centers used for
called together physically. experimentation and exercises, support
Whereas most 3D virtual worlds for a geographically dispersed cross-
have been used as social networking or disciplinary team engaged in creating
sales venues, universities and businesses multimedia products, and even a com-
have considered other uses, finding such mercial application involving expertly
worlds especially effective for teaching tutored whisky-tasting and sales. Here,
and collaborative meetings. Relatively I describe I-Room technology and its
few serious applications (sometimes collaborative uses.
called “serious games” because they use
gaming technology for a serious purpose) The I-Room
have been studied in 3D virtual worlds. An I-Room is an environment for intel-
For some years, the University of ligent interaction. It can provide sup-
Edinburgh has been developing intelligent port for formal business meetings,
56 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
I-Room: Augmenting Virtual Worlds with Intelligent Systems
tutorials, project meetings, discussion groups,
and ad hoc interactions. Users can employ the
I-Room to organize and present pre-existing
information as well as display real-time infor-
mation feeds from other systems such as sen-
sor networks and Web services. It can also help
participants communicate, incorporate voice
channels and teleconferencing, facilitate inter-
actions, and record and act on the decisions
taken during a collaboration.
Using the I-Room concept within vir-
tual worlds gives a collaboration an intuitive
grounding in a persistent 3D space in which
participants’ representations (avatars) appear,
and the artifacts and resources used in the
collaboration are close at hand (see Figure 1).
Avatars can meet each other “face-to-face” in
a virtual world when their human counterparts Figure 1. Example I-Room. The I-Room shows live information
can’t. Some benefits of a real-world meeting feeds and links to external data sources.
are retained through immersion in the virtual
world, and in some cases virtual world meetings Decision makers could use the original
might be an effective alternative to face-to-face I-X/I-Plan collaborative planning technologies
meetings, telephone calls, or video conferences. when local or remote from one another by inter-
Beyond the advantages a shared interaction acting through a shared Web interface. The I-X
space confers, the I-Room can help deliver intel- tool suite includes simple chat and information
ligent systems support for meetings and col- exchange capabilities (using, for example, Jabber/
laborative activities. In particular, we designed XMPP messaging) for discussions between multi-
the I-Room to draw on I-X Technology,1 which ple users such as decision makers and special-
provides human participants with intelligent ist planners. However, the technologies lacked
and intelligible task support, process manage- a simple and intuitive means to enable aware-
ment, collaborative tools, and planning aids. The ness of other decision-making agents’ presence
I-Room can also utilize a range of manual and or share artifacts, and voice and video weren’t
automated capabilities or agents in a coherent used.
way. Participants share meaningful information With the advent of 3D virtual worlds (for
about the processes or products they’re working example, Second Life and OpenSimulator), our
on through a common conceptual model called team was able to link I-X technologies so that
(Issues-Nodes-Constraints-Annotations).2 they could support a community connected
The I-Room framework is flexible enough to via such a virtual worlds meeting space. The
provide participants in I-Room meetings with flexibility of typical scripting facilities in vir-
access to knowledge-based content and natural- tual worlds and their ability to easily connect
language-generation technology that tailors with external Web services made the integra-
utterances to users’ specific experience levels. tion of the intelligent systems relatively easy
without requiring fundamental changes to how
Intelligent Systems Technology the decision-support systems operate. Virtual
One key intelligent system used in the I-Room is worlds also support good connections to Web-
the I-X Technology process support framework based static media and dynamically generated
and I-Plan.1 I-Plan is an intelligent planning aid Web content and can connect with live media
that can offer task-support help, generate and streams, which facilitate a common, shared
refine plans to adapt them to the situation at real-time view of presentations and excel-
hand, support the execution of standard oper- lent links to video teleconferencing for mixed
ating procedures, support the various stages of reality events. In some applications, an avatar
conducting a meeting, help handle post-meeting within the virtual world presents a “camera”
group actions, and so on. view of a collaborative meeting such that it can
SEPTEMBER/OCTOBER 2011 57
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
be relayed to Web observers — that is, users who Although some of these tasks are simple,
connect only via Web presentations technolo- the I-Room can perform others well only if it
gies such as Adobe Connect. This has enabled has access to knowledge about meetings in
voice, video, text chat, and presentation sharing general and the current meeting and partici-
across the virtual world and Web observer com- pants in particular. Linking the I-Room to real-
munities to extend the reach to users unable to world knowledge-based systems can potentially
connect directly for security, firewall, or man- extend the support they offer into this virtual
agement reasons. space, thereby distributing the knowledge they
embody.4
I-X Technology and
I-Room Meeting Support Underlying Concepts
As mentioned, I-X Technology provides intel- for I-Room Collaboration
ligent task support, planning capabilities, and Underlying the use of the I-Room for collabora-
coordination between multiple agents.1 It pro- tion and its ability to link human participants
vides a user interface called an I-X process to intelligent systems support are the following
panel (I-P2)3 that acts as a sophisticated “to-do” concepts:
list. An I-Room created within a virtual world
such as Second Life or OpenSimulator can be a mixed-initiative collaborative model for
linked to I-X systems and agents external to the refining and constraining processes and
virtual world to support collaborative meetings products;
in virtual worlds or in mixed reality alongside communication based on sharing issues,
real-world meeting locations. It can support activities and processes, state, events, agents,
common requirements for meetings by options, argumentation, rationale, presence
information, and status reports;
automatically generating a framework for the use of the ontology for rep-
the meeting, including generic agenda items resenting the processes used and products
(such as review of previous actions, “any developed during meetings;
other business,” and the agreed-on date for I-X Technology and its suite of tools to pro-
the next meeting); vide task and process support;
keeping track of actions and agenda items the use of issue-based argumentation about
during the meeting itself; options;
recording decisions and taking minutes; the use of agent presence models, as in
tracking existing actions and adding new instant messaging, and awareness of agent
ones; context, status, relationships within an
providing access to minutes from previous organizational framework, capabilities, and
meetings; and authorities; and
automatically generating an outline of the external shared repositories of descriptions
meeting minutes. of processes, products, and other domain-
defined objects.
Through an object in the virtual world,
called the I-X helper, the I-Room can sense Together these provide a principled, intelligible,
avatars’ presence and respond to commands and extendable basis for collaboration between
directed to it. The I-Room can provide addi- the people and systems involved.
tional support by
Connecting an I-Room
monitoring participants’ comings and goings to a Virtual World
in the meeting; Meeting participants in an I-Room log in via
prompting in-world “screens” to display the their avatar in a virtual world viewer. The I-X
meeting agenda or any relevant images, helper, which can be any convenient object in
media, documents, or webpages at appropri- the virtual world, contains scripts that act as a
ate times during the meeting; and conduit for channeling communications to the
unobtrusively documenting the meeting’s participants connecting via their avatars. The
progress and outcomes. I-X helper communicates through one nominated
58 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
I-Room: Augmenting Virtual Worlds with Intelligent Systems
I-X agent to the various I-X services via a com- Virtual world server I-X services
munications channel (which, for Second Life, I-X helper HTTP
Virtual world
for example, uses a mixture of HTTP requests I-X comms
capabilities
comms XML-RPC
and responses and communications via XML
RPC). Messages from either end can be queued Chat I-X helper I-X agent for VW
and sent later if the I-X helper or the nominated
I-X agent aren’t available. Chat Listen
The I-X helper can communicate with ava-
tars in the I-Room via text chat channels in VW viewer I-X
the virtual world. It also uses dedicated pri- process
Avatar panel
vate channels to communicate with and con-
trol suitable devices in the virtual world, such Participants Participants
as screens. The helper can operate such devices
within an I-Room by loading a note card held Figure 2. The I-X helper. The helper connects the virtual world to
within the virtual world itself that contains a I-X services, such as planning aids and knowledge-based systems.
description of the virtual world capabilities it
should know about, and information on how to zones based on the flow of “observe, orientate,
communicate instructions to those capabilities. decide, act” (the “OODA Loop”) alongside the
Specific capabilities to provide flexible display software to make for simple deployment and
of images, external webpages, and I-X agent- setup for trials.
oriented information are also incorporated. Some I-Rooms have been running continu-
The I-X helper provides a sensor for determin- ously since early 2008, and a number have been
ing when avatars come into range, so that the used for live events, workshops, collaboration
helper can report them as joining and leaving meetings and discussions, training exercises,
the I-Room or meeting. The I-X helper also lis- product design and review meetings, scien-
tens on a specific chat channel for instructions tific project reviews, social occasions, and so
that it can handle itself either using external on. This has included meetings in which par-
I-X services or virtual world agents, or object ticipants have been located on three continents.
capabilities it’s told about through the capa- Some I-Rooms are constantly available to their
bilities note card. This lets avatars and other users through publicly accessible virtual worlds
objects in the virtual world use the I-X helper such as Second Life. Others are deployed rap-
to request services, such as noting action items idly (within minutes) on demand. The Artificial
or taking minutes and recording decisions. In Intelligence Applications Institute (AIAI) at the
general, it also lets I-Room participants interact University of Edinburgh regularly opens an
with external I-X agents and lets those agents I-Room in support of teleconferences to give a
communicate with and control devices in the visual indication of presence, rich media shar-
virtual world (see Figure 2). ing, and simple ways to initiate back-channel
More details about I-Rooms and the ways interactions for participants, even when tradi-
in which we can use them in virtual worlds is tional video and audio channels outside those
available at http://openvce.net/iroom. available through the virtual world are in use
with collaborators. We’ve worked with compa-
I-Room Applications nies such as Disney, EADS (Airbus), Glenkeir
At the University of Edinburgh, we have Distilleries/The Whisky Shop, Kodak, Slam
deployed I-Rooms in Second Life (on publicly Games, and others.
accessible areas) and in OpenSimulator (on I-Rooms are also being applied to a range
privately hosted servers that can run behind of national and international crisis and emer-
secure firewalls, if necessary). The software gency response situations,4 homeland security,
for the external I-X services and an example unmanned aerial vehicle (UAV) mission moni-
virtual-world-based I-X helper are available as toring, team training, and simulation exercises.
open sou rce code. We’ve prov ided sa mple One series of experiments for the Whole of
I-Room 3D models that support the workflow in Society Crises Response Community (WoSCR)
typical operation centers — for example, with involved a regional response to an escalat-
a central meeting space and surrounding work ing swine flu incident.5 WoSCR used a virtual
SEPTEMBER/OCTOBER 2011 59
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Virtual World Architectures
world I-Room for meetings, alongside a Web Acknowledgments
2.0 team collaboration website and wiki. We The I-X and I-Room projects have received funding from
performed an evaluation via questionnaires several sources, including DARPA, the US Joint Forces
during and after the experiments, and the par- Command/Army Research Labs/Alion (OpenVCE.net proj-
ticipants reported that the facilities offered ect), the European Regional Development Fund, and the
positive advantages over traditional methods of School of Informatics at the University of Edinburgh.
meeting and sharing information, which usu- The university and project funding partners are autho-
ally involve teleconferences, email, and file rized to reproduce and distribute reprints and online
sharing. copies for their purposes notwithstanding any copyright
annotation hereon. The views and conclusions contained
herein are those of the authors and should not be inter-
n I-Room provides a shared persistent space
A with intelligent systems support for interac-
tion and collaboration between users, systems,
preted as necessar ily representing the off icial poli-
cies or endorsements, either expressed or implied, of other
parties.
and agents. It allows for the integration of a
range of intelligent system aids, services, and References
agents into the meeting. An I-Room consists of 1. A. Tate, “Intelligible A I Planning,” Proc. 20th Brit-
elements inside a virtual world and external ish Computer Society Special Group on Expert Sys-
knowledge-based and intelligent systems. This tems, Int’l Conf. Knowledge Based Systems and Applied
especially includes the I-X planning, process, Artificial Intelligence (ES 2000), Springer, 2000,
and task support aids, but has also involved pp. 3–16.
knowledge-based and expert systems to access 2. A. Tate, “: An Ontology for Mixed-Initiative
large semantic knowledge stores, and natural- Synthesis Tasks,” Proc. Workshop on Mixed-Initiative
language-generation capabilities. Intelligent Systems (MIIS), Int’l Joint Conf. Artificial
At the University of Edinburgh, we’re refin- Intelligence (IJCAI 03), ijcai.org, 2003; www.aiai.ed.ac.
ing the core I-Room artificial intelligence con- uk/project/ix/documents/.
_______________
cepts and technology and making them more 3. A. Tate, J. Dalton, and J. Stader, “I-P2 — Intelligent
generic. This work includes the development of Process Panels to Support Coalition Operations,” Proc.
generalized links to knowledge-based systems; 2nd Int’l Conf. Knowledge Systems for Coalition Opera-
capability modeling to identify and exploit tions (KSCO 02), ksco.info, 2002, pp. 184–190.
opportunities in virtual worlds; and seman- 4. A. Tate et al., “I-Room: A Virtual Space for Intelligent
tic tagging of various media and communica- Interaction,” IEEE Intelligent Systems, vol. 25, no. 4,
tion streams that constitute a virtual meeting 2010, pp. 62–71.
to allow a higher level of context-sensitive 5. A. Tate et al., “Virtual Collaboration Spaces and Web
support, with documentation, indexing, and 2.0: Bringing Presence to Distributed Collaboration,”
playback facilities. We’re creating several vir- Reshaping Research and Development Using Web 2.0-
tual world capabilities to augment I-X support Based Technologies, M. Baker, ed., Nova Science Pub-
for intelligent interaction in virtual meeting lishers, 2011.
spaces, virtual operations centers, and training
rooms. Work is also under way to explore the Austin Tate is the director of the Artificial Intelligence
synergy between physical instrumented meet- Applications Institute (AIAI) and holds the Personal
ing spaces and virtual-worlds-based I-Rooms to Chair of Knowledge-Based Systems at the University
better support collaborative distributed decision- of Edinburgh. His research interests include emergency
making groups. response using advanced knowledge and planning
We’re able to provide intelligent decision- technologies, and collaborative systems, especially
support tools independent of virtual worlds, using virtual worlds. Tate has a PhD in machine intel-
but our experience indicates coupling them ligence from the University of Edinburgh. He’s a fellow
has advantages. Intelligent systems can be of the Royal Society of Edinburgh, a fellow of AAAI,
layered on top of existing virtual worlds and an IEEE Intelligent Systems senior advisory board
platforms, and this is facilitated by scripting a.tate@ed.ac.uk.
member. Contact him at _________
facilities in the object-oriented programming
environments that many flexible virtual worlds Selected CS articles and columns are also available
provide. for free at http://ComputingNow.computer.org.
60 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
___________________________
____________________
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Access Control
A User-Activity-Centric
Framework for Access Control
in Online Social Networks
Today’s ever-evolving online social networks (OSNs) need an effective and
usable access control framework. OSN users typically have discretionary
control over their content, relationships, and interactions, while the OSN’s
policies consolidate these individual choices into specific access and filtering
decisions. OSN access control can be built around the concept of user activity.
To this end, the authors distinguish usage activity from control activity and
identify four core control activities: attribute, policy, relationship, and session.
Their user-activity-centric framework enables future extensions as needed.
O
Jaehong Park, Ravi Sandhu, nline social networks (OSNs) related users’ preferences and policies.
and Yuan Cheng present a domain that’s distinct Consider the user relationship graph
University of Texas at San Antonio from traditional access control. that Figure 1a shows. Her e, Homer
Although discretionary access control m ig ht not wa nt h i s coworkers to be
lets users configure access to their own notified of his activity. He might also
resources, they typically do so in terms want to prevent Bart from viewing any
of user identities, group or role mem- violent content, sharing contact infor-
bership, and similar attributes. Access mation, or becoming a friend of Hom-
control in OSNs is driven more by user er’s coworkers. We call the expression
relationships based on social graphs, of Homer’s policies control activities. In
such as friends and friends of friends. both lattice- and role-based access con-
In typical access control systems, a trols, such control activities are admin-
user accesses stored content, whereas istrative ones — that is, administrators
in OSNs, additional activities occur, or security officers define control poli-
such as “poking” another user or rec- cies for users. In OSNs, users participate
ommending other users as friends. The in control activities on related users
targets of these activities are other users and content.
rather than shared content. Myriad OSN services are available
Furthermore, OSN systems make today, but users’ control capabili-
and enforce control decisions for user ties within these services are still rudi-
activities by collectively referencing mentary and will likely require further
62 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
A User-Activity-Centric Framework for Access Control
enhancement. For instance, a user might not want Coworker
Homer Mr. Burns
to reveal his location information or might want
to use additional privacy rules on some occa-
sions. Current OSNs rarely offer such options. Parent of Friend
In this article, we propose developing an
access control framework for OSNs around the
concept of user activity. Our framework accom- Bart Ned
modates personalized privacy preferences for (a)
user activities and resources by separating indi-
vidualized user and resource policies. Its scope Activities (A) Attributes (T)
OSN’s
goes beyond traditional access control in that it Policies (P)
activity
lets users control general usage activity as well decision
as control activities such as attribute, policy, Target
relationship, and session controls. users
(UT)
Users Sessions Action
Access Control Framework (U) (S) (ACT)
Target
Figure 1b shows a conceptual depiction of resources
our framework (its formalization is beyond Attributes (T) Attributes (T)
(RT)
our scope here). It comprises three main com- Policies (P) Policies (P)
Attributes (T)
ponents: users, sessions, and activities. Each Constrained by Policies (P)
activity consists of an action, zero or more tar- (b) (for example, subset)
get resources, and zero or more target users.
Figure 1. User-activity-centric framework. We can see (a) an
Users example of online social network (OSN) user relationships and
A user is a representation of a human and is (b) the various framework components.
associated with user attributes and policies. User
attributes are properties or information about the privileges if the session is on a mobile device).
user, such as a unique ID, name, address, age, or A user can have multiple concurrent sessions if
friend list. User policies are rules expressing pref- the OSN permits, whereas a session belongs to
erences or limits. The user or his or her related exactly one user (indicated by the double versus
users (such as parents) directly manage some single arrowheads in the figure).
attributes and policies. The OSN system manages Although current OSNs don’t support this
others, often as a consequence of various user capability, we believe future OSNs will find it use-
activities (as with consumable attributes, such as ful to support sessions with user-controlled attri-
a credit balance, or a reputation attribute based butes and policies. For instance, a user might be
on aggregated ratings from other users). allowed to disable some attributes or policies in
some sessions, as when Homer doesn’t want to
Sessions reveal his friends’ information to other users. He
A session is a representation of an active user can achieve this by creating a session that doesn’t
who has logged into the OSN (we borrowed the convey his friends’ information. On the other
term from role-based access control models1). hand, some user attributes and policies might need
The user-versus-session distinction is impor- to be required for a session that performs cer-
tant if only to distinguish between those who tain actions. For example, an OSN system might
are online and those who aren’t. In the sim- mandate some user attributes and policies in all
plest case, a session inherits all the user’s attri- sessions, such as a user ID or a basic geographic
butes and policies. More generally, a session location. We believe the relationship between ses-
might inherit only some, or might inherit them sion and user attributes and policies provides a
in a slightly modified form, such as substitut- fertile arena for developing more nuanced access
ing “over 18” for an actual age (represented control and privacy in future OSNs.
via the “constrained by” relation in Figure 1b).
A session might have additional attributes (such Activities
as an IP address or access to a device and its The notion of activities encompasses both gen-
location) and policies (for instance, limited eral usage activities and users’ control activities.
SEPTEMBER/OCTOBER 2011 63
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Access Control
A session initiates each activity on the user’s If Homer’s session has a policy that says it
behalf. The OSN decides whether the activity is doesn’t ever want to chat, Ned’s attempt to chat
permitted. A session can have multiple activi- will fail.
ties, whereas each activity is initiated by only
a single session. Each activity comprises an OSN Activity Decision
action, target resources, and target users. Ultimately the OSN system consolidates all the
necessary individual policies and attributes
Action. Each action is an abstract function together with its own policies and uses them to
available to OSN users via a session. Examples decide whether to permit specific users’ activity
include when a user reads or writes a comment, requests. Assume Homer has a policy that says
likes another user’s posting, invites another anyone who is his coworker or a direct friend of
user to be a friend or group member, or indi- his coworker can’t be a friend to his children.
rectly triggers an activity notification action Using this policy, the OSN makes sure Bart’s
that’s delivered to friends. User actions can be policy reflects Homer’s policy by either updat-
carried out on target resources, target users, ing Bart’s policy or evaluating Bart’s parents’
or both. For example, read and write actions policies each time Bart attempts an activity.
require target resources, whereas friendship If Bart (in a session) tries to send a friendship
recommendation actions require two or more invitation (an action) to Ned (a target user),
target users, and typical notification actions the OSN evaluates Bart’s policy and possibly
require both (that is, multiple target users will those of his parents, then verifies whether any
receive notification of an acting user’s activity of Ned’s friends (the target user’s attribute) are
information, such as a comment on a picture). Homer’s coworkers.
Target resources. Target resources are those Discussion
involved in an action. They can include users’ Our framework has some distinctive charac-
shared content; profile information; user, teristics. The first is policy individualization,
resource, or session policies and attributes; and which is essential for access control in OSN
any other digital information that users can environments. Unlike in traditional access con-
access or manage in the OSN. By considering pol- trol systems — such as lattice- or role-based
icies and attributes (in addition to shared content) access control, where a single, system-wide
to be part of the resource abstraction, our frame- security policy is applied to all users — OSN
work supports users’ ability to partially control users have their own security and privacy poli-
their own attributes and policies as well those of cies and attributes, which the OSN uses col-
related users. Furthermore, the framework cov- lectively to make decisions on user activities.
ers the policies and attributes of these policy and Individuals or related users can manage these
attribute resources. For example, Bart’s “no access policies and attributes themselves.
to violent content” policy could have its own pol- Another characteristic is the separation of
icy stipulating that only Homer can change it, or user and resource policies. Some policies are
an attribute that provides information about the specific to individual users, whereas others
policy creator. As another example, a video clip’s are specific to resources, so certain activity
provider attribute can have a policy that says controls should be enforced with user policies
only the provider’s friends can read the attribute (such as a filtering policy2) and others using
information. Although, theoretically, this chain- resource policies. For instance, using resource
ing can continue indefinitely, we believe practi- policies to filter out violent content from Bart
cal OSN systems won’t likely provide policies and (and other users) would require adding one
attributes on policies and attributes beyond one rule per excluded user in the resource policies
or two levels. of every violent resource, which isn’t scalable.
Including the rule “no access to violent con-
Target users. Target users are the recipients of tent” in each excluded user’s policy is better.
an action. For example, if Ned invites Homer Unlike others’ work on OSNs, 2–6 which
as a friend or for a chat, Homer’s the target focuses exclusively on user relationships, our
user while Ned is the acting user. (More pre- framework also supports user-relationship-
cisely, Homer’s sessions receive the invitation.) independent access controls. More specifically,
64 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
A User-Activity-Centric Framework for Access Control
it can support attribute-based access control in Acknowledgments
general, such as the authorization component of This work is supported by grants from the US National
usage control.7 Science Foundation and the state of Texas.
Our framework also supports sessions
that represent active users, which allows for References
enhanced controls that we don’t find in existing 1. R.S. Sandhu et al., “Role-Based Access Control Models,”
OSN services and literature. Specifically, a user Computer, vol. 29, no. 2, 1996, pp. 27–38.
can minimize shareable attributes and change 2. B. Carminati et al., “A Semantic Web-Based Framework
his or her policies to have better security and for Social Network Access Control,” Proc. 14th ACM
privacy control, while the OSN system ensures Symp. Access Control Models and Technologies, ACM
that this doesn’t violate other users’ policies. Press, 2009, pp. 177–186.
Many existing OSNs (such as Facebook or 3. B. Carminati, E. Ferrari, and A. Perego, “Enforcing
MySpace) allow a session with some additional Access Control in Web-Based Social Networks,” ACM
attributes or policies that the OSN controls but Trans. Information and System Security, vol. 13, no. 1,
don’t enable any user-controllable session attri- 2009, pp. 1–38.
butes or policies. Much of the recent literature 4. P.W.L. Fong, M. Anwar, and Z. Zhao, “A Privacy
on OSN access controls doesn’t distinguish a Preservation Model for Facebook-Style Social Network
session from a user.2–6 Systems,” Proc. 14th European Symp. Research in
The recent OpenSocial specification seeks Computer Security, Springer, 2009, pp. 303–320.
to standardize API language specifications for 5. P.W.L. Fong, “Relationship-Based Access Control:
OSNs,8 and is complementary with our frame- Protection Model and Policy Language,” Proc. ACM
work. Proposals for OpenSocial Access Control Conf. Data and Application Security and Privacy
Lists (ACLs), Activity Privacy API, and Album (CODASPY 11), ACM Press, 2011.
and MediaItem Privacy API suggest API speci- 6. A. Cinzia Squicciarini, M. Shehab, and F. Paci,
fications for ACLs that are attached to resources “Collective Privacy Management in Social Networks,”
in OSNs.9 Unlike our framework, OpenSocial Proc. 18th Int’l Conf. World Wide Web, ACM Press,
narrowly defines activity to mean information 2009, pp. 521–530.
(a log) about events (such as user actions), which 7. J. Park and R. Sandhu, “The UCON ABC Usage Control
our framework views as a resource. Thus, the Model,” ACM Trans. Information and System Security,
OpenSocial Activity Privacy API is mainly for vol. 7, no. 1, 2004, pp. 128–174.
user activity notification controls and defines 8. OpenSocial Specification 1.1, OpenSocial, 2010; www.___
a specification language for policies that are opensocial.org/specs.
attached to the user activity log. In our frame- 9. C. Renner, Privacy in Online Social Networks, master’s
work, users can control activity notification by thesis, Swiss Federal Institute of Tech., Zurich, 2010.
specifying either user policies or resource (for
example, activity log) policies, depending on Jaehong Park is a research associate professor at the Insti-
whether the notification policy applies to a spe- tute for Cyber Security at the University of Texas at
cific user or a specific activity. jae.park@utsa.edu.
San Antonio. Contact him at ___________
n contrast to traditional access control appli-
I cation domains, OSNs are uniquely centered
around users’ usage and control activities.
Ravi Sandhu is the founder and executive director of the
Institute for Cyber Security, holds the Lutcher Brown
Endowed Chair in Cyber Security, and is a professor in
Studying access control issues simply based the Department of Computer Science at the University
on user relationships is insufficient to com- of Texas at San Antonio. Contact him at ravi.sandhu@
________
prehensively understand security and privacy utsa.edu.
_____
issues in OSNs. Our proposed user-activity-
centr ic f ramework prov ides a concept ual Yuan Cheng is a doctoral student in the Department of Com-
sketch for understanding the essential nature puter Science and the Institute for Cyber Security at
of OSN access control. This framework will the University of Texas at San Antonio. Contact him at
provide a foundation for future development ycheng@cs.utsa.edu.
____________
of access control policies and models for OSNs
with enhanced security and privacy protection Selected CS articles and columns are also available
support. for free at http://ComputingNow.computer.org.
SEPTEMBER/OCTOBER 2011 65
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Web-Scale Workflow
Editor: Schahram Dustdar ________________
Principles of
Elastic Processes
Cloud computing’s success has made on-demand computing with a pay-as-you-
go pricing model popular. However, cloud computing’s focus on resources and
costs limits progress in realizing more flexible, adaptive processes. The authors
introduce elastic processes, which are based on explicitly modeling resources,
cost, and quality, and show how they improve on the state of the art.
P
rocess automation and workf lows are model. Such a service economy mechanism
familiar concepts in modern computer sci- should be an integrated part of process
ence. Increasingly, data-intensive applica- models.1
tions play a crucial role in this domain — our Quality of service (QoS) within processes.
online and interconnected society produces Because services realize each process in a
massive amounts of data. Sources include workflow, QoS becomes an important notion
sensor-equipped environments, such as smart for two reasons. First, when we uniformly
buildings, social media, and financial markets. regard computation as service, we can view
To harvest the valuable information hidden in a workflow as a compositional service. Thus,
these “data blobs,” we can often apply the con- its quality must be well defined by the qual-
cept of processes to streamline data processing ity of its component services. Second, QoS is
and analytical steps. Currently, we can apply related to the resources services require and
such processes for both static and real-time data thus the cost of those resources.
from different sources and deliver the analytical
results within a structured enterprise comput- We propose the concept of elastic processes
ing environment. However, we argue that such (EPs), precisely defining the various facets of
a computing paradigm lacks some necessary elasticity that capture process dynamics in
features for modern Internet-scale information cloud and human computing. The main prop-
processing, where both cloud and human com- erties for modeling EPs’ economic and physical
puting1 are heavily employed. dynamics are resource elasticity, cost elasticity,
Cloud computing and human computing have and quality elasticity (the “Elasticity in Related
the following common features that we must Disciplines” sidebar provides the general defini-
address for process automation: tions for elasticity that we consider in our work).
Elasticity captures one essence of cloud com-
Dynamic resource requirement and provi- puting: when limited resources are offered for
sion. Both cloud and human computing potentially unlimited use, providers must man-
environments are based on the concept of age them elastically by scaling up and down, as
provisioning adequate resources as services needed. However, as is common today, under-
in a demand-driven fashion based on a price standing and supporting elasticity purely from
66 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Principles of Elastic Processes
Elasticity in Related Disciplines
I n computer science, the term
been used as the academic synonym of
has recently
thanks to Amazon’s premier cloud service offering, the Elastic
,
elasticity measures a function’s responsiveness or sensitivity to
changes in parameters in a relative way. In general, the formula
for the elasticity of Y with respect to X is
Compute Cloud (EC2). dy X
e (Y , X ) = ,
The current Wikipedia definition of elasticity in physics dx Y
states that “elasticity is the physical property of a material where (Y, X) is short for “the elasticity of Y with respect
when it deforms under stress (for example, external forces) to X,” and / is the derivative of Y with respect to X. In
but returns to its original shape when the stress is removed. economics, elasticity is an effective way to measure demand
The relative amount of deformation is called the strain.” When and supply responsiveness. This notion of elasticity should be
applied to computing, elasticity naturally reflects the on- adequate to apply to the resource, quality, and cost dynamics in
demand nature of cloud service provisioning: it states that the service-oriented computing, especially in the context of cloud
amount of resources an application uses or a provider offers computing.
can expand or contract based on influences such as demand.
Another related definition of elasticity is found in econom- Reference
ics, which describes it as “the ratio of the percent change in one 1. E. Dowling, , 3rd ed., McGraw-Hill,
variable to the percent change in another variable.”1 That is, 1980.
a resource-management viewpoint is crucial for future processes in the elasticity concept. Taking Amazon
rather restrictive. Resources’ require- context of service-based comput- as an example, the following price
ments aren’t determined only by the ing. Let’s look more closely at cost models are based on cost elasticity
application using them. If we really and quality elasticity, which are estimation:
treat computation as a service, then discussed much more rarely than is
we must consider all aspects of a ser- resource elasticity. On-demand instances are a pure
vice that might impact the demands pay per use-on-demand model, in
on a resource. Cost Elasticity which customers don’t have long-
The proposed EP is a novel con- Cost elasticity describes a resource term commitments and are free
cept that significantly enriches com- provision’s responsiveness to changes from planning.
putational processes’ properties in in cost. Service providers apply it Spot instances occur when spot
the context of cloud computing and when defining price models for cloud prices fluctuate over time accord-
service-oriented computing in gen- computing systems. In this context, ing to supply-demand status and
eral. Existing workflows are limited cost elasticity is also referred to as other factors Amazon consid-
to resource elasticity by adjust- utility computing, in which resources ers. Users bid a maximum price
ing machine power, while cost and such as computational services pro- they’re willing to pay for these
quality are barely considered. How- vided by virtual machines, data instances and run them as long
ever, these three main properties are transmission on the network, and as the spot price bidding price,
interdependent, and we must study storage services provided on differ- until the instance is explicitly
them based on a uniform founda- ent storage hierarchies are charged terminated, or the price rises
tion. Our aim is to build a proper based on a pay-as-you-go pric- above users’ bidding price.
modeling, reasoning, and execution ing mechanism. In defining a price
framework in which we can specify model for utility computing, the cost With the spot price option, Ama-
and monitor these properties to build incurred to support the computing zon can use higher spot prices dur-
a quantifiable, proactive, and predic- capacity level is the baseline for the ing peak times and lower prices
tive resource-capacity-management design. These cost items include the during off-peak times to shape cus-
system for Internet-scale process investment, provisioning, and main- tomer behaviors such that flexible
automation that integrates multiple tenance of processor, memory, hard users would tend to consume more
clouds and various forms of human disk, and network with, respectively, during off-peak times and avoid
computing. desired clock frequency, memory pu rchases du r i ng pea k t i mes.
size, size of disk space used, and data This would flatten aggregate usage
Elasticity Properties transmission cost. Based on these fac- over time, which, in turn, would
We’ve identified elasticity consider- tors, providers can develop dynamic decrease Amazon’s maintenance
ing resources, cost, and quality as pricing models based on the cost costs. In this sense, price is intuitively
SEPTEMBER/OCTOBER 2011 67
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Web-Scale Workflow
Elastic process system Costs as Figure 1 illustrates. We identified
Costs five primar y research challenges
Resources
Customer 1 that informed our model’s design,
Quality Resource provider 1
and discuss these in detail later.
First, let’s look at EPs’ physical and
economic properties.
- Elastic reasoning
mechanism (RC 3) Physical Elasticity Properties
- Reusability and adaptive Costs An EP must decide how to use exist-
execution (RC 4)
Costs
- Formalism for elastic Resources
ing resources in its environment in
Customer n process system (RC 5) an optimal way (one that can meet
Quality Resource providers m
multidimensional demands but with
Specification of constraints Self-describing a maximum benefit). The EP envi-
and preferences (RC 1) resources (RC 2) ronment is dynamic, with diverse
resou rce t y pes (computat iona l,
data, and network resources). These
resources are also dynamic, as are
their quality and cost models. Based
controlled not only by cost elasticity Response time isn’t the only qual- on quality and cost, an EP might use
but also by the incentive effect on ity criteria used. Other quality mea- different sets of resources as well as
customers. surements such as the result quality its processing activities to produce
in an approximation-based comput- multiple outputs. On the other hand,
Quality Elasticity ing process can help provide a new some demands might have similar
Quality elasticity measures how class of cloud algorithms. The Aqua requirements, so the same resources
responsive quality is to a change in approximate query answering system and processing elements in the EP
resource usage. The elasticity comes developed at Bell Labs is an example can produce multiple outputs. Such
from a feature inherent to cloud of a system that makes trade-offs behaviors ref lect an EP’s internal
applications — that is, to have a considering quality aspects in query physical elasticity properties.
well-defined quality elasticity mea- processing.3 Traditional query pro-
surement, an application service’s cessing focuses on generating exact Economic Elasticity Properties
underlying algorithm requires that answers. However, when huge data First, let’s distinguish between an
the service’s quality improvement stores are involved, providing an EP and resources for building EPs,
be monotonic to the consumption of exact result might take an unaccept- which can be any kind of machine
the resource needed. In other words, ably long time. In many cases, exact or human computation and network
the more resources consumed, the answers aren’t required, and approx- resource; machine computation can
better the achievable quality. The imate or quick results are preferred. come from (virtual) computational
main issue here is to associate a ser- Aqua is a system for quickly execut- machines or software services atop
vice with a measurable quality and ing queries by providing approximate machines. Providers make resources
the cost function, which computes answers tailored to data warehous- available, and each resource has cer-
the resource requirement for a given ing environments. When we couple tain properties, such as quality and
quality, such as execution speed. In such an approximation process with cost. An EP’s function (for example,
this case, a service’s result is deter- a monotonic resource consumption translation) is a static property that
ministic, but its execution speed is model, we can build an elastic que- accepts certain input data sources
scaled based on the required resource. rying system based on the notion of and produces some results. The func-
In cloud computing, some computa- quality elasticity. Recent research tion is modeled and implemented as
tional forms have this desired prop- in data space as an approximation- a set of interdependent activities. It’s
erty. For example, MapReduce is a based type of search computing is an built from existing components but
scalable programming framework important attempt toward an elastic differently than are static processes.
that lets users process data elasti- search paradigm.4 As with its physical elasticity
cally.2 It has a desired quality elas- properties, an EPs’ economic elastic-
ticity that states that execution speed Conceptual Model ity properties include resource, cost,
is scalable to the increase of servers To realize EPs, we propose a conceptual and quality elasticity. An EP uses
in a distributed file system. architecture of an EP environment, resources provisioned by any provider
68 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Principles of Elastic Processes
Partially Elastic Processes
S ystems considering quality or cost when deciding on
resource usage are not novel. The novelty is in explicitly
modeling quality, cost, and resources allowing for reasoning and
Current facility-management techniques have enabled sensor
infrastructures that can collect different types of facility infor-
mation. Furthermore, data resources available on the Internet,
making trade-offs. We call processes considering only parts of such as weather information and maps, can be combined with
these aspects “partially elastic processes.” One example can be facility data to support complex data analysis processes. In
found in the integration of machine and human capabilities for sensor networks, energy awareness is an essential property,
processing. Recently, we’ve moved from pure machine compu- and indeed a large body of research on energy-efficient sensor
tation processes (such as traditional, compute-intensive work- networks exists, mostly with a focus on routing, but also on
flows) to a combination of machine and human computation. energy-aware resource allocation for process-oriented tasks. 2
We’ve seen that people and software services can participate Because energy consumption generates costs, this can be seen
in processes to perform certain tasks, such as image evalua- as a partially elastic process as defined previously.
tion. Given that people have heterogeneous skills and interests,
human processing systems start to explicitly consider quality References
for “resource allocation” — that is, for assigning a task to a 1. B. Satzger et al., “Stimulating Skill Evolution in Market-Based Crowdsourcing,”
suitable worker. This can lead to results that meet predefined (BPM 11), to appear,
quality requirements.1 2011.
A further example of partially elastic processes can be 2. K. Akkaya and M. Younis, “A Survey on Routing Protocols for Wireless
found in data analysis in sustainable facilities and smart cities. Sensor Networks,” , vol. 3, no. 3, 2005, pp. 325–349.
at any place and used at any time, satisfies its requirements. Ultimately, the refinement and composition of
as long as their capabilities meet the an EP can deal with multiple service the EP’s resource, cost, and quality
constraints the processes require, objectives. In the simplest case, the to different levels — activities within
such as minimum spending costs. EP would serve one consumer (as an EP, fragments within an EP, and
Essentially, resource elasticity is an with an analysis of Facebook activi- the whole EP — and also apply the
internal property that isn’t exposed ties) and utilize one provider (such as different operation and modeling
to consumers. For quality elasticity, Amazon). In the most extreme case, principles at these levels.
however, an EP can offer different an EP will have N concurrent con-
models, which are accessible to the sumers and access to a market of M Research Challenges
users. They depend on functions, providers. N consumers would give Existing solutions haven’t been able
costs, and resources used. Simi- K requirements (input data, cost, to deal with all the properties we’ve
larly, an EP considers different cost quality), and K N. So, EPs must be mentioned (the “Partially Elastic
models and presents those models to able to deal with trade-offs between Processes” sidebar provides exam-
consumers. requirements. ples for existing solutions). To build
EPs have several properties that real systems with these properties,
Operation and enable them to compose modeling we must address several research
Modeling Principles principles, including overlaying EPs, challenges for interfaces between
In our view, an EP’s basic operation function composition, and dynamic EPs, consumer demands and envi-
principles are its ability to monitor, property composition. We can out- ronments, and elastic properties.
manage, and describe dynamic prop- line modeling principles as follows.
erties; the dynamic refinement of An EP must model its function as a Specification of
process functions based on quality static property. The EP’s results are Constraints and Preferences
(that is, new functions such as data based on requirements concerning Compared to traditional process
enrichment or data cleaning can be cost and quality, modeled as a set execution, elasticity requires giving
added to improve quality); the abil- of constraints; this model influences more autonomy to the infrastruc-
ity to determine cost based on mul- the resource elasticity. Furthermore, ture and the processes themselves.
tiple resource cost models; and the modeling can also describe how an Each process consumer or user who
ability to provide elasticity across EP can communicate with other EPs. wants to utilize the EP system (EPS)
providers — that is, an EP could This communication can be based on defines a process enriched with con-
spread and combine components the abstraction of a service interface straints and preferences specifying
from different providers, as long as it such as REST or SOAP. We can apply cost and quality trade-offs. The EPS
SEPTEMBER/OCTOBER 2011 69
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Web-Scale Workflow
takes this tuple and will eventually based on a skill profile, track record, based on heuristics and partial infor-
present the result to the user. How- or whether the human is available to mation are needed. Techniques such
ever, users must still be able to control process some task. as prediction, optimization, auctions,
the system behavior with simple and virtual markets are candidate
and intuitive interfaces. They need Elastic Reasoning Mechanism ingredients for the final adaptive
a means to express their constraints With multidimensional dynamic execution recipe.
and preferences in a human-centered demands, an EP must be equipped The EPS allows for adaptive
way. They should make statements with an elastic reasoning mecha- process execution and can react to
about cost and quality rather than nism (ERM) to decide how to utilize changes in the environment and par-
resources. Intuitive human-centered resources in an optimal way. We can tially merge processes for optimized
models need a mechanism for trans- regard an ERM as an optimization execution. In Figure 1, for instance,
lation into computer-readable for- system that takes dynamic resource the blue and green processes share a
mats and vice versa if the system is to and cost information from the common computation, which we can
interact with users about constraints environment to maintain a cloud’s reuse for efficient execution.
and preferences (for instance, by rec- dynamically generated capacity and
ommending removing a constraint, price information (computational, Formalism for Elastic
resulting in high costs and low qual- data, and network resources). Such Process Systems
ity gains). an environment is usually available A formal system for studying elastic
as part of a cloud management plat- computing can contribute to model-
Self-Describing Resources form, such as Eucalyptus.5 ing and understanding EPs. As in
For the actual processing, the EPS any process calculus, such a system
maps parts of the processes onto Reusability and must be built on a well-defined set
resources (machines or humans), Adaptive Execution of operators over processes. Differ-
taking into account the specified Executing processes in an elastic ent from traditional communicating
requirements. Thus, it must know way, in compliance with user-defined process calculi, the system’s opera-
about available resources’ exis- constraints and preferences, can be tors should mainly focus on model-
tence and capabilities. To that end, highly challenging. While several ing processes’ elastic features and
resources must provide a descrip- related works on adaptive process their composition.
tion containing information about execution exist, they generally don’t
their availability and corresponding consider combined resources, costs,
costs. and quality. Existing refinement e’ve identified cost and qual-
The challenge here is that we
envision EPs “living” in heteroge-
techniques for process structures,
for instance, focus on performance-
W ity as main facets to consider
for process execution. We argue that
neous environments with different related qualit y (such as ser vice future processes should be able to
hardware resources, load character- availability) but not on result quality take a description of quality and cost
istics, administration, ownership, (better images). Runtime refinements requirements. The execution envi-
laws, and privacy policies. Each are basic — for instance, component ronment needs the intelligence to
resource must deal with this degree replacement — while complex refine- determine the actual resource usage
of heterogeneity to describe itself. ments such as fragment replacement based on that description. This leads
Different levels of detail are pos- are supported only in offline (not to elastic processes.
sible, and some information will be continuous and elastic) processes.
optional, but the description should To achieve a trade-off between these References
be comprehensible to anyone. aspects in a large-scale heteroge- 1. S. Dustdar and K. Bhattacharya, “The
To improve scalability, we propose neous environment requires addi- Social Compute Unit,” IEEE Internet Com-
a hierarchical description methodol- tional research efforts. puting, vol. 15, no. 3, 2011, pp. 64–69.
ogy: a cloud could, for instance, have Because the environments we’re 2. J. Dean and S. Ghemawat, “Map-
its own description that’s an aggre- considering are highly dynamic, Reduce: Simplified Data Processing on
gation of the “sub-cloud” description, process execution can’t be sluggish Large Clusters,” Comm. ACM, vol. 51,
which, in turn, comprises numerous or even static. It must focus on con- no. 1, 2008, pp. 107–113; http://doi.acm.
single machines, each with its own tinuous monitoring and re-planning. org/10.1145/1327452.1327492.
description, too. Resources might In such large, complex environ- 3. S. Acharya et al., “The Aqua Approxi-
also be humans (or social compute ments, exact algorithms drop out, mate Query Answering System,” Proc.
units1), whose description might be but approximate decision approaches ACM SIGMOD Int’l Conf. Management
70 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Principles of Elastic Processes
of Data (SIGMOD 99), ACM Press, 1999,
pp. 574–576; http://doi.acm.org/10.1145/ PURPOSE: The IEEE Computer Society is the world’s largest association of computing
304182.304581.
________ professionals and is the leading provider of technical information in the field.
4. K. Belhajjame et al., “Feedback-Based MEMBERSHIP: Members receive the monthly magazine Computer, discounts, and
Annotation, Selection, and Refinement opportunities to serve (all activities are led by volunteer members). Membership is open to
of Schema Mappings for Dataspaces,” all IEEE members, affiliate society members, and others interested in the computer field.
Proc. 13th Int’l Conf. Extending Data- COMPUTER SOCIETY WEBSITE: www.computer.org
base Technology, ACM Press, 2010, Next Board Meeting: 13–14 Nov., New Brunswick, NJ, USA
pp. 573–584.
EXECUTIVE COMMITTEE
5. D. Nurmi et al., “The Eucalyptus Open-
President: Sorel Reisman*
Source Cloud-Computing System,” Proc.
President-Elect: John W. Walz;* Past President: James D. Isaak;* VP, Standards
9th IEEE/ACM Int’l Symp. Cluster Com-
Activities: Roger U. Fujii;† Secretary: Jon Rokne (2nd VP);* VP, Educational Activities:
puting and the Grid (CCGRID 09), IEEE CS
Elizabeth L. Burd;* VP, Member & Geographic Activities: Rangachar Kasturi;† VP,
Press, 2009, pp. 124–131; http://dx.doi. Publications: David Alan Grier (1st VP);* VP, Professional Activities: Paul K. Joannou;*
org/10.1109/CCGRID.2009.93. VP, Technical & Conference Activities: Paul R. Croll;† Treasurer: James W. Moore,
CSDP;* 2011–2012 IEEE Division VIII Director: Susan K. (Kathy) Land, CSDP;† 2010–
Schahram Dustdar is a full professor of com- 2011 IEEE Division V Director: Michael R. Williams;† 2011 IEEE Division Director V
puter science (informatics) with a focus Director-Elect: James W. Moore, CSDP*
on Internet technologies and heads the *voting member of the Board of Governors †nonvoting member of the Board of Governors
Distributed Systems Group, Institute BOARD OF GOVERNORS
of Information Systems, at the Vienna Term Expiring 2011: Elisa Bertino, Jose Castillo-Velázquez, George V. Cybenko, Ann
University of Technology (TU Wien). Dust- DeMarle, David S. Ebert, Hironori Kasahara, Steven L. Tanimoto
dar is an ACM Distinguished Scientist. Term Expiring 2012: Elizabeth L. Burd, Thomas M. Conte, Frank E. Ferrante, Jean-Luc
Contact him at dustdar@infosys.tuwien.
______________ Gaudiot, Paul K. Joannou, Luis Kun, James W. Moore
ac.at; www.infosys.tuwien.ac.at/. Term Expiring 2013: Pierre Bourque, Dennis J. Frailey, Atsuhiro Goto, André Ivanov,
Dejan S. Milojicic, Jane Chu Prey, Charlene (Chuck) Walrad
Yike Guo is a computing science professor in
EXECUTIVE STAFF
the Department of Computing, Imperial Executive Director: Angela R. Burgess; Associate Executive Director, Director,
College London. His research is in large- Governance: Anne Marie Kelly; Director, Finance & Accounting: John Miller;
scale scientific data analysis, data min- Director, Information Technology & Services: Ray Kahn; Director, Membership
ing algorithms and applications, parallel Development: Violet S. Doan; Director, Products & Services: Evan Butterfield;
algorithms, and cloud computing. Con- Director, Sales & Marketing: Dick Price
yg@doc.ic.ac.uk;
tact him at __________ www.doc.
COMPUTER SOCIETY OFFICES
ic.ac.uk/~yg/.
_______
Washington, D.C.: 2001 L St., Ste. 700, Washington, D.C. 20036-4928
Phone: Fax: +1 202 728 9614
Benjamin Satzger is an assistant professor
hq.ofc@computer.org
Email: _____________
of computer science in the Distributed
Los Alamitos: Phone: +1
Systems Group, Institute of Informa- help@computer.org
Email: ___________
tion Systems, at TU Wien. Contact him Membership & Publication Orders
satzger@infosys.tuwien.ac.at; ___
at __________________ www. Phone: Fax: Email: help@computer.org
___________
infosys.tuwien.ac.at/staff/bsatzger/.
____________________ Asia/Pacific: Watanabe Building, 1-4-2 Minami-Aoyama, Minato-ku, Tokyo 107-
Phone: Fax: tokyo.ofc@
Email: ______
Hong-Linh Truong is a post-doctoral scientist computer.org
in the Distributed Systems Group, Insti-
IEEE OFFICERS
tute of Information Systems, at TU Wien.
President: Moshe Kam; President-Elect: Gordon W. Day; Past President: Pedro A.
truong@infosys.tuwien.
Contact him at ______________
Ray; Secretary: Roger D. Pollard; Treasurer: Harold L. Flescher; President, Standards
ac.at; www.infosys.tuwien.ac.at/staff/
___
Association Board of Governors: Steven M. Mills; VP, Educational Activities: Tariq
truong/.
____ S. Durrani; VP, Membership & Geographic Activities: Howard E. Michel; VP,
Publication Services & Products: David A. Hodges; VP, Technical Activities:
Donna L. Hudson; IEEE Division V Director: Michael R. Williams; IEEE Division VIII
Selected CS articles and columns Director: Susan K. (Kathy) Land, CSDP; President, IEEE-USA: Ronald G. Jensen
are also available for free at http://
____
ComputingNow.computer.org.
__________________
revised 2 August 2011
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
View from the Cloud
Editor: George Pallis ___________
Routers for the Cloud
Andrei Agapi, Ken Birman, Robert M. Broberg, Chase Cotton,
Thilo Kielmann, Martin Millnert, Rick Payne, Robert Surton,
and Robbert van Renesse
Today’s Internet often suffers transient outages, but as increasingly critical services
migrate to the cloud, much higher levels of Internet availability will be necessary.
T
he stunning shift toward cloud computing a software patch or migrated within the clus-
has created new pressures on the Internet. ter. The resulting sequence of events can take
Loads are soaring, and many applications several minutes, during which BGPD might be
increasingly depend on real-time data stream- unavailable or not yet fully resynchronized. The
ing. Unfortunately, the reliability of Inter- resulting routing changes can ripple throughout
net data streaming leaves much to be desired. the entire Internet, triggering routing events far
For example, at the University of Washington, from the one on which BGPD had to be restarted.
the Hubble system (www.cs.washington.edu/ Could events of this kind account for the issues
research/networking/astronomy/hubble.html)
__________________________________ Hubble saw? On a typical core router, it can take
monitors Internet health using all-to-all connec- two or three minutes to restart BGPD from scratch.
tivity and throughput tests between hundreds Moreover, BGPD might need to be restarted as
of end points through the Internet. The effort has often as once per week. Thus, it’s entirely possible
revealed transient periods of very indirect rout- that BGPD restarts are a significant factor.
ing, Internet “brownouts” (performance prob- In this article, we report on a new software
lems), and even “black holes.” All these problems architecture that can help mask BGPD outages,
are surprisingly common, even when looking at greatly reducing their disruptive impact. More-
routes entirely within the US or Europe. over, the same techniques should be applicable to
Here, we focus on routing in the Internet’s daemons associated with other important Inter-
core, at extremely high data rates (all-to-all data net routing protocols (we’ve already used the
rates of 40 Gbits per second are common today, approach for two different BGP implementations,
with 100 Gbits/s within sight). These kinds of and an Intermediate System to Intermediate Sys-
routers are typically implemented as clusters of tem [IS-IS] routing daemon). High-availability
computers and line cards: in effect a data center routers are just one of many developments that
dedicated to network routing. The architecture is will slowly reshape the Internet in response to
such that individual components can fail without the challenge and opportunity cloud comput-
bringing the whole operation to a halt. For exam- ing represents — the sidebar “An Internet for
ple, network links are redundant; if one link fails, the Cloud” describes how our efforts fit into this
there will usually be a backup. Such a router could shifting computing landscape.
even run routing protocols of different types side-
by-side, making the actual routing decisions by A Close Look at BGP
consensus — if some protocol instance malfunc- Before drilling down on BGP availability, it
tions, its peers would simply outvote it. might be helpful to be more precise about what
But suppose that a routing protocol (for clar- availability means for a core Internet router.
ity, we focus on the Border Gateway Protocol Routers drop packets during capacity overload
[BGP], implemented by a BGP daemon [BGPD] (TCP flow control adapts based on overall path
hosted on some node within the router) needs capacity), so it would make no sense to insist
to be restarted after a crash or updated with that a reliable router deliver every single packet.
72 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Routers for the Cloud
An Internet for the Cloud
C loud computing, particularly in conjunction with increased
device mobility, is reshaping the Internet. We’re seeing
unprecedented shifts in demand patterns, a broad spectrum of
that unless the Internet can evolve to meet the demands, the
associated cloud computing enterprises might consider building
new networks that would be dedicated to their use. Compa-
new quality expectations, and a realignment of the entire field’s nies such as Google, Netflix, Amazon, Microsoft, and others
economics. The implications are far-reaching. are insisting on the need to craft virtual enterprise networks.
The main text of this article focuses on , one If these are to share the same optical fibers used for other
of several key properties today’s cloud computing applications purposes, these and other cloud computing providers will need
demand. The need is most obvious in voice-over-IP (VoIP) tele- guarantees of disruption-free bandwidth, predictable laten-
phony and video streaming: for such uses, even the briefest cies, and hands-on control of routing policy control: “my traffic
disruptions can cause connections to seize up or fail in ways from A to B will traverse such-and-such a route,” or “requests
that are highly visible to the end user. If we can crack the “high- from user X will be routed to data center Y,” to list just a few
availability barrier,” we can imagine a future in which the Inter- examples. A new network-control paradigm has emerged (the
net carries all such traffic. so-called Open Flow standard; www.openflow.org) with enthu-
Yet high availability is merely the first step in what will be siastic backing from the cloud computing community. Moreover,
an evolutionary process. Cloud applications also need better with such a large part of the economy Internet-dependent,
techniques for guaranteeing steady, very high data rates; the there are growing calls to harden the network so that it can
ability to prioritize traffic; and robustness under routing-level offer rock-solid defense against attackers, be they hackers or
attacks. Content-distribution networks have been central to cyber warriors under command of national adversaries.
the static Web’s success: What will be the analogous paradigm The challenges are significant, but the payoff will also be big.
for the Web of dynamic content, such as video streams shared Today, many of the top technical people in the field are racing
by large numbers of users, gaming applications, or virtual reality to offer competing ideas. For many of the topics listed, rather
immersion? The answers to such questions could transform the than having no solutions, we might soon have a buffet of choices
Internet’s roles. to pick from. These are exciting times to work in the field of
Indeed, many cloud computing uses are so important (both networking, and the best part of the story is that so much of it
in the terms of their scale and the associated revenue streams) has yet to be written.
Accordingly, we adopt an approach Hardware and link failures jointly
first used in telephony, where avail- accounted for almost a third of out- Other causes
ability measures the percentage of ages. With redundant hardware and 9%
23% Router
time when almost all calls go through links, both factors have since been misconfiguration
36% IP routing
(that is, only a small percentage are sharply reduced — putting ever greater failures
dropped, and in an uncorrelated emphasis on IP routing’s reliability. 32% Physical link
failures
way). The wired telephone infra- This need for software that can
structure is engineered to guarantee sur vive hardware outages is vital
99.999 percent availabilit y: the because we must minimize the per-
“5-nines” standard. centage of time that the routes the
In a one-year reliability study of router is using are inconsistent with
IP core routers in a regional IP ser- those its neighbors use — for example,
vice provider network conducted by because the router has yet to apply
the University of Michigan, router routing updates that the neighbors
interface downtime averaged roughly are already employing. A more com-
955 minutes per year, which doesn’t plete discussion of IP routing failures
even reach the “3-nines” level. Figure 1 is available elsewhere.1
shows the breakdown of problems that BGP is designed for use in net-
this study identified. The results sup- works composed of interconnected
port the view that redundant hard- autonomous systems (ASs). An AS
ware has great potential: back in could be a network operated by some or set of ASs, tracking both direct
2004, when the university conducted ISP, or might be a campus or corpo- neighbors and more remote ones. A
the study, most deployed routers were rate network. BGP maintains a table BGPD instance runs on a router and
monolithic (nonclustered), and many of IP networks, or “prefixes,” that uses path availability, network poli-
links played unique, critical roles. represent paths to a particular AS cies, or operator-defined databases of
SEPTEMBER/OCTOBER 2011 73
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
View from the Cloud
Router control‐processor cluster
runs the FTSS service called graceful restart, which exploits
routing tables that were downloaded
into the hardware line cards prior to
Remote the crash. Assuming the crash left
BGPD the routing tables intact, when the
new BGP service starts up, the router
BGP state will still be running using the old
FTSS (d)
(c) routing table, a bit like an airplane
(a) on autopilot. The router won’t be
adapting to new routing updates and
R
P-R P- is thus frozen in time, but at least it
TC TC
was initially in a consistent state.
FTSS Graceful restart tells the neighboring
BGPD BGPD’
(b)
routers to continue to route packets
Shim through the impacted router, even as
the restarting BGPD resynchronizes
with its peers. The problem, however,
Original host Backup host
is that while this is happening, BGP
updates continue to stream in at a
furious pace, so routing tables can
become inconsistent within seconds.
This creates a strong motivation
shim to improve routing daemon avail-
fault-tolerant storage service ability. For example, some work has
distributed hash table aimed at running BGP in a movable
virtual machine (but VM migration
is slow, and offers no help for fault
tolerance), and some hand-tuned BGP
migration mechanisms exist. 2 Our
approach offers fault tolerance, can
support BGP upgrades (patching), and
works with routing daemons other
routing rules (patterns the operator BGPD is recovering when an update than BGPD, yet is fast and built from
has defined) to select preferred routes. arrives. surprisingly simple technologies.
It then advertises reachable prefixes Imagine that some router experi-
by publishing sets of attributes that ences an event that forces it to restart Fault-Tolerant BGP
include the paths. As routing changes, BGPD. When BGPD fails or migrates, Our new approach uses software to
BGPD exchanges updates with its the TCP links from it to the BGPDs on transform a standard BGPD imple-
peers that might add to the list of neighboring routers disconnect (break). mentation into a fault-tolerant ser-
reachable prefixes or retract some Those neighbors will sense the failure vice. It involves minimal changes
prefixes; those peers are expected to and try to route around the affected to the existing BGPD, the operating
update their own states accordingly. router, but the alternative routes might system, and existing protocols such
BGP allows BGPD instances to apply be poor ones, and sometimes no backup as TCP, IP, and UDP. The first step is
routing updates in an unsynchro- routes are available (recall that we’re to “wrap” BGPD in a fault-tolerance
nized, distributed manner, but nor- focused on the Internet’s core, where layer, the fault-tolerance shim. The
mally the delay between when one data rates are so high that only Internet shim helps the underlying routing
router applies an update and when “backbone” links and routers can han- protocol handle failures in ways
its neighbor does is negligible, hence dle the load). This, in turn, can trigger invisible to remote peers.
this asynchrony isn’t noticed: most secondary routing decisions at routers Figure 2 illustrates the approach.
routers are working with ver y further away, and so forth. The solution combines the existing
similar routing tables at any given So, how can we make BGPD BGPD with several new components.
moment. However, one important case more available? Currently, the main The first is fault-tolerant state stor-
exists where the lag can be larger: if approach is to activate a BGP feature age (FTSS), in which the shim stores
74 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Routers for the Cloud
BGP state and other data that must be FTSS key-value model to also support
preserved across failures. The second FTSS is a fault-tolerant storage solu- record linking and offers efficient ways
component is the shim itself. The solu- tion that saves and replicates state so to traverse linked data structures.
tion routes BGP connections between that in the event of a failure, a state-
BGPD and its peers through the shim, dependent component can recover its BGPD
so that the shim can see all incoming previous configuration. In our archi- As noted, we made only minor
and outgoing updates as well as any tecture, the shim is the only com- changes to the existing BGPDs with
changes to the routing table. This lets ponent that interacts directly with which we worked (we’ve applied our
the shim checkpoint all this informa- FTSS, using it to store the wrapped methodology to two, so far: Quagga
tion so that any incoming update will BGPD’s state, incoming and out- BGPD and a proprietary Cisco BGPD).
be securely logged in FTSS before our going BGP updates, the routing infor- The main change was to have BGPD
BGPD actually sees it, and any out- mation table, and a small amount connect to the shim rather than
going or routing table update will be of additional state associated with directly to its remote peers. A side
securely logged before being sent to TCPR. FTSS runs on all nodes within effect is that without further modifi-
a neighboring peer or installed into the router; in our target setting, this cation, when BGPD restarts, the shim
the hardware. would range from a few dozen nodes can supply the initial routing state:
The shim can also support multiple to several hundred. rather than informing remote peers of
routing protocols running side-by- FTSS is implemented as a one-hop, the restart, the shim itself senses the
side, a configuration that often arises in-memory, performance-optimized restart, pulls the needed state from
in the core Internet, where an AS distributed hash table (DHT). Each FTSS, and pushes it into BGPD at a
might have internal routing protocols state record has a unique ID (basi- very high data rate. In our experi-
t hat it uses to manage its own cally, a file name and a block num- ments, using state typical of real
network, and a separate BGP routing ber), and FTSS uses this as a key. core-Internet routing conditions, this
layer that talks to neighboring ASs. It The component maps the key to a took as little as 1.5 to 4 seconds. The
uses a form of voting to select among few nodes within the router (recall remote peers, of course, remain com-
competing routing “proposals” in such that the router is a cluster), and FTSS pletely unaware of the event. Finally,
cases, combining the routing protocol agents on these nodes replicate the when the remote peer set changes,
outputs to create the routing table that update. Lookup works the same way. BGPD informs the shim so that it can
will be downloaded into hardware. FTSS maintains full membership manage the associated connections.
Of course, the shim itself can expe- tables (with at most a few hundred
rience a failure, so we’ve designed it nodes in each router, and often far TCPR
to store its state in the FTSS, enabling fewer, the full address list easily fits TCPR is a TCP-splicing technology.
it to recover rapidly on a different in memory). Consequently, FTSS can The approach is best understood by
node. The last component of our solu- perform requests with a single RPC first considering the behavior of a
tion can “splice” the new TCP con- to each target node. FTSS also lever- standard NAT box: it has the effect of
nections (which the shim creates) to the ages parallelism: we break the BGP grafting a TCP end point that thinks
old TCP connections that it was previ- state into a large number of small itself to be connected to server X
ously using to connect to remote peers. chunks and spread these over many on port P to a server that might
Called TCPR (for “TCP with session machines, doing PUT and GET oper- really be running on machine Y using
recovery”), this splicing technology ations in parallel, and in this way port S. The NAT box translates
works somewhat like network address gain roughly an order of magnitude back and forth. TCPR works in much
translation (NAT), but rather than in speed. Even when we take into the same way but at the level of the
translating source and destination account delays associated with the byte-sequence numbering used within
addresses in NAT-style, TCPR also need to replicate data for robustness, TCP’s sliding window protocol.
updates the TCP sequence numbers. this yields a fast, flexible store. In The key idea is very NAT-like: when
The effect is to connect the new con- fact, accessing remote memory in this a restarting BGPD’s shim wrapper tries
nection to an existing, active, TCP manner is approximately two orders to connect to a peer, TCPR intercepts
connection that is open at a peer, in a of magnitude cheaper than file I/O the three-way handshake so that the
manner that won’t lose any data and to a standard local disk, and many remote peer won’t see a connection
imposes just milliseconds of delay. orders of magnitude faster than reset. Instead, it computes the “delta”
We’ve focused primarily on the remote file I/O. To support check- between the randomly chosen initial
shim; let’s next look at our approach’s points and complex object stor- sequence number for the new connec-
other components in more detail. age, FTSS extends the usual DHT tion and the sequence numbering used
SEPTEMBER/OCTOBER 2011 75
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
View from the Cloud
in the old connection. As packets are fall within the window of normal Our effort was supported by Cisco and is a
sent back and forth, TCPR adds or sub- asynchrony between BGP peers in part of the NEBULA project within the US
tracts the delta, depending on which the core Internet. Overall, the abi- National Science Foundation’s Future Internet
way the packets are going. Thus the lity to fail and recover transparently, Architectures (FIA) program (see http://r3.cis.
new connection end point finds itself coupled with the ability to test new upenn.edu/paperspdfs/R3_WP_Full.pdf).
______________________
talking to the old remote end point. versions and configurations of rout-
TCPR handles the TCP options used in ing software in production without References
routing protocols such as BGP, includ- risk, eliminates many of what used 1. C. Labovitz, G.R. Malan, and F. Jahanian,
ing the MD5 signatures. In our experi- to be the biggest causes of downtime. “Internet Routing Instability,” IEEE/ACM
ments, TCPR splicing takes as little as Trans. Networking, vol. 6, no. 5, 1998,
350 microseconds, and having TCPR oday’s cloud computing systems
on the path has a negligible impact on
TCP connection performance.
T are appealing for their low cost
of ownership, amazing scalability,
pp. 515–526.
2. E. Keller, J. Rexford, and J. van der
Mer we, “Seamless BGP Migration with
TCPR and the shim cooperate and flexibility. The cloud even brings Router Grafting,” Proc. Networked Sys-
in several ways. First, TCPR delays environmental benefits: users share tems Design and Implementation (NSDI 10),
outgoing acknowledgments until the computing resources, which are used Usenix Assoc., 2010, pp. 16–30.
shim confirms that it’s backed up the more efficiently, and the data centers
associated incoming data; this ensures are typically located near power- Andrei Agapi is a PhD student at Vrije Univer-
that, after a crash, the new BGPD generating sources: by using the net- siteit, Amsterdam, and a software engi-
won’t see any gaps or duplicated bytes work to move data to a data center, the neer with Cisco Systems. Contact him at
in the incoming data stream. Simi- need to move electricity to widely scat- aagapi@few.vu.nl.
___________
larly, the shim backs up any outgo- tered computing devices is reduced.
ing data so that, after a node crash, the However, for many applications, net- Ken Birman is the N. Rama Rao Professor of
recovered shim/BGPD pair can finish work routing instabilities make the Computer Science at Cornell University.
transmitting any data that was being cloud less reliable than it needs to be. Contact him at ken@cs.cornell.edu.
___________
sent at the time of the crash. Finally, Our work tackles a root cause for
the shim backs up parts of the TCPR this problem, and by dramatically Robert M. Broberg leads the Reliable Router
state, enabling TCPR itself to recover improving router availability, offers Research Effort and is a Distinguished
if a node running it crashes and the a path toward better stability in the Engineer at Cisco Systems. Contact him
TCPR daemon must restart. Internet as a whole. The technique at rbroberg@cisco.com.
____________
is incrementally deployable (mean-
Solution Performance ing that it can be rolled out without Chase Cotton is a senior scientist with the
As this article was going to press, we change to routers that run existing University of Delaware. Contact him at
were just finishing our port of the full protocols) and brings immediate ben- ccotton@udel.edu.
__________
fault-tolerant BGP implementation to efit to any path that traverses even
an actual CRS-1 router and hadn’t yet just a few routers using our approach. Thilo Kielmann is an associate professor at
measured recovery times or the corres- Wit h enough router s using t he Vrije Universiteit, Amsterdam. Contact
ponding router-availability levels in method, we could imagine that VoIP him at kielmann@cs.vu.nl.
___________
a true Internet deployment. However, telephony could achieve the same
we do have a full implementation (or even better) quality of service seen Martin Millnert is writing his master’s thesis
running on a testbed, and were able in wired telephone networks, and at Cisco Systems. Contact him at martin@
_____
to experiment with it using realis- that other kinds of streaming media millnert.se.
______
tic BGP routing tables and update applications could be deployed with
traffic. The results are encourag- sharply improved quality guarantees Rick Payne is a software engineer at Cisco Sys-
ing: complete recovery finished in relative to what’s feasible today. tems. Contact him at rpayne@cisco.com.
___________
as little as 30 ms for a BGPD that had
no routes to recover (for instance, Acknowledgments Robert Surton is a PhD student at Cornell Univer-
one with an empty routing table) and We are deeply grateful to professors Jonathan sity. Contact him at burgess@cs.cornell.edu.
_____________
405 ms for a BGPD with a large rout- Smith (University of Pennsylvania) and Doug
ing table containing 157,975 entries. Comer (Purdue University) for helping us iden- Robbert van Renesse is a principal research
These numbers were essentially tify this research topic, and for their encourage- scientist with the Department of Com-
unchanged when we tested with BGP ment and advice at many stages. We also thank puter Science at Cornell University. Con-
updates arriving every 130 ms, and John Denisco for his invaluable assistance. tact him at rvr@cs.cornell.edu.
___________
76 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Take the
CS Library
wherever
you go!
www.computer.org/csdl/epub_info.html
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Standards
Editor: Barry Leiba ___________
Inside the Identity
Management Game
Techniques for managing authentication and authorization are critical to the
next round of Internet innovation. Cloud-based services, the social Web, and
rapidly expanding mobile platforms will depend on identity management to
provide a seamless user experience. Although a number of standards have been
advanced, an Internet scale identity solution remains elusive.
T
here’s an old saying in American baseball – of OpenID and OAuth (the Web Authorization
“You can’t tell the players without a pro- Protocol) solutions has major advantages for
gram,” which seems particularly relevant connection-driven RESTful API developers and
to the current state of online identity manage- is being widely deployed. Large service providers
ment. The combination of a protracted develop- such as Microsoft, Facebook, Google, Yahoo, and
ment cycle, shifts in technologies and use cases, PayPal all contribute to development efforts.
and legal requirements for both privacy and Many of the major standards organizations
security have all led to the creation of a vital but are represented in the identity ecosystem, but a
somewhat fractured landscape. number of key specifications come from smaller
Early authentication schemes relied on creat- efforts with open source roots. Identity manage-
ing site-specific user accounts with their corre- ment has also drawn the attention of govern-
sponding user names and passwords. The World ments, policy makers, and advocacy groups, as
Wide Web and its proliferation of sites and services well as industry consortia, all of which bring
has resulted in a site-by-site account management their own expectations and requirements to the
pattern that’s been a strain for users and service table. This diverse set of players has led to a
providers alike. Increasing use of the Internet as a proliferation of organizations, each with its own
way to share and manage protected resources has set of participants, preferred development tools,
also brought an additional burden for verification and proposed solutions.
and authorization. The past 10 years have seen
several developments in both the authentication The Identity Ecosystem
and authorization arenas. The primary goal has A good place to begin to get the identity man-
been a Web-based, scalable solution that com- agement big picture is with the ITU Study Group
bines the ease of single-sign-on (SSO) with 17 (the lead study group on identity management)
authorization based on an exchange of identity- and the ISO/IEC Joint Technical Committee 1/SC
related assertions across security domains. 27 Working Group 5 (identity management and
A number of problems must be solved before privacy technologies). Both these groups have
we’ll see a robust, full-featured, Internet-scale taken on defining frameworks for identity man-
identity management system in place, but progress agement and collecting and harmonizing common
has been made on authentication/authorization terms used in developing identity- and privacy-
solutions for the Web. Two in particular are gaining related standards. A quick review of current work
broad acceptance. The relatively mature SAML/ programs also provides a useful catalog of open
SOAP paradigm and SAML-based federations have design issues as well as the large number of outside
traction in enterprise, educational networks, and activities they’re tracking. The definitions docu-
e-government. The rapidly advancing combination ments are both freely available and recommended
78 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Inside the Identity Management Game
Editor’s Introduction
A s we’ve taken to using the Web for more and more interre-
lated things, it’s become important to identify ourselves to
many different organizations – “domains” in Internet terms – and
Identity management has become a significant issue on the
Internet, and there are many organizations working on the
problem. In this issue, Lucy Lynch of the Internet Society gives
to want those identities to work, in some fashion, between domains. us an overview of the landscape. Next time, we’ll take an in-
Perhaps we want to share information between our Facebook and depth look at the US Government’s approach to identity man-
Flickr accounts or would like to have one “wish list” on several agement, in NIST’s National Strategy for Trusted Identities in
shopping sites. Perhaps we just don’t want to have to remember Cyberspace (NSTIC).
myriad sign-on identifiers. We need “identity management.”
reading, as most new identity efforts established by members such as https://www.
any member state (see _________
begin with (yet another) attempt to Microsoft, Equifax, Google, Novell, eid-stork.eu/index.php?=61).
___________________
find a common vocabulary. Oracle, and PayPal to advance the Another recent example is the pro-
OASIS, the W3C, and the IETF all user-centric, wallet-like metaphor IMI posed National Strategy for Trusted
provide standards that underlie cur- offers. In 2011, Microsoft declared Identities in Cyberspace (NSTIC) being
rent identity management designs, its own Information Card imple- driven by the US government to seek a
and we can combine these building mentation “feature complete” and partnership with private enterprise to
blocks in multiple configurations. announced that it won’t be shipping manage authenticated citizen engage-
OASIS supplies SAML and the Web CardSpace 2.0. A lthough the ICF ment with government sites. Three
services (WS-*) suite of standards, is still intact, most partners are cur- organizations have already stepped
as well as the Identity Metasystem rently more focused on OpenID/ up to provide trust framework ser-
Interoperability (IMI) specification OAuth implementations. v ices t hat meet N IST SP 800-63
used for Information Cards. The W3C’s Any overview of the identity eco- requirements for levels of assurance
HTTP architecture, URIs, and the system wouldn’t be complete without in some fashion: The Open Identity
service-related SOAP are leveraged some consideration of its implement- Exchange (OIX) will provide list-
by federated and distributed identity ers and adopters. There is a mailing ing services and support the devel-
solutions. The IETF provides several list, a code repository, or an event to opment of additional frameworks.
relevant standards, including HTTP, match nearly every interest. Com- The Kantara Initiative will serve as
the Simple Authentication and Secu- munities range f rom t he loosely a special assessor and will leverage
rity Layer (SASL), Transport Layer aligned Identit y Commons to the its existing certification programs to
Security (TLS), and Public-Key Infra- more formal European-Commission- provider auditors and interoperabil-
structure (PKIX) along with numer- funded Stork project. The former ity testing. InCommon, an Internet2
ous active efforts including OAuth, houses a few working groups but is consortium of inter-federated edu-
Abfab (Application Bridging for Feder- best known for hosting the semi- cational institutions, will provide
ated Access Beyond the Web), and the annual Internet Identity Workshop an interface to research and educa-
recently proposed Web Object Encryp- (IIW), which has focused on user- tion with strong levels of assurance
tion and Signing (WOES) standard. centric identity. Meetings feature a based on its own internal controls.
The more loosely organized open self-organizing structure that lends
source community has also contrib- itself to brainstorming and advanc- Advancing SAML Federations
uted some key specifications, and sev- ing small specifications. A recent After the initial SAML 1.0 standard
eral new organizations have formed such specification is Simple Cloud was published, two complementary
to house and drive these efforts. The Identity Management (SCIM), which projects adapted the protocol and its
OpenID Foundation (OIDF) is a non- used the spring 2011 IIW meeting associated capabilities to address their
profit that hosts numerous active to solidify interest in work on stan- own use cases. The Liberty Alliance
working groups, publishes specifica- dardizing common API-based solu- was formed by a consortium of major
tions, and manages the open-code tions already in the market. The Stork software vendors and focused on
repository. Oauth.net is an even less
_______ project aims at implementing an EU- federated cases for large enterprises,
formal effort including an active set wide interoperable system for rec- including governments. Meanwhile,
of implementers organized around ognition of eID and authentication the Internet2-based Shibboleth project
the original OAuth 1.0 specification. that will enable businesses, citizens, focused on higher education needs.
The Information Card Foundation and government employees to use These early adopters of federated
(ICF) was an industry consortium their national electronic identities in identity solutions were supporting
SEPTEMBER/OCTOBER 2011 79
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Standards
organizations and enterprises with While early adopters were developing process and is widely used in current
large user bases, signif icant pro- standards, building tools, and extend- start-up efforts.
tected resources, complex authoriza- ing use cases, the world around them
tion patterns, and data and services changed. OpenID for
spread across multiple domains. As Social applications turned the Lightweight Identity
early adopter development efforts “authenticate, then authorize” model As Web 2.0 users looked for ways to
progressed, OASIS continued work inside out as users rushed to connect. collaborate with others across mul-
on SAML 1.1 and ultimately SAML Mobile phones and other Internet- tiple sites and services, the need for
2.0, adding features such as attri- enabled devices began to efficiently a simple, persistent way to identify
bute profiles, metadata capabilities, use native applications. The new oneself became a compelling issue.
and the use of pseudonyms. OASIS generation of innovators viewed the Some users wanted t he abilit y to
has also advanced the WS-* suite Internet from inside the Web and represent themselves with a single
of specifications, which addresses brought a new set of languages and identifier, whether publishing a
several identity-related concerns. The tools to bear on development. While set of photos or posting comments
Liberty Alliance work moved toward still focused on Web services, coders on a friend’s blog. The process of
formalizing the requirements for “cir- looked to JavaScript Object Notation adding a new account for ever y
cles of trust,” with proposed frame- (JSON) and REST to build their APIs. site was cumbersome and often
works for testing interoperability The features that had made SOAP disappointing, as individuals often
and compliance with US National attractive to SAML users were viewed found their preferred user name
Institute of Standards and Technol- as too rigid and too difficult to man- taken and grew frustrated with
ogy (NIST) levels of assurance. With age in the fast-paced Web 2.0 world. managing multiple accounts and the
the publication of its Assurance The increasing use of Web-based related passwords. Security concerns
Framework, the Liberty board took APIs favored the REST model, which also grew as users recycled pass-
a decision to wind up the Alliance can bypass SOAP, SAML, and the words among sites, with little regard
and contribute all Liberty assets to Web Services Description Language to the relative values of their bank-
the Kantara Initiative. The Shibbo- (WSDL) in favor of a simple exchange ing-related account versus their blog
leth project continues to develop the of well-defined, consistent HT TP accounts. Lightweight SSO became a
Shibboleth federation software and messages between client and server. goal. The social identifier was con-
the OpenSAML libraries. The reuse of existing HTTP architec- ceived as something unique but that
SA M L federations are deeply ture features allows for immediate wouldn’t require a high degree of
embedded in education, govern- interoperability. On the other hand, proofing.
ment, and corporate intranets and SOAP-based exchanges enabled cus- The proposed solution was to let
have been customized to address the tomized message vocabularies that users create and asser t an iden-
security concerns of verticals such weren’t guaranteed to interoper- tity that would be widely accepted,
as healthcare and banking. All this ate. SOAP provided for methods for thereby letting them use a single pass-
activity has produced a mature but back-channel exchanges that included word and present a unified persona
complex set of standards that have both security- and privacy-enhancing online. The pattern would require
evolved to meet primary stakehold- features and permitted use with pro- coordination among three parties:
ers’ needs. Sophisticated problems tocols other than HTTP, but came the end user (data subject), the ser-
related to inter-federation, discov- with heavy ties to XML. Although vice provider (relying party, or RP),
ery, user privacy, data minimization, XML can also be used in the REST and a designated identity provider
informed consent, and service provi- model, the trend has been for a more (IdP). Web developers, and blog soft-
sion “below the Web” are active topics stripped-down approach. JSON, based ware implementors in par ticular,
for developers. Issues and solutions on a subset of JavaScript, is purpose- introduced several models for decen-
are driven by the federation opera- built for data exchange and bills tralized authentication, and then
tors, and users and service providers itself as the “fat-free alternative to these efforts were merged into the
are sometimes seen as problems to be XML.” The social Web environment OpenID 1.0 specification in 2005. In
managed, rather than as full partici- is driven by a rapid development the OpenID scenario, a user creates
pants in the identity exchange. cycle and a need to enable seam- an account with the IdP of his or her
less exchanges among multiple end choice and can then use an agent —
SOAP vs. REST, XML vs. JSON points to deliver a coherent experi- usually browser-based software — to
Why haven’t SAML federations solved ence for users. The combination of negotiate authentication. If the IdP
the identity management problem? REST and JSON has enabled that doesn’t recognize the asserted OpenID,
80 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Inside the Identity Management Game
or if the user refuses the request from along with some proposed solutions. and struggled to create a simple set
the RP, authentication fails. The 2.0 release was also supported of workf lows that provided good
While OpenID sought to solve the by the completion of patent-related user experiences in diverse environ-
SSO problem for users, the three- nonassertion agreements from all ments. As an illustration of the loose
party authentication dance brought key contributors to earlier OpenID organization around the work, one
new issues. Two in particular are specifications. specification author, Eran Hammer-
worth mentioning, both related to Lahav, leveraged his blog to detail
RP adoption. The first is the so- OAuth for User these issues and chronicle how
called “NASCAR problem” (referenc- Authorized Delegation OAuth and related delegation mecha-
ing the proliferation of sponsor logos With decentralized authentication nisms were deployed. In 2008, the
plastering race cars), which arises well under way, attention turned to OAuth document editors introduced
because users must pick an OpenID the problem of authorization in the their work to the IETF, and Oauth 1.0
from among the many available Web 2.0 context. The original OAuth has since been published as RFC 5849
options. Although RPs can provide specification (from 2006) aimed to (http://tools.ietf.org/html/rfc5849).
a generic text-entry box for OpenID complement OpenID and let users The IETF then chartered a working
entry, this proved to be confusing, delegate access to an API acting on group to look at formal standard-
and sites quickly began displaying the user’s behalf to share a protected ization of the OAuth 1.1 protocol.
logo buttons of the most popular resource with the data requester. The Although a few original participants
OpenID providers, such as Facebook, metaphor often used to describe this continued to work on the IETF vari-
Google, and Yahoo. This simplified functionality is the “valet key” you ant of OAuth, work also continued in
the user experience and helped drive would hand to a parking lot atten- the deployment community with little
adoption among a few IdPs. But as dant. Such a key will only let the attention being paid to the IETF effort.
new providers entered the identity valet drive the car within a limited By 2009, several OAuth imple-
market, the number and placement range and might block access to the mentations existed in the wild, and
of logos became problematic. on-board radio or phone. the or iginal core specif ication’s
The second issue is particular The concept is simple: users limitations were beginning to cause
to those RPs that aren’t also IdPs. authorize limited access to resources fractures in the development com-
By agreeing to accept authentication (photos uploaded to a website) to munity. In April of 2009, a major
from the large external IdPs, the RP another service provider, who then securit y v ulnerabilit y based on
loses some control over its relation- might print the photos or release them session attacks shook the commu-
ship with any given user and his to a blog writer for reuse. The access nity, and a competing proposal —
or her associated identifying data. grant is accomplished through the OAuth Wrap, introduced at the IIW
Although this might be appealing to exchange of a shared secret between meeting in November — fur ther
users, it doesn’t provide much incen- users and the first-party service, divided efforts. In April 2010, vari-
tive to service providers. Meanwhile, which then grants access to the third ous authors introduced a new draft
the large providers can leverage party via a token. The token need not proposal for OAuth 2.0, incorporat-
OpenID to extend existing relation- reveal either users’ identifying infor- ing features from OAuth Wrap. This
ships and manage internal delegation mation or their long-lived authentica- work is advancing in the IETF, and
among their own service offerings. tion credentials, and doesn’t give the additional drafts have been submit-
The OpenID 2.0 specification was third party service full access to users’ ted to deal with security consider-
published in late 2007. It added first-party accounts. In OAuth terms, ations and token usage. The various
functionality, including a format for the third party is the consumer (that is, documents are headed for working
extensions to allow for attribute the consumer of the token). group approval, but some outstand-
exchange, and also added several The exchange of tokens and the ing issues must still be closed out.
new identifier types, such as the desire to protect users’ identity and Meanwhile, OAuth implementa-
OASIS-sponsored Extensible Resource resources brought an increased tion and deployment continues to
Identifier (XRI) as well as a special need for security and the inclusion grow, and issues with interoperablity
identifier for Open ID providers (OP). of cryptographic requirements. The arise, depending on which draft is
The new identifiers were intended deployment scenarios also covered used for guidance.
to aid in discovery. The specifica- Web-based applications, desktop cli-
tion also included a security consid- ents, and mobile applications. Early OpenID Connect
erations section that outlined some implementers found the cr yptog- The last OpenID specification (2.0)
risks associated with using OpenID, raphy elements difficult to manage was published in 2007, is seriously
SEPTEMBER/OCTOBER 2011 81
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Standards
Scorecard in Identity Management Standardization
T he following organizations are working on standards
related to identity management:
Open Identity Exchange (OIX; http://openidentityexchange.
org)
__
Kantara Initiative (http://kantarainitiative.org) — proceeded
The OpenID Foundation (OIDF; ht tp://openid.net /
by the Liberty Alliance
foundation/)
_______ — OpenID 1.0/2, OpenID Connect
InCommon Federation (w w w.incommon.org /about .
OAuth (community site; http://oauth.net) — OAuth 1.0/
html)
___
OAuth Wrap
US National Institute of Standards and Technology (NIST;
Internet Engineering Task Force (IETF; www.ietf.org) —
www.nist.gov)
OAuth/WOES/Abfab/HTTP
Identity Commons (www.idcommons.net)
World Wide Web Consortium (W3C; www.w3.org) —
Information Card Foundation (ICF; http://informationcard.
HTML/HTTP/SOAP
net/foundation)
Organization for the Advancement of Structured Informa-
International Telecommunications Union (www.itu.int/
tion Standards (OASIS; www.oasis-open.org) — SAML/
ITU-T/studygroups/com17/index.asp)
_________________________ — ITU-T Study
XML/WS-*/XRI
Group 17
Shibboleth Project (www.shibboleth.net) — Shibboleth/
www.
International Organization for Standardization (ISO; ____
OpenSaml 1.0/2.0
iso.org)
____
US National Strategy for Trusted Identities in Cyberspace
(NSTIC; www.nist.gov/nstic)
outdated, and no longer ref lects We’ll continue to engage developers and using multiple devices to access mul-
either the current state of deploy- potential deployers about OpenID Con- tiple accounts and services. We’re
ment or potential new use cases — nect at upcoming OpenID Summits ... to also encouraged to store protected
some of which will require a higher better understand, critique, refine, test, resources in various locations across
level of assurance than we can and ready OpenID Connect for prime the Internet. Just managing our own
obtain with just a self-asserted iden- time. (See http://openid.net/2011/05/20/ personal contacts and calendars can
tifier. In addition, the use of OAuth, openids-second-act-openid-connect.)
_______________________ be a challenge. When we add the
which wasn’t considered in the 2.0 desire to share some of our informa-
case, has become common. Whereas Still very much a work in prog- tion with others while continuing to
XRI is now moribund, the Exten- ress, OpenID Connect is intended to protect our most sensitive data, the
sible Resource Descriptor (XRD) is work with OAuth 2.0 and JSON-based issues become even more complex.
nearing completion and might now token formats for encryption and Identit y management imple-
be preferred for discovery in some signing to create a social Web iden- mentations have come a long way,
cases. OpenID also needs to work tity stack, and will impose some new but greater coordination among the
both with native applications and in requirements, such as using Secure current players is necessar y. The
mobile devices, features not explic- Sockets Layer (SSL) to help address dominant models each bring useful
itly addressed in the current version. ongoing security concerns. With properties to the table, but conver-
T he OI DF cont inues to house the addition of attribute exchanges gence has been slow. As new part-
development efforts and has moved and artifact bindings, this proposed nerships such as the NSTIC advance,
toward a more formal structure with stack begins to resemble older SAML and ser vices like mobile Internet
dedicated working groups, each with federations — built with R EST- access and cloud computing gain
a charter and a mutually agreed- ful APIs and using JSON instead of traction, integration will become
on intellectual property regime. In XML. even more important. It’s time to
2010, two of those working groups, The Connect work depends on play ball!
OpenID Artifact Binding and OpenID both the IETF’s successful standard-
Connect, combined their efforts to ization of OAuth 2.0 and the out- Lucy Lynch champions the Trust and Iden-
add ress ex tended use cases and puts of the recently proposed WOES tity Initiatives for the Internet Society.
account for the use of OpenID in working group, also in the IETF. Her interests include the development
conjunction with OAuth. Initially and deployment of Internet-scale trust-
labeled OpenID ABC, this work is enabling technologies and policies. Lynch
now titled OpenID Connect. In a he need for identity management
recent announcement, OIDF executive
director Don Thibeau says, in part,
T will only continue to grow. As
individuals, we now find ourselves
has an MS in mass communications from
the University of Oregon. Contact her at
lynch@isoc.org.
_________
82 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Beyond Wires
Editor: Cecilia Mascolo ________________
When the Shift Hits
the (Television) Fan
The television ecosystem faces growing attention and audience fragmentation
thanks to an explosion of content sources and content consumption devices.
In this article, the author looks at some of the interaction paradigms, viewer
behaviors, and innovative social and dual-screen experiences that seek to dis-
rupt or redefine the traditional models of audience measurement and content
distribution, in an effort to address these challenges.
I
can clearly recall the day my dad brought You were defined by what you watched, and woe
home our first color television. It was a Sony betide the teenager who wasn’t up on the latest
Trinitron (yes, this was in the eighties), and soap opera plot twists from Santa Barbara or was
it held pride of place in our living room for quite unfamiliar with the newest music videos top-
a few years. We lived in Africa at the time, and ping the UK charts. Most families still operated
with just a single national broadcast channel, with a single television in their living rooms, but
our viewing options were limited to just under homes became a battleground for control of the
12 hours of televised content a day, combining remote. These were the days before DVRs took
American and British fare with news, weather, over — so live events like cricket matches took
and local programming. We kids were allowed precedence over taped shows, leaving the losers
to watch up to an hour of television a day — in these battles praying devoutly for reruns.
after the homework was all done, of course. But Looking back, those days seem remarkably
it was definitely family time. Mom would make simple and uncomplicated. We had choices, but
tea and biscuits, dad would put away his work the regulated programming schedule allowed
papers, and we’d all curl up on the couch to for some degree of social activity around the
watch a documentary on Norman Carr and his television. Today, I stare in wonderment at my
fascinating life studying lions in the wild. toddler as he rapidly zips his fingers around the
Fast forward a decade, and we were in India password lock on my tablet, flicks his way to
just as satellite television began its unstoppable the screen with the blazing red Netflix icon, and
march into the average Indian household. Sud- with another click or two settles in to watch
denly it was no longer a question of what to the misadventures of The Cat in the Hat. And
watch but what not to watch. Specialized sports, that’s just the start. He knows to turn on the
movie, and music channels abounded. Even the big-screen television and press “DVR” on the
national broadcasting agency got in on the act, remote to see the listing of recorded shows. He
spawning multiple channels and creating more knows to switch HDMI inputs to bring the PS3
diverse schedules tailored to national, regional, online so he can watch Curious George’s latest
and local needs. Television went from being just shenanigans on DVD. Broadcast television poses
enter tainment to becoming a status symbol. the least attraction to him because it doesn’t
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 83
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Beyond Wires
guarantee his being able to see con- service like Netflix, Internet live- back on our tablet-based cable pro-
tent that he likes when he likes it. streamed events (such as the recent vider application to see real-time
Our propensity to turn on the tele- French Open) accessible to viewers streams of different channels when
vision only to watch real-time con- on any capable networked device, conflicts in schedule occur. We get
tent (news or sports) has done little and, last but not least, Internet por- our movies through Netflix (and yes,
to diminish his conviction that all tals (for example, Hulu) that enable it’s been a long time since I saw the
the “f un” content lives in the on- on-demand viewing of broadcast inside of a movie theater) and catch
demand space. While this isn’t rep- content for some provider-approved up on missed content episodes via
resentative of the larger population, time window af ter t he or iginal Hulu when convenient. Such viewer
it’s definitely a growing trend today broadcast. behaviors are further compounded
as people lead busier lives and carry Concurrently, the television-as- by concerns about attention frag-
more gadgets that support rich con- device role is being redefined by the mentation, with research showing
tent consumption. emergence of big-screen televisions an increasing trend in multitasking
with multiple HDMI inputs, allowing around content consumption.
A Changing Ecosystem for a variety of multimedia-capable Although such behaviors have
In industr y, this trend has been devices to be interfaced seamlessly for social ramifications (notably, a decrease
broadly categorized as X-shifting. true lean-back viewing and interac- in social interactivity from family-
For instance, time-shifting trends tion. Where once the television served time viewing), they also have huge
(such as personal video recorders) primarily as a conduit for broadcast implications to content providers and
let users consume live or broadcast content, today it serves as a rich and advertisers. The television is known to
content at their convenience instead interactive display for cable content, have a high impact on consumer pur-
of at the scheduled hour. Place- gaming consoles, set-top boxes, media chase behaviors, not just in increased
shifting solutions (such as from centers, and even basic PCs. Market brand recall and awareness but also
Sling Media; www.slingmedia.com/ research shows that increasing num- in inf luence wielded by celebrity
go/placeshifting)
_____________ let users access bers of viewers have connected their endorsements and product placement
and consume content in their homes PCs to their television sets to view in popular shows. Audience fragmen-
from anywhere. Motorola’s Follow Internet content on the big screen or tation makes it harder to track viewer
Me TV solution (see http://tinyurl. view (and share) stored multimedia preferences and intent, and target
com/3buhj45) takes it to the next level, content (photos, videos, and so on) them for relevant merchandise or
enabling seamless device-shifting with a larger audience in the home. content recommendations.
during content consumption — letting Second, we’re seeing a significant In this context, two types of
a user pause his or her content on shift in user behavior around con- viewer populations are of most con-
one device in the home and resume tent consumption, driven largely by cern: the cord-cutters and the cord-
it on a second device in a different the fragmentation in the content and nevers. Cord-cutters refer to viewers
room, without missing a beat. While device ecosystem. Whereas tradi- who subscribed to cable services but
all these shifts have catered increas- tional viewing behaviors were cen- subsequently “cut the cord,” rely-
ingly to user convenience in content tered around shared or social “family ing almost exclusively on Internet
consumption, they’ve also exposed time,” current behaviors skew toward content, on-demand streaming ser-
two key shifts in the television eco- individualized viewing tailored to vices (like Netflix), and free over-
system itself. personal interests and convenience. the-air broadcast programming in
First, we’re seeing an increas- In lieu of a single television set in the an attempt to cut costs. On the other
ing separation in the roles of televi- family room, the average American hand, cord-nevers (a term defined by
sion as content and as device. A few home boasts multiple sets (in the the tech media) refer to a new gen-
decades ago, these roles were insepa- kitchen, bedrooms, and study) along eration of viewers who’ve avoided
rable. Watching television implied with numerous other multimedia- cable subscriptions from the start.
consuming live broadcast content on capable portable devices such as lap- While cord-cutters represent some
the television set in the living room. tops, smart phones, and tablets. loss in revenue to cable operators,
Today, television-as-content can be I can see evidence of this in my cord-nevers are completely invisible
consumed from numerous sources own home. We watch news and live to them and thus pose a challenge to
in diverse ways: broadcast content events in the family room together. future growth. As a result, there’s
supplied by cable operators and sat- The DVR is dominated by child- increased interest in and excitement
ellite providers, DVDs purchased, friendly fare for on-demand view- about new technologies that can
rented, or streamed from a third-party ing. My spouse and I tend to fall connect cable and content providers
84 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
When the Shift Hits the (Television) Fan
to new and existing audiences in a user engagement w it h t he f ir st a manual process, with the viewer
manner that supports these chang- screen. On the other hand, inter- being asked to select his or her cur-
ing behaviors. active applications on the second rently watched program from an
screen generate not just richer and onscreen list or guide shown on the
Rise of the Mobile Devices more personalized analytics (on an companion device. This takes effort
Enter “companion devices” for tele- individual rather than household and distracts from the viewing expe-
vision viewing. Previous market basis) but provide additional real rience. Time-zone conflicts coupled
research shows that many TV view- estate and targets for continued with local or regional scheduling
ers multitask during content con- engagement with the user before, changes create additional complex-
sumption, often watching television after, and during content consump- ity in any one-size-fits-all solution.
on one screen (primary) but doing tion. The catalyst in the change has This has created new opportunities
interactive tasks on a second screen been the emergence and popularity for audio and video fingerprinting
(mobile or PC). This “dual-screen” of tablet devices like Apple’s iPad solutions (such as Yahoo’s IntoNow
behavior has two kinds of impact. (iOS) and Motorola’s Xoom (Android) produc t), wh ich detec t conte x t
First, it creates a divided attention that provide not just larger real through simple media capture and
model in which users aren’t always estate for complementary informa- analysis on the second device.
engaged with the onscreen content tion or interactions, but also have
(or ads), translating to lower viewer- improved hardware capabilities to Search and Advertising
ship numbers or missed revenue from support rich video playback directly Television is a hugely visual medium
related advertising or content sales. on the device. Cable providers have with rich audiovisual context for
Second, it allows for third-party ser- since released numerous tablet- most user queries. Given deficien-
vices to be presented to viewers for and smart-phone-targeted applica- cies in integrated information search
richer interactions related to the first tions that range in capability from capabilities within existing televi-
screen content. Examples of such ser- synchronized remote controls (for sion and cable boxes, an opportunity
vices range from simple search (“Who example, the Comcast Xfinity app) exists for developing complementary
made the dress worn by character X that let users navigate and control search and response solutions lever-
in show Y?”) to chatter (for example, their set-top box from a compan- aging the rich input and context-
Twitter, Facebook) and social TV ion device, to live streamed content capture capabilities on mobiles. Some
applications (such as GetGlue and (such as the Cablevision Optimum of our work at Motorola Mobility
Miso) where viewers “check in” to app), where viewers can essentially (for instance, TV Answers) has explored
meet other fans and earn badges or get any subscribed channel streamed value propositions such as crowd-
exclusive content access. Data from live to their tablets within the con- sourcing within this domain, using
these services show surges in activ- fines of their home wireless network. the large population of television
ity around popular television shows So, where do we go from here? We viewers as a “social sensor” for both
or live events (for instance, http://
____ will see a veritable flood of Android- creating and curating responses.
blog.twitter.com/2011/02/superbowl. and iOS-based tablet and smart phone Potential also exists for new dual-
html), with the potential for richer
___ devices entering the market this screen advertising strategies that can
measurement and analy tics (see year, along with the software devel- leverage companion devices as inter-
http://blog.getglue.com/?p=7736). opment kits, testing harnesses, and active advertisement real estate or
From a Beyond Wires perspec- application market support required as tools for determining ad cam-
tive, the ability to create loosely syn- to create, deploy, and sell interac- paigns’ effectiveness around tele-
chronized application experiences tive applications. The “companion vised content.
concurrently across television and device” applications market is nascent
mobile domains is a game changer, but rapidly growing to accommodate Gamification
effectively opening up the otherwise the various needs and challenges the Gamification refers to the inclusion
closed television ecosystem to dis- social and interactive television eco- of game mechanics in non-game
ruptive and innovative new applica- system faces today. contexts to make an experience
tions and enablers. Some key opportunities lie along more fun or engaging for users.
The television and cable industry the following paths. Although the dominant use case for
has been quick to see the challenge gamification has been loyalty pro-
and the opportunity in dual-screen Context Synchronization grams (as evidenced by the success
behaviors. On one hand, divided Today, correlating the mobile experi- of Foursquare [https://foursquare.
attention (across two screens) dilutes ence to television content is mostly com]), there’s significant interest
SEPTEMBER/OCTOBER 2011 85
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Beyond Wires
and utility to extending the concept repositories of content comes chal- Acknowledgments
to television. Applications such as lenges in content discovery. Social Thanks to all my colleagues and peers within
Miso and GetGlue have translated interactions and smart recommend- the Applied Research Center, and to the many
the Foursquare paradigm to social ers will become increasingly rele- folks on Twitter who have engaged me in
television, awarding viewers with vant as users try to fi nd new ways interesting discussions on this topic. I hope
badges or exclusive content access to connect and consume content. The I did you all justice.
in exchange for loyalty to programs. key challenge in social television
Content portals like USA Networks’ applications is to balance the degree Nitya Narasimhan is a distinguished member
characterarcade.com have applied of peer interactivity (lean-forward) of technical staff within the advanced
gamification to pre- and post-content with the degree of attention to con- concepts group in the Applied Research
viewing experiences, using immer- tent (lean-back) required by users in Center at Motorola Mobility. Her cur-
sive games and contests to get view- different contexts. rent interests span scalable Web ser-
ers more invested in a show or its vices, interactive mobile and television
characters. Plenty of opportunity applications, crowdsourcing and social
remains for exploration and inno- he take-away message for us as
vation in this space, starting from
game elements for user interaction
T researchers developers, and tele-
vision viewers is this: Television
search, and context-aware frameworks.
Narasimhan has a PhD in computer engi-
neering from the University of California,
to creating games with a purpose was, is, and will continue to be the Santa Barbara. She’s been a researcher,
that tap into crowdsourced television dominant source of entertainment developer, and innovator in industry for
behaviors to generate useful data or for most users. But, it should no lon- nearly a decade. Contact her at nitya@
____
content. ger be viewed as an isolated device motorola.com.
or a closed ecosystem. The commu-
Social and Serendipitous nity exists. The tools and enablers
Applications are being built. The early explorers Selected CS articles and columns
With fragmented viewing comes and adopters are out in force. Will http://
are also available for free at ____
isolation. And with increasingly large you join us? ComputingNow.computer.org.
Silver
Bullet
Security In-depth interviews
Podcast with security gurus.
Hosted by Gary McGraw.
www.computer.org/security/podcasts
*Also available at iTunes
Sponsored by
86 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
The Functional Web
Editor: Steve Vinoski _________
Scala Web Frameworks:
Looking Beyond Lift
S
cala is a hybrid object-oriented and func- Zenexity, has worked hard to create a developer-
tional programming language for the Java friendly experience.
Virtual Machine (JVM) that’s growing in Installing Play is easy. You download the zip
popularity. Two previous Functional Web columns file, expand it in a location of your choosing, and
presented the Lift framework, the best-known add the base directory to your environment’s PATH
Web framework written in Scala.1,2 In terms of its variable, so the play command is on your path.
prominence and full feature set, Lift is the Scala To install the Scala module, r un this
analog of the Ruby world’s Ruby on Rails. command:
But other frameworks exist in the Scala
world, just as alternatives exist to Rails in the play install scala
Ruby world. One size doesn’t fit all needs. A full
list of Scala frameworks is available at http://
____ Now you can create a Scala Web application in a
doi.ieeecomputer societ y.org/10.1109/ M IC.
__________________________________ directory of your choosing:
2011.104. Some are full-stack frameworks for
______
building multi-tier applications. Others are “point” play new SampleScalaApp --with scala
tools for specific parts of an application, like tem- play run
plate libraries for generating webpages (analog-
ous to Java Server Pages). Still others focus on The new application SampleScalaApp is now
building particular kinds of networked servers, in a directory of the same name. Play’s built-in
like REST response servers that are “headless.” Web server starts via the run command. By
Space considerations prevent us from dis- default, it listens for requests on port 9000. If
cussing all these tools. It’s hard to choose just a you go to http://localhost:9000 in your browser,
few representative examples, but here I focus on you’ll see the page shown in Figure 1, which
three: Play, a full-stack, commercially supported provides instructions for what to do next.
application framework; Scalatra, inspired by the The directory structure Play creates for an
lightweight, popular Sinatra framework; and application will be familiar to Rails programmers.
Finagle, a highly scalable, headless server library. Because Play (and Rails) are designed to grow
gracefully as applications become large, Play puts
Play code for different application responsibilities in
Play (www.playframework.org) is a Java-based separate files so file sizes remain manageable.
Web framework with a very capable module The SampleScalaApp/app directory has a
architecture that makes it straightforward to view subdirectory for views, which hold the
write plug-in modules. Scala support is imple- webpage templates, a models subdirectory for
mented as a module. It permits the use of Scala domain classes, and a controllers subdirec-
throughout the stack, including webpage tem- tory for the responders to user actions. However,
plates and the database query layer. because Scala code doesn’t require the direc-
A professional Web application developer tory structure to match the package structure,
accustomed to the polish and ease of use provided you can put the files for your controllers
by Rails will feel at home with Play. Its creator, and models in the app directory, if you prefer.
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 87
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
The Functional Web
Template('now -> new Date)
}
def list = {
new Template(
"contacts" -> Contact.find(
"order by name,
firstname ASC").
list())
}
...
}
The sidebar, “An Aside on Scala
Syntax” offers a brief explanation of
some Scala features used in this and
subsequent examples.
The list method instantiates a
new HTML page Template to format
the response. The latter is passed
GET / Application.index
key-value pairs, in which the keys
GET /contacts Application.list
are names of variables that will be
POST /contacts Application.create
referenced in the HTML template — in
POST /contacts/{id} Application.save
this case, a contacts variable. A
GET /contacts/{id} Application.form
find method on a singleton named
GET /contacts/new Application.form
Contact, which corresponds to a
POST /contacts/{id}/delete Application.delete
domain model object of the same
name, is called to query the database
# Map static resources in /app/public to the /public URL
for all the contacts, ordered by name.
GET / staticDir:public
The query result is converted to a
ZenContact/conf/routes file. Scala list. (At the Java byte-code level,
Contact.find will look exactly like a
static find method defined in a tra-
ditional Java class named Contact.)
The simple examples that come with value appears in this position in an Here is the Contact domain model
the Scala module do just that. incoming URL path. The id will be class defined in ZenContact/app/
Configuration of various proper- passed to the controller for use as a models.scala (again simplified for
ties, such as the database persistence database lookup key, for example. brevity):
settings, occurs in SampleScalaApp/ Using the routes from Figure 2, the
conf/application.conf. Routing URL URL http://localhost:9000/contacts package models
requests to the controllers that handle will get routed to the list method in /* imports ... */
them is defined in SampleScalaApp/ the Application singleton object,
conf/routes. which is defined in ZenContact/ case class Contact(
Let’s look at the ZenContact sam- app/controllers.scala, which looks id: Pk[Long],
ple application that comes with the like this (simplified slightly for @Required firstname:
Scala module to see examples of what brevity): String,
these various directories and files @Required name: String,
might contain. Figure 2 shows the package controllers @Required birthdate: Date,
routing table for ZenContact. It cov- /* imports ... */ @Email email: Option[String]
ers all the life-cycle steps required to )
view and manage a list of contacts. object Application extends
First, the expression {id} defines a Controller { object Contact extends
variable id that will be given whatever def index = { Magic[Contact]
88 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Scala Web Frameworks: Looking Beyond Lift
An Aside on Scala Syntax
F or readers unfamillar with Scala syntax, here are a few
pointers:
Compared to Java, Scala import statements use the “_”
A method definition begins with def. Types for return val-
ues are usually inferred, and parentheses are usually omit-
ted if there are no arguments. The method body begins
character instead of “*” as a wildcard. after the “=” sign.
Semicolons are inferred. Scala supports the syntax key -> value to pass key-value
The object keyword declares a singleton object. The run- pairs to maps and methods that want them.
time will only instantiate one instance. Scala uses objects to Pattern matching is like switch statements on steroids. In
hold methods and fields that would be declared static in pattern-matching expressions, each potential match begins
Java classes. with the case keyword, followed by a match expression
When the case keyword is used, it adds extra features to and the body to execute if the match succeeds. The match
a class, including a corresponding singleton object (called a expression and body are separated by “=>”.
) with the same name (used for factories, pattern You subclass with the extends keyword. Using the with
matching, and so on). keyword, you can implement pure interfaces or mix in addi-
The whole class body is the primary constructor, so the tional behaviors. Both pure Java-like interfaces and mix-ins
constructor argument list is passed after the class name. are defined using a feature called .
You can handle integration with exceptions used in JDBC. Anorm also method will ignore any rows that
Play’s Java-based object-relational embraces the view that SQL itself is don’t match one of the cases, effec-
mapping (ORM) layer using annota- the best domain-specific language tively implementing a filter.
tions (such as the @Required anno- for talking to your database, so you Play provides a rich, well-designed
tation on some of Contact’s fields) should embrace it and not try to hide framework for building multi-tier
and having the “companion” single- from it. Anorm makes it easy to con- Web applications that will feel
ton Contact extend a Magic class vert back and forth between Scala familiar to the Ruby on Rails devel-
that provides the find method, for collections and data from queries or oper moving to Scala. The Scala
example. data that’s used for updates. You can module adds powerful APIs that
So, what are the benefits of using parse results with pattern match- exploit Scala’s functional program-
Scala? All the code you would write ing and a built-in parser combinator ming features.
in Java becomes more concise in library.
Scala, and you gain the additional Here’s an example query adapted Scalatra
benefit of Scala’s rich collections from the Anorm documentation: One popular alternative to Rails
library. A great illustration of this is in the Ruby world is a lightweight
the new Anorm API in Play’s Scala val countries = framework called Sinatra. It’s ideal
module (http://scala.playframework. SQL("Select name,population for quickly building lightweight
org). It isn’t a traditional ORM, but
__ from Country")().collect { Web applications with minimal
a wrapper for the lower-level Java case Row("France", pop:Int) code, where massive scalability and
Database Connectivity (JDBC) API. => ("France", pop) interoperability with extensive third-
Anorm embraces a view I discuss case Row(name:String, pop:Int) party services are less important.
elsewhere,3 that there are benefits to if(pop > 1000000) => Compared to Rails, Sinatra is easier to
working directly with the collections (name, pop) use for websites without database per-
that your database driver provides, } sistence requirements, for example.
as long as those collections offer Scalatra (https://github.com/scalatra/
useful methods for working with Country is a database table, and scalatra)
_____ started as a port of Sinatra
them. In contrast, the benefits of the block passed to collect uses to Scala, but has since added new
converting back and forth between pattern matching to select the rows capabilities of its own.
those collections and domain objects we care about. In this case, we select Recall that in Play, you normally
don’t always outweigh the disadvan- France and all other countries where define routing, controllers, models,
tages of extra runtime complexity the population is greater than 1 mil- and views in separate files. This sep-
and overhead. lion (note that Scala case matching is aration of concerns makes sense for
Anorm wraps JDBC with Scala eager; that is, the first match “wins”). larger applications. In Scalatra, you
collections semantics and more con- Each case “body” returns the tuple can define everything in one file,
venient handling of the checked (name, population). The collect which is very convenient for small,
SEPTEMBER/OCTOBER 2011 89
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
The Functional Web
/* package declaration and imports ... */
that will be rendered with the Sca-
// UrlSupport and ScalateSupport are "traits"; late template engine (http://scalate.
// mixins of additional behaviors. fusesource.org). The second section,
__________
class TemplateExample extends ScalatraServlet shown in Figure 4, defines how the
with UrlSupport with ScalateSupport { application should respond to vari-
ous requests.
// Scala supports embedded XML literals, which we Setting up a Scalatra project and
// use to create this page template. They are mapped running it in development mode isn’t
// to a Seq (sequence) of Node objects. as straightforward as it is for Play.
object Template { Some familiarity with Maven or the
Scala build tool, sbt (https://github.
__________
// """multi-line string""". com/harrah/xsbt/wiki) helps. The
def style() = Scalatra README.markdown file that
""" comes with the distribution describes
pre { border: 1px solid black; padding: 10px; } the details.
body { font-family: Helvetica, sans-serif; } Once you have the project set up
h1 { color: #8b2323 } and running with the example code
""" in Figures 3 and 4, you will get the
page Figure 5 shows when you go
// The expression { title } will be replaced to http://localhost:8080 (the default
// with the value for the title method argument, port). The “hello world” link at the
// using the Scalate template engine. bottom takes you to the same page.
def page(title:String, content:Seq[Node]) = { Clicking the “date example” link
produces Figure 6, which demon-
strates the parsing and handling of
{ title } URL path values.
{ Template.style } Note how the route definition
automatically decomposes the URL
path /date/2009/12/26 into year,
{ title } month, and day values.
{ content } Finally, clicking the “form exam-
ple” link yields Figure 7. (I entered the
hello world word “Hello!” into the text field before
date taking the screen shot.) Clicking the
example “Submit” button produces Figure 8.
form example The value in the form text field,
Hello!, was passed as a parameter
with the POST and used by the appli-
} cation to prepare the response shown
} to the user.
Although Scalatra requires very
little code to create applications,
it actually scales better than you
might expect because it uses Jetty
(http://jetty.codehaus.org/jetty/) as
the underlying Web server.
Scalatra is a great tool for quickly
simple applications. As the applica- I adapted from the examples that building lightweight Web applica-
tion size grows, you can separate come with the distribution. (Actu- tions, especially if you’re already
responsibilities into different files. ally, a web.xml file is also required familiar with Scala and Java tools,
Let’s look at a simple one-file to configure the Web server.) The like sbt and Jetty. As with Play and
example of a Scalatra application, first section, which is shown in its Scala module, Scalatra lets you
broken into several sections, which Figure 3, defines an HTML template use the power of Scala collections
90 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Scala Web Frameworks: Looking Beyond Lift
beforeAll {
and other functional features to contentType = "text/html"
minimize the code you write and }
maximize your ability to transform
data as needed. // Routing: HTTP GET request for URL
// http://server:port/ (i.e., empty path)
Finagle get("/") {
Template.page("Scalatra: Hello World",
Finally, let’s consider Finagle (https://
_____
Hello world!
twitter/github.com/finagle), which Referer: { (request referer) map {
was developed at Twitter for building Text(_) } getOrElse { none }}
very fast, RPC-style servers using Route: /
Netty, a client–server socket API )
based on Java’s New IO (NIO) library. }
Finagle is designed to meet Twitter’s
needs for extreme scalability. // Routing: HTTP GET request for a URL with
Finagle is a good example of a very // the path "/date/YYYY/MM/DD", where Y, M,
focused server development tool that // and D will be assigned to the year, month,
// and day parameters, respectively.
doesn’t attempt to provide a full Web
get("/date/:year/:month/:day") {
stack. Instead, it focuses on serving Template.page("Scalatra: Date Example",
a specific need — the development of
fast, lightweight client–server net- Year: {params("year")}
working applications, in which the Month: {params("month")}
ability to scale is paramount. Day: {params("day")}
For clients, Finagle offers connec-
tion pooling, load balancing, failure Route: /date/:year/:month/:day
detection, failover, retry, and other )
features important for distributed, }
reliable, and scalable client access to
// Routing: HTTP GET request that will return
services. For servers, Finagle offers // a form with one text field.
“backpressure” (a defense against get("/form") {
denial-of-service attacks or other Template.page("Scalatra: Form Post Example",
rogue clients), service registration,
and support for protocols like HTTP, Post something:
Comet, Thrift, and Memcached/
Kestrel.
For the purposes of this col-
Route: /form
umn on the functional Web, Fina-
)
gle demonstrates the elegance and
}
power of compositional semantics
that are common in functional lang- // Routing: HTTP POST request, invoked when
uages such as Scala. Finagle uses // the form is submitted using POST.
an elegant composition mechanism post("/post") {
for handling the parallel paths of Template.page("Scalatra: Form Post Result",
normal and exceptional processing You posted: {params("submission")}
that any Web application must Route: /post
handle. )
}
Consider the ser ver example
shown in Figure 9, which is adapted
protected def contextPath =
from an example in the distribution. request.getContextPath
It demonstrates an HTTP server that }
separates exception handling from
normal control-flow processing and
how they’re composed together to
build the service.
SEPTEMBER/OCTOBER 2011 91
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
The Functional Web
92 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Scala Web Frameworks: Looking Beyond Lift
Note the composition of error types that are subclassed by Handle- that is, the andThen method, which
and normal response handling in the Exceptions and Respond, respec- composes invocation of the two
definition of myService. The under- tively, support a composition protocol apply methods in the objects so that
lying SimpleFilter and Service that’s common in Scala libraries — HttpServer handles exceptions first,
SEPTEMBER/OCTOBER 2011 93
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
The Functional Web
/* package declaration and imports ... */
object HttpServer { then normal processing. In either
/* A simple Filter that catches exceptions and case, the Respond object returns
* converts them to appropriate HTTP responses. */ a response asynchronously (using
class HandleExceptions a Future) to the client. Note this
extends SimpleFilter[HttpRequest, HttpResponse]{ model’s power in separating con-
def apply( cerns and building services that
request: HttpRequest, compose from smaller pieces.
service: Service[HttpRequest, HttpResponse]) = {
// "handle" is invoked asynchronously.
eb application development
// If an exception occurred, it sets the
// corresponding error status code.
service(request) handle { case error =>
W might be approaching 20 years
old, but we’re still learning new
val statusCode = error match { tricks as we apply the elegance, con-
case _: IllegalArgumentException => FORBIDDEN cision, and power of functional pro-
case _ => INTERNAL_SERVER_ERROR gramming ideas. The example Web
} and service frameworks I discussed
val errorResponse = here — Play, Scalatra, and Finagle —
new DefaultHttpResponse(HTTP_1_1, statusCode) demonstrate these capabilities, while
errorResponse.setContent( leveraging the best established fea-
copiedBuffer(error.getStackTraceString, UTF_8)) tures in traditional object-oriented
errorResponse // return value frameworks.
}
} References
} 1. D. Ghosh and S. Vinoski, “Scala and Lift:
Functional Recipes for the Web,” IEEE
/* The service itself. Simply echoes back "hello!". Internet Computing, vol. 13, no. 3, 2009,
* Note that no error handling is required here! */ pp. 88–92.
class Respond extends Service[HttpRequest, HttpResponse]{ 2. D. Pollak and S. Vinoski, “A Chat Appli-
def apply(request: HttpRequest) = { cation in Lift,” IEEE Internet Computing,
val response = new DefaultHttpResponse(HTTP_1_1, OK) vol. 14, no. 3, 2010, pp. 88–91.
response.setContent(copiedBuffer("hello!", UTF_8)) 3. D. Wampler, Functional Programming
Future.value(response) // asynchronous for Java Programmers, O’Reilly Media,
} 2011.
}
def main(args: Array[String]) {
Dean Wampler is a principal consultant at Think
val handleExceptions = new HandleExceptions
Big Analytics (http://thinkbiganalytics.
val respond = new Respond
com). He specializes in Scala and “big
__
data” analytics using the Hadoop ecosys-
// Compose the error Filter and Service together:
tem of tools. Wampler has a PhD in phy-
val myService: Service[HttpRequest, HttpResponse] =
sics from the University of Washington.
handleExceptions andThen respond
He’s the coauthor of Programming Scala
(2009) and the author of Functional Pro-
val server: Server = ServerBuilder()
gramming for Java Developers (2011),
.codec(Http())
both published by O’Reilly Media. He’s a
.bindTo(new InetSocketAddress(8080))
member of IEEE and the ACM. Contact
.name("httpserver")
dean@deanwampler.com
him at ________________ and
.build(myService)
follow him on Twitter, @deanwampler.
}
}
Selected CS articles and columns
http://
are also available for free at ____
ComputingNow.computer.org.
__________________
94 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Practical Security
Not Reinventing PKI until
We Have Something Better
Trinity College Dublin
P
ublic-key infrastructure (PKI) underlies the use cases to which it’s been put. In addition,
many Internet protocols and applications, numerous core features intended to be part of
providing widely implemented and well- a PKI have never really seen widescale deploy-
studied mechanisms for using asymmetric cryp- ment, and the PKI-related business models that
tography in support of key distribution and have grown up in the past decade have attracted
authentication applications and protocols. PKI, criticism. We’ll examine each of these aspects to
for example, is used as part of the Internet Key motivate our discussion.
Exchange (IKE) portion of IP security (IPsec) used First, because X.509 was designed long ago
in many virtual private networks (VPNs). It’s for one thing, but continues to be used for lots
also used in the handshake phase of the Trans- of other things (ironically, not including X.500
port Layer Security (TLS) protocol that secures authentication), some technology mismatches
most Web services. In addition, many other less exist. For example, having to select a “notAfter”
common, or less visible, applications use PKI. or expiry date for a public-key certificate is
PKI originated in the 1980s as part of the often inconvenient. Additionally, in some cases,
ISO’s work on directories, where the basic PKI allowing more than one public key to be con-
standard (X.509) was developed as a way for tained within a single certificate would be bet-
directory user agents to authenticate to directo- ter, whereas in others we might like more than
ries. In the mid-1990s, the IETF started a work- one signer for a certificate. X.509-based PKI
ing group (PKIX) to produce an interoperable can’t (without hackery, at least) do any of these
profile of X.509 for use with Internet applica- things, so these are real shortcomings with real
tions and protocols. Although PKIX’s lifespan impact — the idea that a consumer device such
has been extended many times over the years as a phone should have an expiry date baked in,
via added work items for various ancillary spec- for example, has been a barrier to using PKI for
ifications, the latest iteration of the core PKIX device-specific private keys.
specification is RFC 5280,1 which specifies the Second, the PKI community has never really
profile of X.509 for which PKIX was originally succeeded in solving some core problems —
chartered 16 years ago. mainly, the provision of mechanisms that would
So, given that we have a technology (X.509- enable Internet-scale key enrollment (where
based PKI) that has broad implementation sup- Internet-scale means for users, not Web serv-
port, sees widespread use, and has been under ers); nor have we ever had a working solution
constant development in one form or another for for searching for public keys at that scale. Both
more than two decades, we might wonder whether problems have been solved many times at the
there’s any need at all to consider reinventing enterprise scale, even for very large enterprises,
PKI. Yet various people have proposed doing although perhaps PKI has been significantly
just that from time to time. more costly in these situations than ought to be
the case.
Problems, Problems The PKI business models that have evolved
X.509-based PKI is by no means a perfect tech- over time have also tended toward entrench-
nology and is indeed a poor match for several of ing particular uses of the technology, with an
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 95
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Practical Security
emphasis on the presence of well- needs as they arose. This is in X.509 was much less entrenched
known “root” public keys (or trust contrast to the X.509-based PKI than is now the case. The lesson
points) in Web browsers and operat- approach of focusing on an infra- that I take from the SPKI exercise
ing systems. Currently, these trust structure for many applications. is that there is no point in trying
points’ owners charge, per year, for Both approaches have their draw- to develop something just slightly
public-key certificates, a situation backs: the X.509-based approach better — a new technology must be
that doesn’t work well for noncom- is more likely to produce work that radically better to stand a real chance
mercial websites, for example, but isn’t used, whereas the PGP approach of replacing one as mature as X.509-
that also grates even for commercial is more likely to produce work with based PKI.
sites. The set of trust points embed- limited scope. Having said that, PGP XKMS illustrates yet another recur-
ded into browsers and operating and X.509-based PKI have evolved ring theme — format wars. X.509 uses
systems might also have stifled the to the point where both now provide Abstract Syntax Notation (ASN.1),
market for new PKI services; they’ve similar features. PGP doesn’t have which ISO originally developed as
led application developers toward the associated businesses operating part of its Open Systems Intercon-
using TLS or HTTPS because doing commercial certification services — nection (OSI) framework. The ASN.1
so meant they could inherit some something the PGP communit y family of specifications provides a
trust points for their applications. believes is a strength but the X.509- way to describe data structures used
Storing overlapping sets of X.509 based PK I industr y considers a in protocols, and how those can be
trust points in many browsers and weakness. Although PGP has seen encoded for transmission via net-
operating systems also has a poten- widespread deployment, new appli- works. Because they aim to be very
tially significant security weakness: cations don’t tend to adopt it, outside generic, ASN.1 encoding schemes
each and every trust point is trusted the open source community, because involve quite a lot of unnecessary
by the client to issue public-key X.509-based tools and libraries are overhead and also generally produce
certificates for any name whatso- more common, and substantial indus- hard to read or debug binary for-
ever. So, if any of those (hundreds of) try support exists for X.509-based mat encodings. Development tools
trust points make a mistake and PKI. In principle, however, little else also tend to be less widely available
issue a certificate wrongly, this can of significance differentiates X.509- than for other formats, and for long
affect any Web service, for any cli- based PKI and PGP. So, PGP’s exis- periods, good open source ASN.1
ent on the Internet. Because this has tence alone doesn’t really provide development tools weren’t avail-
happened a few times, and quite any compelling reason to switch able. ASN.1 does, of course, work,
recently,2 the operators of, in par- (in either direction). but it isn’t very developer-friendly,
ticular, large scale Web services SPKI was intended to usurp although you do get used to it after
are now quite nervous about this X.509-based PKI and is arguably a while. Roughly a decade or so ago,
exposure and would like to have based on a more generic PKI model XML became the flavor of the month
some control over this process so in which each relying party (RP) data format, roughly as JavaScript
that any mistakes made have less decides which keys to treat as trust Object Notation (JSON) is today, so
impact. points. This differs from current some pressure existed to redefine
X.509-based PKI implementations, PKI to use angle brackets rather than
Supposed Alternatives where applications and operating ASN.1’s data structure definition and
Given these issues, it’s no surprise systems providers make those deci- encoding scheme.
that alternatives to X.509-based PKI sions. SPKI also offered an authori- The arguments I’ve just described —
have been suggested over the years, zation model (as does X.509), but the essentially, that ASN.1 sucked, and
with perhaps the most significant provision of a common authentica- that because everyone was doing
work done on Pretty Good Privacy tion and authorization infrastructure, everything in XML, it would be
(PGP),3 Simple PKI (SPKI, usually pro- while initially appealing, seems not easier and better all around to for-
nounced “spooky”),4 and the XML Key to work, given that different peo- get the existing work and start over,
Management Specification (XKMS).5 ple frequently develop the different avoiding the mistakes that occurred
Let’s briefly look at each in turn. rules and policies for authentica- in the past — were made in favor of
PGP isn’t really cast as a direct tion and authorization at different developing an XML-based PKI. (You
replacement for X.509-based PKI but times. SPKI failed to catch on, in my might sense that this writer wasn’t
has generally been developed in par- opinion, because it just didn’t offer a convinced.)
allel, with features added over the significant benefit over X.509, even Rather than actually reinvent
years to meet specific applications’ though SPKI was developed when PKI though, consensus was reached
96 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Not Reinventing PKI until We Have Something Better
in the standards development com- being deployed provide for crypto- DNSSEC deployment much more
mu n it y to i nstead develop a n graphic protection for DNS responses likely now than was the case even
XML-based way to interact with an based on a key hierarchy managed two years ago.
X.509-based PKI, which became the by domain registries.6 If public keys
W3C’s XKMS recommendation. But stored within the DNS (and hence What Might Happen
XKMS was intended to be more — associated with domain names) are Instead?
that is, it also aimed to enable XML secured with DNSSEC, then many So, the question is how to properly
consuming applications to interact of the functions required of a PKI plan for and regard PKI’s evolution?
with other forms of PKI that might are provided — using this, DNSSEC- This, of course, brings us into the
supersede X.509. However, XKMS aware applications can find public realm of speculation, which is usu-
has seen basically no real deploy- keys easily and authenticate them ally futile, but sometimes fun.
ment at all in the past decade. In as being associated with a named One area where I would hope to
my opinion, this is partly due to the entity. DNSSEC can thus provide see progress is in developing schemes
realization that yet another data for- either an alternative to or, more that would actually allow for clients
mat won’t in fact make life any eas- likely, an additional level of assur- to use their own key pairs in a PKI. A
ier for developers, but also partly for ance for X.509-based PKI. Indeed, a successful technology for this would
the same reasons that told against relatively new IETF working group greatly help with current problems
SPK I — not enough added benef it (DANE; http://tools.ietf.org/wg/dane/) with passwords and phishing. The
and an entrenched industr y and is tasked with specifying just this recent level of server-side breaches,
community backing for the X.509- functionality. exposing hundreds of thousands of
based way of doing things. If (as I expect) DANE succeeds passwords, might have brought us
So, we’ve had a mature technol- and is widely deployed, then it to the point where it’s worth looking
ogy (X.509-based PKI) and some might ameliorate the trust point again at how to deploy client-side
challengers, only one of which (PGP) “scope” vulnerability described earlier. private-key handling. Although the
has really seen deployment at any DANE, thanks to DNSSEC, poten- protocols and client-side technol-
scale. We might then ask — what tially offers a way to additionally ogy for this have existed and been
would be required to really displace bind DNS names to public keys cer- widely deployed for more than a
X.509-based PKI to any significant tified via X.509 — the difference decade, user interface, private-key
extent? between the two bindings being that management (especially considering
First, I should probably say what the control over the DANE binding mobility), and business issues with
I mean by “displace” — what I don’t is often much nearer to the domain that technology remain and con-
mean is that some putative new operator than with current X.509- tinue to limit the use of client-side
technology would cause us to imme- based PKI implementations in brows- private keys to enterprise use cases
diately stop using X.509-based PKI. ers. DANE, however, is unlikely to and (mostly) niche applications that
Short of catastrophic cryptographic replace the current X.509-based PKI have hidden all the PKI complex-
algorithm breaks, that won’t happen. deployments because it would merely ity from users. We’re now at a point
What I do mean is that the putative replace the too-many unscoped trust where we could revisit this, and, if
new technology would become the points problem with a potentially all the right parties are willing to
technology of choice for new appli- much worse too-many-registrars work on the problem, we could even
cations and protocols that require problem. Although dealing with succeed.
public-key management functions, hundreds of trust points might present What I have in mind for this is
such as authenticated key transport/ difficulties, dealing with hundreds a new HTTP authentication method
agreement and signature verification. of registries and thousands of essen- that involves Web (and other HTTP)
One technology currently being tially unknown (to the RP) regis- servers in asking for TLS mutual
developed is based around using trars would almost certainly present authentication, but that uses a dif-
the DNS to store public keys. The equally bad problems. DANE also ferent key pair for each client for
DNS underlies many services on the requires that DNSSEC be deployed each service, and where the public
Internet and is generally trusted for before it can achieve real utility, and key need not be certified by a public
mapping from names to IP addresses. DNSSEC deployment has been on the certification authority (CA).
To date, that mapping hasn’t usually cusp of happening for many years, Two new pieces of technology
been cryptographically protected, without actually having happened. would be needed to make this work.
but the DNS Security (DNSSEC) However, recent developments in First, servers would have to provide
specifications that are now finally signing the DNS root make broad a key-registration service for each
SEPTEMBER/OCTOBER 2011 97
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Practical Security
separate Web service, where the versions are now much more fre- In summar y, although X.509-
service provider could register a quently updated means that this is based PKI has well-known problems —
newly minted client public key for achievable, if the relevant parties are the most important of which (the
authentication to that service. This interested. scoping of CAs) DANE is addressing —
would need to be invisible to users, The scheme I’ve outlined, which there is at this point no real benefit
with the additional round trip hap- requires no changes to existing in trying to reinvent PKI.
pening between the browser and X.509-based PK I, together with
key-registration service whenever ongoing improvements in how it’s Acknowledgments
the user doesn’t have an existing key deployed (such as DANE) impose a I have been, and continue to be, involved
stored for that service. Note that the very high barrier to entry for any in a number of these PKI related activi-
key-registration service doesn’t need new PKI technology. In particular, ties in a number of ways, from document
to deal with identity — the identi- there’s little point in entering into author to various cat-herding roles. In all
fier associated with the user’s public another format war — for example, of those roles, I’ve usually made fairly
key would be the service’s and not attempting to develop all this func- decent mistakes in my predictions, so caveat
a user identifier. Separating iden- tionality in JSON — because that lector.
tity handling from key management wouldn’t really offer anything new of
should let services build key regis- note, and would likely just consume References
tration into whatever identity han- effort for little positive outcome. 1. D. Cooper et al., Internet X.509 Public-
dling workflow they wish to use; the One reason does exist, however, for Key Infrastructure Certificate and Cer-
user’s key pair is just like a password developing cryptographic APIs and tificate Revocation List (CRL) Profile, IETF
(but better) and need not actually be formats for JSON — just as XML RFC 5280, May 2008; www.ietf.org/rfc/
bound to any identifier for the user application developers can select the rfc5280.txt.
______
when sent over the wire. The service XML digital signature specification 2. P. Hallam-Baker, “The Recent RA Com-
can associate the public key with rather than its ASN.1 equivalent to promise,” blog, 23 Mar. 2011, http:// ____
whatever identity or account han- make their lives easier, the same blo g s .c omo do.c om /i t- s e c u r i t y/d at a
________________________
dling it wishes. tools should be available for JSON -security/the-recent-ra-compromise/.
_____________________
Second, to handle user mobility application developers. However, if 3. J. Callas et al., OpenPGP Message Format,
and the now-common case of users we want the same kind of private- IETF RFC 4880, Nov. 2007; www.ietf.org/
with multiple browsers, we need a key management to be usable for dif- rfc/rfc4880.txt.
_________
way to bind different keys from dif- ferent services, then each one should 4. C. Ellison et al., SPKI Certificate Theory,
ferent devices to the same service- be able to use the same PKI. IETF RFC 2693, Sept. 1999; www.ietf.org/
managed identity or account. This rfc/rfc2693.txt.
________
could simply be a well-known URL 5. P. Hallam-Baker and S. Mysore, XML Key
available at the service where a he scheme I’ve outlined in the
user who’s authenticated with one
key could get a short-lived code or
T previous section illustrates how
I would see X.509-based PKI devel-
Management Specification (XKMS 2.0),
W3C recommendation, June 2005; ___
w3.org/TR/xkms2/.
www.
other value that, when entered into oping in the near term — with some 6. R. Arends et al., DNS Security Introduction
a session authenticated with the key new (or revisited) use cases and and Requirements, IEFT RFC 4033, Mar.
from another device, would bind applications using the technology 2005; www.ietf.org/rfc/rfc4033.txt.
the two keys to the same account. but, as with DANE, modifying the
Again, this divorces key manage- business and trust point models that Stephen Farrell is a research fellow at Trin-
ment from identity or account man- have developed over the past decade ity College Dublin and chief technolo-
agement, leaving the latter to the so as to make the PKI far less visible gist with NewBay Software. His research
service. to end users. interests include security and delay/
Implementing these solutions Research into new models for disruption-tolerant networking. Farrell
wouldn’t be difficult, but would authentication and key management has a PhD in computer science from Trinity
require coordinated action from a should of course continue, and will stephen.
College Dublin. Contact him at _____
wide range of both browser vendors someday produce a technology that farrell@cs.tcd.ie.
__________
and Web services before the new will displace X.509-based PKI, but
scheme could be deployed at scale. I don’t expect that to start happening
However, I hope that the rather large for several years yet, given that no Selected CS articles and columns
costs associated with server data compelling candidate technology is are also available for free at http://
____
breaches and the fact that browser on the table at present. ComputingNow.computer.org.
98 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Peering
Emergent Collectives
Stanford University, retired
I
n recent columns, I’ve referred to an arti- This is an important point with respect to
cle I published in 2005 in a French jour- prediction and why I used the term “emergent.”
nal, on emergent collectives.1 Though I’ve When a system’s behavior is governed more by
given the URL before (http://www-cdr.stanford. its protocol rather than some form of central
edu/~petrie/revue/), some new developments
______________ control, that behavior is hard to predict with
(including a new research project from Belgium today’s tools: we can’t predict emergent behav-
[www.emergent-collectives.be/en/]) mean it’s ior very well. And if you’re predisposed to think
time to summarize and review that article’s con- in terms of central control, then certainly you’ll
tent here, as well as expand on it based on some be consistently fooled.
recent experience. But that’s not all. The Internet itself, the
WWW, music and video file sharing, Wikipedia,
Emergent Collectives and Linux have lots and lots of people contrib-
Explain Disruptions uting huge amounts of content and building out
The article first reviews how certain distributed tremendous networks of information and func-
Internet-based systems have been unanticipated tionality, for free! This goes counter to most
by people who should have been experts, such older notions of economics.
as RIAA executives, Bill Gates, ATM commu- Although I hadn’t read “The Cathedral and
nications experts, Robert Metcalfe, and those in the Bazaar” (w w w.catb.org/~esr/w r it i ngs/
charge of France’s Minitel system. I made the cathedral-bazaar/cat hedral-bazaar/) when I
____________________________
point that despite what Dilbert cartoons would wrote the Revue article, Eric Raymond captures
have us believe, these people aren’t fools, and well the notion that people want to contribute to
asked what has been happening (and continues something larger than themselves. Social net-
to happen) with these disruptions that surprised works’ success suggests that we abstract from
technically savvy people? this motivation because many postings at best
I can’t resist digressing here. France has a can be described as building community by
long history of making iconic, wrong-headed self-advertising. But people have certainly used
Internet initiatives. Most recently, President social networks for good purposes, and the
Nicolas Sarkozy has proposed a “more civi- social motivation is present in any case.
lized Internet” (see http://tinyurl.com/3u946yy). The features of an emergent collective are
As one blogger recently said, good luck with
that French Intranet (or they could just bring a network of information/function nodes
back the Minitel). Here’s a suggestion for Scott that has minimal central control, and that’s
Adams: put Dilbert on loan to the French gov- largely controlled by a protocol specification,
ernment to implement Sarkozy’s suggestions. in which it’s easy for people to add nodes to
Back to the question: How is it that all these the network,
smart folks have been so wrong-headed? A par- and where they have a social incentive to
tial answer: engineers and government officials do so.
are trained to think of systems with a central
control, and they’re dealing with systems where Such networks grow quickly, and their
the control is largely in the protocol. behavior is difficult to predict, especially if your
SEPTEMBER/OCTOBER 2011 1089-7801/11/$26.00 © 2011 IEEE Published by the IEEE Computer Society 99
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Peering
models depend on central control I speculate that Wi-Fi-sharing of individuals who link up for short
and “rational economics.” was a weak emergent collective time periods, supported by new
because many people were sharing Internet technologies that let them
A Failed Prediction not because they wanted to contrib- find each other (possibly via social
What can we predict from this char- ute to something larger but because networks) and coordinate their work.
acterization, if it’s true? Well, one of the routers came shipped with no Such a prediction might be more
my related predictions from 2001 has security as a default. wishful thinking than an actuality,
somewhat failed 10 years later. I pre- Free and open hotspots have yet it has a good chance of coming
dicted that 3G cellular wouldn’t be grown as businesses have increas- true if only because of economic
the “wireless Internet” of the future ingly recognized the economic ben- forces. We’re all becoming self-
and Wi-Fi would (http://www-cdr. efits of doing so, much as I predicted employed. We’d better get used to it
stanford.edu/~petrie/802.11-Stockholm-
__________________________ in 2001. But the emergent collective and at least hope for better Internet
2001/). I wasn’t entirely wrong in
____ of Wi-Fi has largely collapsed; it’s tool support. I still suspect such new
this, but I was wrong in the “emer- instructive to see that the Wi-Fi net- tools will emerge, because a mar-
gent collectives” article to predict work we have today isn’t an emer- ket exists for them: the increasing
the growth of the user-powered gent collective, and that emergent need to outsource larger and more
Wi-Fi network, and it’s instructive to collectives can die, especially if their complex tasks. This is being done
consider this case. social incentive is weak. right now by boutique consulting
I underestimated the power of What can we learn from this case, companies with access to various
cellular providers to throw unimagi- other than that prediction, especially experts. This, too, will become more
nable amounts of cash at the prob- of the future, is difficult? First, that of a commodity owing to economic
lem, including persuading people to the social incentive must be strong, pressure. So good reasons exist to
be fearful of unsecured Wi-Fi net- and we don’t know how to measure believe in such a future — but how
works, even persuading Germany and this. Second, we can’t ignore tra- can we really evaluate the likelihood
Italy to pass very restrictive connec- ditional economic forces. We don’t of emergent collectives, either social
tivity legislation. know how to measure this, either. So or economic, when we have so few
In the US, the cell providers prediction just got harder. tools?
have been remarkably successful
at preventing cities from providing Predicting Emergent A Major Research
free Wi-Fi to their residents. Only Collectives Opportunity
Mountain View, California, has suc- Apple’s “apps” aren’t an emergent Here’s something that needs to hap-
ceeded, and only because of Google. collective as defined initially. They pen in our research world: we should
Finally, even I have a 3G smart- did create a platform in which it was develop a better understanding of emer-
phone, because it’s the cheapest way relatively easy for individuals to add gent behavior based on combinations
for me to get Wi-Fi for my laptop at nodes and value. A sort of proto- of protocols and social incentives.
home, where I write this. The cellular col exists in terms of the API. But Suppose we could design emergent
providers have been much more suc- the incentive is economic. It’s like collectives so that useful behavior
cessful at suppressing this particular an emergent collective, but perhaps resulted?
emergent collective with the security/ easier to predict because of the eco- Were we able to do so, we could
fear tactic than have been the record nomic incentive. Maybe we could certainly achieve a lot more. For
companies with the tactic of suing say there are two types: social and all the discussion in the distributed
teenagers. economic emergent collectives. The agents community about emer-
So consumer-provided free and former remain harder to predict than gent behavior’s value over the past
open 802.11 networks have collapsed, the latter, but mixes will be even 20 years (at least), very little in the
largely due to consumers’ fears that more difficult to predict. way of engineering methods exist
someone would sit in their driveway Another prediction that I’ve made for predicting emergent behavior,
listening to their signals — even though in this space2 is that, because of the much less designing it, although
for years any hackers inclined to sit in creeping commodization of every- there is some related analysis.3 Were
their driveway could easily crack the thing, Amazon’s Mechanical Turk some researcher to actually develop
security most people used. People are (www.mturk.com/mturk/welcome) a general algorithm that would pre-
no longer sharing their Wi-Fi, sadly will become more mechanized and dict collective behavior based on an
leaving broadband providers with that, in fact, more complicated jobs interaction protocol and external
more control than necessary. will be done by “flash companies” constraints, it would be an important
100 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Emergent Collectives
development worthy of an interna- companies, this is an oxymoron: contrar y to the usual enterprise
tional award. they’re designed to run routine pro- theory of effectiveness: Google is
Possibly the most impor tant cesses efficiently and not for the con- “wasting” 20 percent of its produc-
potential application of such new sumer’s benefit, despite mottoes to tivity. Yet, just looking in from the
science would be in computational the contrary and laughably wrongly outside, this seems to be working,
cognition. We still have no computa- named “customer relations systems.” at least with respect to innovation.
tional model of human minds. Such Yet, as John Hagel and John Wouldn’t it be cool if companies
a model would explain, for instance, Seely Brown point out, these same could predict such outcomes prior to
the phenomenon of attention and companies are filled with smart, cre- trying them for real?
how it shifts, or how it affects sub- ative folks who go largely unrecog- Turns out, they can. They could
jective time. We don’t have this nized (http://blogs.hbr.org/bigshift/ simulate new behavior protocols in
model, but we do strongly suspect 2010/04/are-all-employees-knowledge-
__________________________ focus groups, with employees play-
that our minds are composed of dis- wo.html). Such people are frequently
_____ ing the roles of likely types of peo-
tributed (probably stupid) agents, employed in routine jobs and must ple. Such role-playing capabilities
largely unknown to our conscious use their real talents in hobbies and currently exist among management
mind (whatever that is), that some- “skunk projects” while the com- consultants to evaluate leadership
how settle among themselves what pany suffers from being unable to effectiveness (http://mz-x.com/files/
gets brought to our conscious mind’s respond adequately to changing Info_LeadershipSimulation_en.pdf).
_________________________
attention, very unlike our computer market conditions and technologies. Such simulations’ capabilities could
operating systems’ central time- These two authors also discuss the easily be adapted to test new inter-
sharing model. formation of “creation networks” nal business processes designed to
Ver y early work in t his area (www.johnhagel.com/paper_pushpull. address problems and increase cre-
exists,4 but it doesn’t yet tell us how pdf ), wh ich have some relat ion
___ ativity. We don’t have to have the
attention shifts much less predict dis- to emergent collectives but which science to completely predict the
tributed agents’ emergent behavior. focus on practical tactics companies emergent behavior in order to try
A conference devoted to this research might try to encourage productivity out intuitions about how to change
area will take place in November creativity. things for the better.
(http://cogsys.org/acs/2011/home/), Now imagine that networks of For instance, suppose you have a
and we can hope for impor tant like-minded creative people inside problem with mid-level managers not
results. companies could emerge that were taking the initiative but rather always
designed to solve recognized prob- passing decisions up to the next
A Practical Approach: lems by changing the company’s level, which is a counter-innovative
Enterprise Simulation behavior. Such systems would be behavior. The solution is to sim-
We don’t have to wait for such kinds of games in which people ulate an incentive program that
advanced science and discoveries to would be motivated to creatively and rewards decision-making. Problem
use the notion of emergent collec- collectively solve such problems. But with decision transparency? Build
tives to have more near-term impact. we don’t know enough about how to that into the game. Tweak as nec-
I suggested in the emergent collec- predict such emergent behavior, so essar y. Include some t ypical role
tives article1 that companies perform how can we design such systems? behaviors likely to prove problematic
simulations of possible emergent col- Again, we don’t have to wait for and see what happens.
lectives to predict disruptions. Sim- new science and engineering. We I was a part of such a role-playing
ulation is what we know how to do can experiment. This isn’t something game development recently, and the
today: just let the distributed agents firms tend to do — either they’re doing results were quite sur prising and
system run and see what happens. poorly, in which case they don’t want rewarding to all involved in the
I now suggest a very narrow but to waste resources, or they are doing exercise. One key insight was that
important practical application for well, in which case they don’t think the behavior wasn’t determined by
enterprises, based on a recent expe- they need to improve. But companies central fiat but rather by the interac-
rience that led me to view networks can and should always be engaged in tion protocol, including both reward
of people much as an Internet-based such small-scale experiments. and social incentives. I wish I could
technology. Google is running a small exper- talk about this more, but it was a
A lot of money and time is cur- iment by letting some employees small private exercise concerning a
rently going into the topic of “inno- work for one day per week on what- real problem in a real company. How-
vative companies.” But for most ever they think is important. This is ever, it gave me the confidence to
SEPTEMBER/OCTOBER 2011 101
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Peering
suggest that others experiment with with existing tools. In particular, Proc. 2nd Int’l Conf. Software Eng. and
this approach: it can be a practical way role-playing simulations are a prac- Formal Methods (SEFM 04), IEEE Press,
to change an enterprise’s behavior. tical approach to designing protocols 2004, pp. 24–33.
Other than being preoccupied that can achieve new desired behav- 4. W. Br ide we l l a nd P. L a ng le y, “A
with the day-to-day urgencies of get- iors inside enterprises. Computational Account of Ever yday
ting the product out the door, there’s I look forward to new results in Abductive Inference,” Proc. 33rd Ann.
no reason why companies can’t con- this early decade of the 21st century Meeting of the Cognitive Science Soc., Wiley,
duct such role-playing simulations by today’s young researchers and 2011; w w w.isle.org/~langley/paper s/
today, and there are very good rea- practitioners. abduction.cogsci11.pdf.
_____________
sons to do so.
References Charles Petrie retired from Stanford Univer-
y points here are that emergent
M collectives are an important
feature of today’s economy, that
1. C. Petrie, “Emergent Collectives for
Work and Play,” AGIR Revue Generale
de Strategie, Societe de Stratege, La
sity as a senior research scientist with the
CS Logic Group. He received his PhD in
computer science from the University of
impor tant fundamental research societe de ‘information, nos. 20–21, 2005, Texas at Austin. Petrie was a founding
remains to be done in distributed pp. 146–152. member of the technical staff of the MCC
(agent) systems, that these principles 2. C. Petrie, “Plenty of Room Outside the AI Lab, founding editor in chief of IEEE
extend beyond Internet technologies Firm,” IEEE Internet Computing, vol. 14, Internet Computing, founding execu-
into almost any group of people who no. 1, 2010, pp. 92–96. tive director of the Stanford Networking
nevertheless are connected by some 3. C. Rouff et al., “Properties of a Formal Research Center, and founding chair of
protocol that governs their behav- Method for Prediction of Emergent the Semantic Web Services Challenge.
ior, and that more could be done Behaviors in Swarm-Based Systems,” petrie@stanford.edu.
Contact him at ____________
C all icles
for Art
uting
e Comp
vasiv st
IEEE Per ul p ap e r s
o n th
e late
, u s ef ible e, siv
a c ce s s per va
seek s nts in
eve lopme ics
evie we d d g. Top
peer-r m putin
ous co
u biquit a re
e, and y, sof t
w
mobil olog
te ch n
wa re g an d
e ha rd ensin
includ rld s
e , re al - wo ion ,
u c tu r terac t
infras
tr ter in
s: n- c ompu
id eline huma g
or gu c tion, cludin
Au t h .org /m
c/ intera ratio ns, in
p u te r nside ac y.
.com s co n d priv
www s te m rit y, a
/auth
or.htm an d s y y, secu
vasive bilit
per t, scala
t ai ls: ymen
er de deplo
Furth er.org e
mp u t rvasiv
e @ co
rg/pe
uter.o
siv
p e r va
.comp
www
102 www.computer.org/internet/ IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
The Battle for Internet Openness
cont. from p. 104 What isn’t called for is unbridled Cloud systems themselves must be
international identity management. liability for the intermediaries that highly resistant to external and
The NSTIC proposal is broad enough facilitate communication on the internal penetration. Access to infor-
to encompass strongly authenticated Internet, turning them into unli- mation held in cloud computing
pseudo-identities in addition to censed police forces. Rather, it seems systems must be strongly protected
strong personal identities. We might that we need national and inter- even while making it easy for the
think of the former as strongly national norms for due process and legitimate owners to authorize the
authenticated identifiers that, in and discovery of harmful actors. Protect- access and transfer of protected
of themselves, don’t identify persons ing civil rights must go hand-in- information.
or institutions but can be routinely hand with protecting against harm,
and repeatedly validated as “the and it’s the balance between these hat this process and balance will
same party” with whom a person has
had earlier interactions. We can then
two important societal benefits that
we must achieve.
T be difficult to achieve goes with-
out saying. A slippery slope awaits
associate other identifying informa- International efforts to develop efforts to achieve protection in the
tion with these identifiers, if and tools to detect malware, viruses, worms, face of preser v ing openness, and
when this proves necessary. and Trojan horses are required. Sim- popular actions might hide a much
Assuming we don’t want to live in ilarly we need tools to detect various more pernicious agenda in some
a world in which privacy is completely forms of active attack against the cases. The technical community has
expunged, we must consider how to Internet’s infrastructure and edge an important role to play in care-
support anonymity or pseudonymit y devices. Using digital signatures to fully assessing proposed methods
in addition to strongly authenti- protect DNS entries, routing table of achieving protection while pre-
cated, trusted identity. “But, what,” announcements, BIOS fi rmware, the serving the benefits of an open and
you say, “can we do about malfeasance origins of email, and software can vibrant Internet.
in the Internet? How can we identify go hand-in-hand with more resis-
the miscreants?” This calls for devel- tant operating systems, more “sus- Vinton G. Cerf is vice president and chief
oping much better forensic tools and picious” browsers, and perhaps more Internet evangelist at Google. Contact
international ground rules for their use. judicious use of cloud-based services. him at vint@google.com.
__________
Is your career
foundation solid?
_________________________
SEPTEMBER/OCTOBER 2011 103
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
Backspace
The Battle for
Internet Openness
Google
A
s the second decade of the 21st century harms visited on them by their fellow citizens,
opens, the Internet, now a global and grow- especially those outside national jurisdictions.
ing infrastructure, presents challenges Governments in the main are instruments of
unlike any in history. Our ability to speak and civil order and organization. They provide rules
be heard through this global platform is unprec- for interaction among citizens, private sector
edented. In the past, access to mass media was entities, and other national bodies. If our goal
largely confined to reception, and rarely allowed is to preserve the Internet’s ability to absorb
the listening masses to speak to each other in new ideas and uses, we must find ways to pro-
more than a bilateral fashion. tect citizens, institutions, corporate entities,
Moreover, the cost of speaking today is low and governments from abuses visited on them
compared to the cost of accessing mass media through this global medium. We could poten-
in the past. You don’t need to own a radio sta- tially achieve this through purely technical
tion, television station, cable system, or print- means, but this might not only be impossible
ing press to speak broadly. Companies providing but perhaps also undesirable. We might imag-
mass infrastructure have emerged from the ine that eliminating all anonymity could curb
evolving Internet ecosystem. Twitter, Facebook, abuses, but much abuse clearly happens in the
Google, YouTube, Blogger, Amazon, Skype, and real world from sources that are anything but
many other ser vices permit convenient and anonymous. Moreover, reasonable situations
often cost-free access to infrastructure capable exist in which lack of anonymity threatens the
of reaching a global audience. In addition, these freedom of expression that’s valued in the UN’s
platforms are increasingly accessible to mobile Declaration of Human Rights. From whistle-
devices that now number on the order of 5 bil- blowing to the exposure of corrupt government,
lion. That these devices permit voice and video anonymity has a place in the space of Internet
recording, text exchanges, image uploading and expression. At the same time, many interactions
downloading, and even streaming media simply among individuals, between corporate entities,
emphasizes the scope and scale of this 21st cen- among governments, and combinations of these
tury communications environment. would benefit from the ability to confirm their
Much of the Internet’s benefit lies in its open- identities to each other in advance of specific
ness to new applications, new technology, new interactions. Finding a technical means to let
forms of expression, and new users and uses. both anonymity and strong identity coexist is a
It isn’t surprising, therefore, that societies that challenge worth trying to meet.
haven’t been traditionally open to free expres- The White House cybersecurity coordina-
sion might see this openness as a threat to social tor’s office has proposed a National Strategy for
stability or to their populations’ well-being. In Trusted Identities in Cyberspace (NSTIC). This
fairness, the Internet’s openness has also pro- proposition allows for private sector develop-
vided opportunities for a Pandora’s box of ment and provision of trusted identity services
potential harms coming from many directions, and technology. In principle, we need metrics
including those that lie outside particular soci- for the strength and quality of any methods
eties’ jurisdictional boundaries. The conundrum intended to achieve this objective. This is even
that the Internet presents is preserving openness more important as we consider the need for
to new modes of use while protecting users from cont. on p. 103
104 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
2011–2012 Editorial Calendar
Virtual World Architectures (Sept/Oct 2011)
3D virtual worlds such as Second Life, Open Simulator, and so on let users model real and fantasy
worlds. Some of these worlds are extensive, with tens of thousands of avatar “residents,” and require
grids of thousands of machines. Although it isn’t hard to believe that a 3D Web can someday
gracefully complement today’s document-centric Web and that such virtual worlds will model the
Earth in credible detail, this is slow in happening. Educators and serious gamers want to experiment
with virtual worlds but often find the platforms difficult to extend. What are the roadblocks and
how can we accelerate the pace of progress to realize the vision?
Semantics in Location-Based Services (Nov/Dec 2011)
Advances in wireless networks and mobile devices have motivated an intensive research effort
in mobile computing and mobile data services. Along with many advantages, using location
information in a mobile environment can also pose significant research challenges regarding data
management. In this context, different Semantic Web technologies could be adapted and applied to
make intelligent location-based services a reality.
Internet-Scale Data Management (Jan/Feb 2012)
The massive volumes of distributed data on the Internet present a tremendous data-management
challenge. Traditional solutions weren’t designed with the scale, heterogeneity, or volume of
Internet data in mind, and were typically engineered to assume structured data managed by a
single organization rather than the unstructured or loosely structured and federated nature of data
on the Internet today.
Beyond Search: Context-Aware Computing (Mar/Apr 2012)
Context-aware computing offers mobile Internet users an experience that goes beyond user-initiated
search and location-based services. Context awareness sharpens relevance when responding to user-
initiated actions (such as product search and support calls). It also enables proactive communications
through analysis of a user’s behavior and environment, thereby forming the basis for key business
imperatives targeting customer-engagement systems. Even greater opportunity arises from context
use in systems that can make sense of and engage in customer dialogs and forums.
Infrastructures for Online Social Networking Services (May/June 2012)
The proliferation of rich social media, online communities, and collectively produced knowledge
resources has accelerated the convergence of technological and social networks, resulting in a
dynamic ecosystem of online social networking (OSN) services, environments, and applications.
OSN sites’ success is reshaping the Internet’s structure, design, and utility. It’s also creating
numerous challenges and opportunities for the development, deployment, management, and
operation of scalable, secure, interoperable OSNs infrastructures.
Programmatic Interfaces for Web Applications (July/Aug 2012)
The rapid growth of programmatic Web service interfaces for Web applications (open Web
APIs) has revolutionized online content integration and development practices. The increasing
popularity of such Web interfaces raises questions of how
developers should design services and how they should
maintain services’ good performance and scalability.
Programmatic Web interfaces typically use REST style
for communication, or RESTful services implemented
with HTTP, while moving away from more traditional
SOAP Web services. www.computer.org/internet/
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®
__________________________________________________
Distinguish Yourself From the Crowd
Earn Your CSDP
Earning the Certified Software Development
Professional (CSDP) credential is the best way
to prove your abilities, skills, and knowledge. ertified
oftware
By adding the CSDP credential to your resume, evelopment
rofessional
you will demonstrate you are:
➢ Current with best software practices
➢ Connected with industry’s brightest minds
➢ Career-minded and ready for that next
promotion
➢ Committed to advancing the software
engineering profession
To read how the CSDP credential has helped employers and employees, go to:
www.computer.org/getcertified
qM
qM
qM
Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next Page M M
q q
THE WORLD’S NEWSSTAND®