Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

The Business Interactive Entertainment

VIEWS: 88 PAGES: 26

									IBM Digital Media Solutions May 2006

The Business of Interactive Entertainment
Appropriately sizing and hosting your online game project’s infrastructure can improve your bottom line

George Dolbier, Executive Architect IBM Digital Media Solutions Ayalla Goldschmidt, Marketing Manager IBM Digital Media Solutions

The Interactive Entertainment Industry Page 2

Contents 2 2 5 Introduction Anatomy of an industry Critical taxonomy of an industry

Introduction Reaching far beyond the familiar PC and console game realm, today’s Interactive Entertainment industry has a hand in everything from cell phones and cars, to Las Vegas casinos and global currency fluctuation simulation. Today, you can find interactive technology used to simulate everything from battle situations to auction markets, from microbiology to entire societies. In spite of all its diversification and cross-market saturation, the primary goal has remained largely intact and unscathed. What it boils down to is the ability to leverage technology as a means to provide compelling entertainment and socialization products to the general public. As a human endeavor, Interactive Entertainment is, in essence, a collaborative art form nearly as complicated as the games it produces. The typical modern game is a combination of 2-D graphic arts, 3-D computer-generated graphics, scripting, voicing, sound effects and music. And while many production elements are in parallel with the movie-making process, the essential layer of active audience participation brings an added dimension of complexity. As an entertainment medium, as an art form and as a game, each new level of complexity is made possible only by the maturation of computer technology. As a matter of fact, it typically takes a non-trivial amount of sophisticated hardware and software to build a modern game. As a result, providing services and solutions to this industry can be quite daunting to the uninitiated. This paper intends to provide anyone with an interest in the industry to start from very little knowledge, and gain enough experience to be able to understand the language of multiplayer online games, learn how they operate, and understand the important factors to consider when building a multiplayer online game. Moreover, those intrepid literary adventurers who dare trek through the entire paper will be rewarded with an excellent understanding of multiplayer online games. Anatomy of an industry It is widely unknown and sometimes difficult to comprehend, but massively multiplayer online games (MMOGs) are one of the most technologically complex projects currently undertaken by mankind. Games like the popular EverQuest and World of Warcraft easily have as many parts and components as a space shuttle. As such, a game launch in the marketplace can be as spectacular as a Space Shuttle launch. It takes hundreds of people to develop a multiplayer online game, dozens of people to manufacture, advertise, market, distribute,

10 The bigger picture 12 A high-level look at the games 14 Dividing up the globe, online game style 19 Economic factors in hosting a game 22 Dealing with a dud game hosting company 24 Industry direction 25 Conclusion

The Interactive Entertainment Industry Page 3

Highlights Not only does it take an enormous amount of people to launch a game, but it also requires several distinct business models to be in play at any given time. Developers make all of the critical decisions regarding what technology will go into the development of a game, as well as the infrastructure on which the online game will run.

and stock games in retail establishments, and hundreds more around the world to configure, install, maintain, and support it. Not only does it take an enormous amount of people to launch a game, but it also requires several distinct business models to be in play at any given time.
Publishers

This business model consists of those who actually make a game happen. Publishers produce the CDs, assemble and print collateral, assemble and box the game, manage relationships with retailers and various methods of distribution, and above all collect and distribute revenue from game sales. In the tradition of Hollywood movie studios, publishers control the sales channels, along with the means of getting the game out to consumers. In this role, publishers play a large part in the economics of the industry, funding most of the games that are produced. The online game industry was an early adopter of non-traditional means of publishing. For example, ID software is widely hailed as popularizing the (pre-existing) shareware software distribution mechanism with releases of Castle Wolfenstein and Doom. And today companies like Game Tap are trying to do the same thing with the software rental model.
Developers

When it comes to who actually creates the games, developers focus on the creative and technological aspects of games. Sometimes referred to as studios, developers have gained a reputation of actively shunning the economic side of the industry (often only referred to using the derogatory term “the suits”). Nonetheless, developer organizations are tasked with making all of the critical decisions regarding what technology will go into the development of a game, as well as the infrastructure on which the multiplayer online game will run.
Operators

If a multiplayer online game is developed in Korea, who will run it and collect revenue in Germany, or any other country, for that matter? This is the job of the game operator. This particular business model is nearly unique for the multiplayer online games industry. An operator in the multiplayer online game industry is a company that “hosts” and “runs” the online component of a game on behalf of the publisher or developer. Often operators are owned by or have strong associations with publishers. Operators primarily provide the traditional co-location hosting infrastructure, but also provide industry and game environment expertise, local billing, support and customer care services in the local language and in the local time zone.

The Interactive Entertainment Industry Page 4

A cyclical industry

In the authors’ estimate, the Interactive Entertainment Industry cycles last between four and five years. These cycles are typically precipitated by shifts in technology with the end initiated by a market flood. These market floods normally occur when a large number of vendors produce copies of the same trend-setting game play. The first example of this crash occurred in 1977, when the then fledgling video game market was flooded with “pong” type clone games1.

This crash coincided with the introduction of the Atari VCS (later renamed 2600) in 1977. In 1977 Atari sold only 250,000 VCS systems. In 1978 550,000 were sold, and in 1979 the Atari 2600 notched the best-selling Christmas present. A year later, Atari shipped more than 2 million units. Between sales of the 2600 console and the licensed content of the game cartridges (the money has always been in the software), Atari reported more than $2 billion U.S. in profits. Sales continued to double each year, and in 1982 Atari shipped 8 million units.2 Unfortunately by 1983, a dearth of poor titles and several other factors caused sales to plummet and the industry to crash.3
1 2 3

http://en.wikipedia.org/wiki/Video_game_crash_of_1977 Atari 2600 history http://en.wikipedia.org/wiki/Atari_2600 http://en.wikipedia.org/wiki/Video_game_crash_of_1983

The Interactive Entertainment Industry Page 5

Highlights How a game is played has a dramatic impact on the infrastructure required to support it.

As of the writing of this paper, we have observed studio closures, massive consolidation in the industry, definite signs of an industry in slump. Yet the end of 2005 saw the launch of the first set of “next generation” consoles. We have also seen online gaming reach subscription levels even exceeding analyst estimates. Mobile gaming has also seen an unprecedented growth rate, both in traditional handheld devices and games on non-traditional devices like phones and kiosks, and even some cars with in-dash computer systems come with games. If the past has taught us anything, the next several years will be an exciting time with an incredible boom in game quantity, and quality, complemented with an equal boom in players. New forms of game play will be invented and Interactive Entertainment’s appeal will broaden. Critical taxonomy of an industry In the movie industry, films are segmented into types, as in action & adventure, drama, romance, horror, or comedy. Games too are classified into many genres such as first-person shooters, puzzle games, casual games, party games, roleplaying games, racing games, and simulations. But unlike movies, the game industry has added a new dimension to its genre classifications by classifying how a game is played and whether it can be played with more than one player. It is important to know which type of game is being discussed. How a game is played has a dramatic impact on the infrastructure required to support it. In the following section we will explore game types.
Single player games

The most familiar type of electronic game is the single player. We are all familiar with the varieties of this type of game. It is you against the computer or against the odds. From a technical perspective creating games is a deeply challenging task, but from a business perspective, creating a game in this genre is just like creating any other consumer product, and the business does not incur any product development or operations costs after the product has shipped. Unlike online game developers, the only infrastructure required is related to normal business operations.
Multi User Dungeon, Multi User Dimension, Multi User Domain

Multi user dungeon, dimension or domain (MUD) games are typically textbased, multi-user, role playing games. These types of games allow multiple people to log on to a central server and share a generally text-based adventure.

The Interactive Entertainment Industry Page 6

Highlights MUDs require very little infrastructure compared to other forms of online play such as multiplayer online games and MMOGs. The infrastructure required for multiplayer online games is not as complex as MMOGs. From an infrastructure perspective, MMOGs and persistent worlds are identical.

These games date back to the time of serial consoles, mainframes, and minicomputers. MUDs pre-date not only PC games, but the PC itself. According to the Wikipedia article on MUDs, the first one started in 19774. From an infrastructure perspective, a MUD requires very little when compared to other forms of online play such as multiplayer online games and MMOGs.
Multiplayer online games

Multiplayer online games cover a broad array of games that support single to dozens of players. They are targeted more at casual games, where play is limited to teams of up to 32 players. The types of games here are anything from team-based first person shooter games like Half Life and Unreal Tournament, to online versions of more traditional parlor games like poker, chess, and checkers. It stands to reason that the infrastructure required for these types of games is not as complex as massively multiplayer online games.
Massively multiplayer online games

MMOGs include household names like EverQuest, World of Warcraft, EVE Online, Guild Wars and Lineage II to name a few. These games get deployed globally and support millions of players. This type of game requires the most complex of infrastructures, and infrastructures that are replicated and deployed within datacenters spread around the world. Another term often used by the development community to describe MMOGs is the term “persistent world.” From an infrastructure perspective, they are identical.

Figure 1 Touting itself as the “world’s largest game universe,” EVE Online is an MMOG set in a science fiction based persistent world. Unlike most MMOGs that split a large player base up among small parallel worlds or shards of the same game world, EVE is unique in that all of its players inhabit the same game world.
4

Optimized grid applications: http://www.ibm.com/developerworks/library/gr-opgrid/

The Interactive Entertainment Industry Page 7

Persistent worlds
A persistent world is a simulation that keeps running whether players are logged in or not. The primary difference between persistent worlds and MMOGS is the audience. Persistent worlds tend to target smaller audiences and have non-traditional customer sets or business models. Second Life from Linden Labs is an excellent example of a persistent world that is not considered an MMOG. Persistent worlds are often the appropriate classification for simulation and gaming technology used outside the entertainment industry, such as weather forecasts, battle simulations, global climate models and oceanographic simulation.

Figure 2 Brazil-based Hoplon Infotainment’s MMOG, TaikoDom, broke the traditional mold of having games exist in one or more shard architectures (the term reflecting how the game universe is broken into parallel worlds with players assigned depending on geography) by recently adopting a more modular structure. Now Hoplon’s game platform places all users in a single shard or game world, but the structure is modular, with dedicated software modules handling specific functions (such as physics modeling) for all players.

Massively social games
One of the best things about playing in MMOGs is the option to interact with large numbers of real people in a variety of collaborative and competitive situations. This has attracted people to online games from vast geographic and socioeconomic spectrums. Since a person is represented in the game by an avatar that is socially, geographically, and economically equal to all other players (at least it starts out that way), games have a democratizing effect on the communities that play them. This effect enables interactions between people that are often not physically possible due to geographic distance or any other of a number of factors. A relatively new subcategory of MMOGs, termed “massively social games,” has arisen to leverage the ability of games, comprising of global communities. Players actually form social networks, encouraged by game devices such as traveling virtual pets that carry messengers from desktop to

The Interactive Entertainment Industry Page 8

desktop or virtual 3D chat worlds. These games users get to decide the look and feel, and what will happen, and can control access to their world in ways that are not possible in the physical world.

Figure 3 GoPets is an example of a relatively new subcategory of MMOGs in which players normally create and customize their own content and easily interact with others throughout the globe. Comparably low system requirements and an easy-to-use interface make massively social games appealing to a large audience and fun for all ages.

Platforms

The game platform is the end user access device to the game, also technically known as the client side. The major game platforms are console, PC, handheld and mobile devices. Another platform that is typically overlooked is arcade. In the United States, the arcade industry has never matched its heyday in the late 1970s and early 1980s. At the same time, arcades still remain very popular in Japan and greater Asia. Recently, handheld and mobile devices have steadily become serious business. Mostly popular in Asia and Europe, these platforms are finally catching the eyes of the big industry players as indicated by recent acquisitions. This paper however, will focus on the two most popular game platforms—the console and the PC.
Game console

By now most people, and especially those with kids at any point in the last 25 years, possess intimate knowledge of this primary accessory for the television. In the early 1980s there was Atari and Intellivision. The latter half of the same decade brought us Nintendo and Sega. Today PlayStation leads the market followed closely by Xbox® and Nintendo. While PlayStation and Xbox target a similar demographic, Nintendo’s audience tends to be younger and the company’s Game Cube, Game Boy, and Nintendo DS hardware lines remain profitable and globally popular in that segment.

The Interactive Entertainment Industry Page 9

Highlights To date, all current and nextgeneration game consoles have the ability to connect to the Internet over broadband connections.

Console game systems represent a huge segment of the gaming market within the U.S., Japan and parts of Europe. An ever-evolving market, all current and next generation consoles to-date have the ability to connect over broadband connections. Yet this segment of console play has yet to prove its economic viability. Ever since the Nintendo Entertainment System first hit the market more than a decade ago, many people have repeatedly predicted the demise of PC game play. What these people fail to understand is the economics of writing software. As long as PCs are a commodity, there will be an economic draw to develop games for them.
PC

Every year there seems to be an article that predicts, or laments, the demise of PC gaming. In the authors’ humble opinion, PC gaming is not going away any time soon—for many, many reasons. For example, PC games have the lowest barrier to entry for developers. PCs also are arguably the most popular game platform in the whole of Asia. In addition, network-based game play was popularized on PCs. The open nature of PCs allow for companies to experiment with new and innovative hardware and software. Because PCs have fundamental utility outside of game play, there will always be a very large installed base of them, which in turn creates a market. Technologies developed for the PC seem to continually find use within PC games, starting with the mouse, and color graphics PC-based technologies such as 3-D acceleration, multichannel audio, networking support, Web-based games, audio input, realtime text and voice chat, and even multithreading have been adopted by the PC game’s adventurous developers to enhance the play experience. There is no indication that this dynamic will change any time soon, so we expect PCs to be a globally popular platform for all types of games for the foreseeable future. From an infrastructure perspective, it used to be the case that you could build an online game server infrastructure that would support any online-capable client platform. With the recently introduced console platforms, that no longer seems to be the case. This new trend introduces an additional layer of complexity for game developers, which is beyond the scope if this paper.

The Interactive Entertainment Industry Page 10

Highlights Technically speaking, publishers don’t actually produce the game’s content; however, nearly all major publishers today own studios that do. Turning a playable game into a hit takes passion, determination, finesse, experience, innovation and luck.

The bigger picture
A high-level look at the companies

So now that we have taken a brief look at the industry players and have a good handle on the industry as a whole, we take a closer look at both the business models and the technology that make up the Interactive Entertainment industry. It should be noted that any intrepid student completing the following section should be ready for their first trial at the E3 (Electronic Entertainment Exposition) trade show.
Publisher

As mentioned previously, in Interactive Entertainment, the publisher acts as the industry’s primary business engine. As such, publishers consist of fullscale sales and marketing departments, manufacturing (often referred to as production), operations, as well as traditional IT departments. Likewise, publishers face many of the same business issues as print publishers, or any other company whose business is to market and distribute consumer products. Even so, interacting with a publisher is different from interacting with a traditional business. Fundamentally, a publisher’s success or failure is dependent on the quality of the technology it publishes. For this reason, technology decisions are often made by the chief technology officer (CTO) and/or vice president of engineering, with strong input from the organization’s chief creative officer (CCO). Technically speaking, publishers do not actually produce the game’s content; however nearly all major publishers today own studios that do. Like other parts of the entertainment industry, games companies will often develop budgets and purchase content on a project-byproject basis. These games projects normally take 18 to 36 months to develop, with budgets in the millions of dollars. Much of the development hardware infrastructure takes place on high-end PC graphic workstations.
Development studio

A development studio’s primary concern is producing compelling content. Much of that effort is artistic in nature. It takes sophisticated processes and a tremendous amount of technology to turn creative vision into a playable game. Turning a playable game into a hit takes passion, determination, finesse, experience, innovation and luck. From a business perspective this is very risky. There have been volumes of materials published in print and on the Internet about how risk aversion often leads to creative suppression and failure in the games industry, as well as the cinematic and live-action

The Interactive Entertainment Industry Page 11

Highlights When it comes to making infrastructure-related decisions, it is the CTO who owns the majority of that responsibility. Total cost of ownership, environmental issues and cost of management is of utmost concern to online operators, greatly impacting their profit margin.

entertainment industries. Balancing the need to reduce risk with the industry’s drive to innovate is the only path to success. The games industry is starting to standardize on a method for conducting this balancing act. In fact, several years ago, the Interactive Entertainment industry began to adopt a development methodology that introduces new innovations into a stable game in very small increments. As a result, the industry seems to be standardizing on a two-week development iteration timetable. That is, innovations are created, introduced into the game, tested and declared stable in about two weeks. Cycles that are shorter than two weeks have been found too unstable for both engineers and artists to be productive. Conversely anything longer than two weeks starts to introduce non-trivial (and sometimes fatal) schedule and business risk. Within a game development studio, there are two primary teams—creative and technical—required to work together to strike the delicate innovation/risk balance. Each team is led and managed by executives and production, with major decisions being made by the CTO and CCO. When it comes to making infrastructure-related decisions, it is the CTO who owns the majority of that responsibility.
Online operator

For multiplayer online games, it is all about the operations. The role of the online operator is to provide a combination of ISP, co-location provider, and managed service provider services. Specifically, these services may include payment processing, customer support and content distribution. Many times, publishers will even own an operator organization. For online operator organizations, total cost of ownership, environmental issues and cost of management is of utmost concern, greatly impacting their profit margin. Unfortunately for online operators, the technology they use is almost always dictated by the developer. An operator’s primary business goal is to minimize the operating expense of a multiplayer online game, while maximizing its performance, availability and support. This may seem like an impossible situation, but with experienced staff and the careful selection of optimal technology, multiplayer online game operations have been found to be quite profitable in all geographies with Internet access. The economics of running an online operation are covered later in this paper.

The Interactive Entertainment Industry Page 12

Highlights One fundamental difference between the game World of Warcraft and Ford Motor Company’s ERP application is the performance requirements of the transactions.

A high-level look at the games In the following section, we will delve deeper within the inner workings of two major game types. These game types deserve an in-depth treatment due to the complexity of their infrastructure. This is not to detract from the complexity of a single player game, but bluntly speaking, the DVD in which a single player game is recorded and shipped fully encapsulates its complexity. In other words, when a single player game is shipped, it is static. It does not technically change, and the business does not need to do anything to enable players to continue to play the game. On the other hand, multiplayer online games just get started when the game is finally shipped on CDs. The reason is that multiplayer online games involve not only the player’s system, but also the network, servers, storage, customer care, and billing systems required to generate revenue from an online application. All of these systems must be maintained, updated, expanded and contracted as long as players are playing the game.
Interactive Entertainment is not IT: The key difference…

One fundamental difference between the game World of Warcraft and Ford Motor Company’s ERP application is the performance requirements of the transactions. In traditional database-centric systems processing, the ability to move volumes of transactions is critical. In all games, the latency with which you can process transactions is critical. This is a subtle, but critical differentiator between multiplayer online games and traditional IT systems. In a game, it does not matter how many transactions you can process in a minute. What does matter is how fast your round-trip turnaround time is for those transactions.
Multiplayer online game

In a word, the primary difference between an MMOG and a multiplayer online game is “persistence.” What is meant by this is there is no persistent state carried from game to game. Another way to look at the distinction between a multiplayer online game and persistent world is that a multiplayer online game stops playing when the players leave. A persistent world is a simulation that keeps running whether human players are interacting with the world or not. Multiplayer online games started out as simple electronic versions of traditional card and board games such as chess, checkers and solitaire. These games are relatively small with greater limits to the number of players involved at one time.

The Interactive Entertainment Industry Page 13

Highlights Without the need to maintain a large game world state, and without the need to support large numbers of players, the infrastructure required for multiplayer online games is considerably simpler than that of MMOGs. Some significant technical barriers prevent players of a global game from playing a single game—namely the speed of light and the speed of electron travel in copper.

Online versions of solitaire and other casual games usually have one player, while chess, checkers and the like have two players. Interestingly enough, first person shooter games also fall into this category, typically having fewer than 32 players. Without the need to maintain a large game world state, and without the need to support large numbers of players, the infrastructure required for multiplayer online games is considerably simpler than that of MMOGs. From a technology standpoint, a typical commodity 1U server can host several instances of even the most demanding first person shooter. However, even with their simplicity, back-office functions of multiplayer online games are just as complex as that of the MMOG. The reason for this is because multiplayer online games still want to reach the same number of players in total as an MMOG. This number drives the sizing requirements for registration, community, customer support, billing, and so on.
Massively multiplayer online games

Every “thing” in the game is a pile of data. Every rock, spell, item, action, and ability in the game is a pile of data recorded in a database.

It is amazing to wonder how four million people from across the globe can play a single game at the same time. This is a common misconception, but the reality is (except for a few notable exceptions) all those millions of players are not playing a single game at the same time. It needs to be understood that there are some significant technical barriers that prevent players of a global game from playing a single game—namely the speed of light and the speed of electron travel in copper. Most multiplayer online games strive to deliver a frame rate of greater than 60 Hz. That means every player action has to be sent and received in less than 1/60th of a second or 1/120th of a second one way. These barriers make it nearly impossible for a player in Beijing to play with or against a player in New York. Then there is the question as to what an MMOG or persistent world actually is. Technically speaking, these games are much more than just a rich client that provides the immersive experience and high-performance graphical user interface (GUI). As it turns out, most games are a combination of huge real-time distributed simulation engines sitting on top of a very mundane high-performance database. Simply put, every “thing” in the game is a pile of data. Every rock, spell, item, action, and ability is a record in a database somewhere. So in essence, the game is just a very interesting way for thousands of people to run very simple queries against some very boring data.

The Interactive Entertainment Industry Page 14

It stands to reason that MMOGs are truly global applications. Games like World of Warcraft, EVE Online, Lineage, and Guild Wars are played 24 hours a day, every day, around the world. Current commodity networking technology does not provide the level of response time needed to support a workload of thousands or millions of concurrent players moving around a virtual world, so until we figure out how to make Internet Protocol (IP) packets go faster than light, games companies will be forced to figure out how to service a global community. In the meantime, the industry seems to have settled on a relatively standard pattern for addressing this by dividing the problem and global space into smaller manageable units. Stealing some “movie magic” from Hollywood, the game industry usually uses a trick to support global players. This trick is to create multiple copies, also known as instances, of the game world, and host those copies in key areas on the Internet, and in areas of high player concentration. This “trick,” or technique, drives demand for local online operators, and provides players the ability to have a system local to them. Dividing up the globe, online game style In order to manage the complexity of online operations, the industry uses several terms and a standard methodology for breaking down the physical world into manageable-sized chunks.
Globe

To address ever-growing global demand, multiplayer online games take the same approach as with an MMOG. Companies select sites in geographies where it is believed there is sufficient market to warrant the investment. Each global player community is serviced by several strategically located datacenters. Each datacenter may serve a region or country. For example, in Asia you may have server centers in Korea, China, and Hong Kong, whereas in North America you may locate sites on the West and East Coasts (see figure 4, next page).
Site

Multiplayer online games will often have many more sites than an MMOG, and even have a licensing mechanism by which individuals and companies, such as broadband operators, can host instances of their game.

The Interactive Entertainment Industry Page 15

Figure 4 Taking your game global adds an additional layer of complexity, including network requirements and distributed server resources

Each site is designed to support a player community. The size and geographic distribution of the player community is highly variable. If the publisher or developer wishes to have a small number of sites to keep operational costs down, those sites will require hefty infrastructures to support potentially very large user bases. This is often the case for MMOGs and persistent worlds.
Commentary on game world size

Of the two major types of multiplayer online games, there are two different “patterns” used to provide the multiplayer experience. In general, player communities will be split up into copies of a game world or in instances of a battlefield, racetrack and so on. Player segregation is based more on artistic and playability reasons rather than technical ones. The Interactive Entertainment industry does not have a massively interactive literary tradition on which to rely. That is to say, some experiences just are not very fun with 10 million of your fellow humans. The academic way to look at this is there is no other form of entertainment that allows for thousands or even millions of people to concurrently actively participate in a sport or storytelling. Game developers determine the optimal number of concurrent players.

The Interactive Entertainment Industry Page 16

Highlights For multiplayer online games, the number of concurrent players in an instance of the game is primarily determined by the technology used in the engine. The most common approach to support large numbers of players is to create copies of the game world in geographically dispersed locations.

For MMOGs this determination is made arbitrarily. That is, game designers pick a number of players a world will hold based on entertainment and economic considerations. Game developers try very hard to maximize this number in order to reduce infrastructure. But even with years of work, a single game instance can still feel crowded and unworkable at times. For multiplayer online games the number of concurrent players in an instance of the game is primarily determined by the technology used in the engine. Most multiplayer online game engines created for sports, first person shooter, and racing games can adequately handle 32 concurrent players, but many of these games are targeted at teams of eight people or fewer.
Instance

Instance has two uses within the industry. The first use describes an “instance” of an arena, race track or battlefield for multiplayer games. The second use of the term “instance” refers to a portion of an MMOG world, typically a dungeon or maze that is dynamically generated. A big difference between MMOGs and multiplayer online games is that often multiple instances of multiplayer online games can be hosted on a single server, whereas a single copy of the MMOG instance may need hundreds of servers.
Shard

Shard is an industry term used to describe a copy of a massively multiplayer online game world. As described above, an MMOG game world is sized by playability reasons and economic reasons. As of the writing of this paper, the most common approach to support large numbers of players is to create copies of the game world. Different companies use different terms to describe copies of the game, such as worlds, realms, and shards, but they all tend to mean the same thing. Each copy of the game world will support only “X” number of players. If you have 10,000 players in your world and a single shard can only support 5,000 users effectively, then you need to two shards to support your entire player community. As soon as you can adequately predict that your number of players is going increase over 10,000, you run out and buy hardware, acquire new software licenses, and build up your next shard. This process continues until the player base reaches its peak. This approach has existed in games for quite some time. It is an approach that reduces capital risk early in a game’s life, and is a growth strategy that is easy to articulate to senior management. It seems that the industry has taken this approach as gospel and not given it much mainstream challenge.

The Interactive Entertainment Industry Page 17

Highlights

The farther away from the world instance players are, the more latency players will experience; those players will be at a competitive disadvantage, as well as perceive the game play as generally poor.

It has always been technically possible to create an MMOG that is supported by a single-world instance. The leading reason for creating multiple game worlds is that it is nearly impossible for a game studio to create the massive amount of content that would need to exist in order for a single world to be entertaining to, say, 100,000 concurrent players. There are many other reasons for multiple shards, and they seem to fall into two categories: playability and technical. The playability reasons for multiple shards often center on the fact that most games have the collection of scarce resources, like vendors or quest items as key playability elements. If there is only one copy of the game world, this scarcity could dramatically reduce the enjoyment of a game. Imagine if all 100,000 players in a world were competing for the same two items. Playability can be adversely affected when game world areas of high interest are overcrowded or even if 10% of them where in the same city—talk about overcrowding! The technical reasons against single game worlds are uninterrupted scalability and latency. The initial infrastructure required to support a single-world instance would indeed need to be larger than any one shard, and the singleworld instance would need to be designed so that it could seamlessly expand and contract to match player demand, without downtime. This is a non trivial game design feature that is generally not supported by commercial middleware. The second technical reason against a single-world instance is the speed of Internet communications. That is, if there is only one copy of the game world it will have to be physically located somewhere on the planet, and that somewhere will inevitably be a long way from some percentage of your customer base. The farther away from the world instance the players are, the more latency players will experience, and therefore those players will be at a competitive disadvantage, as well as perceive the game play as generally poor. So what are the down sides to this shard approach to MMOG scalability? Just like there are playability and technical upsides to multiple shards, there are several well-known playability and technical downsides to this approach. On the playability side, the biggest complaint about multiple shards often comes from first-time players. Often a new player to an MMOG is joining because they want to play the game with their friends. This unfortunately requires pre-knowledge, that is, you have to know which copy of the game world your friends are playing on even before you start. This very often leads to potential players saying they cannot find friends and giving up on the game. Also a player’s playability gained from social experience is limited by the multiple-shards model. That is, once a

The Interactive Entertainment Industry Page 18

Highlights Once a player is locked into a particular copy of the game world, their in-game social experience is strictly limited to the players on that particular game server. Allowing players to add content to the game brings up all sorts of issues, including issues of game governance.

player is locked into a particular copy of the game world, their in-game social experience is strictly limited to the players on that particular game server. In addition, a game’s virtual economy is restricted by limiting the number of buyers and sellers active in each instance of the game world. This can increase the frustration level of players of a game because availability of goods is decreased, forcing price inflation within each instance of the game world, so players cannot find the items they need, and players also have a harder time selling their goods. There are some technical drawbacks to multiple game worlds as well. Each game world requires its own hardware infrastructure. Often each copy of the game world runs on independent hardware. In each copy of the game world, not all hardware components are fully utilized. Yet the game developer and operator still pay the full price for these expenditures. Maintenance is also an issue. Each copy of the game world must be maintained individually, increasing the workload on the game operations staff. A few notable companies have created single-instance MMOG games, and to date their experiences have been positive. It seems there are some emerging resolutions to the playability and technical roadblocks. First, the huge issue of content creation for single-instance MMOGs must be solved, and playercreated content may be a solution. Player-created content is not a new concept in online games, but the ability for players to extend the game world removes this largest barrier to single-world online games. Allowing players to add to the game brings up all sorts of issues, including issues of game governance. It has been demonstrated repeatedly, both inside and outside the games industry, that interest communities can dramatically broaden the appeal base of a game, as well as extend its revenue generating lifetime. Other playability issues can be addressed as well. The resource scarcity issue can be solved by creative application of game play, such as a quest system that manages quest resources based on the number of active players requiring those resources. Vendor and area crowding can be avoided via creative approaches to trading. “Virtual” vendors and quest-giver NPCs are just an example. One other nasty side effect of shard-based scalability is that it tends to punish the player segment that generates the most revenue: long-time and loyal players. A game design that locks a player’s character to a particular shard prevents those players from benefiting from new infrastructure investments. These players can

The Interactive Entertainment Industry Page 19

Highlights There are very well-documented techniques that enable a software system to expand and contract dynamically, such as optimized grid applications.

get discouraged to the point where they cancel their subscriptions. Players who have been loyal to a shard-locked game have invested time and effort in their characters and will have a disincentive to “start over” on other shards. Technical issues plaguing single-world games can now also be creatively addressed. There are very well documented techniques that enable a software system to expand and contract dynamically, such as optimized grid applications. These technologies would allow the technical infrastructure required to support a game to expand without disruption to the existing game. This leaves us with one last technical barrier to single-world implementation: latency, or the effect a player’s physical location has on game play. Fortunately, other industries that rely on real-time systems have had to face and solve this problem. A geographically distributed financial trading desk is an excellent example of a system that has to respond to user interaction in real time. One of many techniques is the employment of distributed coherent cache, but this is just one of many others. The benefits of a single instance online game to the developer are a dramatically reduced cost of development, deployment and operations. The benefits to the players are dramatically expanded playability. The technology exists. The business models exist. There are no real barriers standing between you and your first single-shard MMOG. Economic factors in hosting a game Operating an online game is a highly complex endeavor with much of that complexity coming from the balance between economics and the need to provide a high-performing infrastructure.
A brief introduction to hosting economics

Why is hosting an MMOG so expensive? Well, cramming a bunch of computers into a closet and wiring them up may not sound expensive, but it is. In actuality, the technology in a datacenter is in and of itself admittedly simple. However, the challenge revolves around the fact that a datacenter is where computing, real-world physics, and economics all come together. These facilities are highly secure buildings, housing highly specialized rooms that require immense electrical and cooling capacity, as well as custom fire-suppression systems and a whole host of other features. Without going into
5

http://www.ibm.com/developerworks/library/gr-opgrid/

The Interactive Entertainment Industry Page 20

Highlights A hosting-location provider’s costs can be broken down into rent, power, network, cooling and miscellaneous expenses.

too much detail, a hosting-location provider’s costs can be broken down into rent, power, network, cooling and miscellaneous expenses.
Rent

As you might expect, the room that houses an online masterpiece is a special one. The physical infrastructure for housing computers or machine rooms is often much more complicated and involved than those housing mere humans. Many times, these machine rooms come fully equipped with raised floors to prevent flooding, overhead cable trays for cable management, as well as underfloor cable management, plumbing for water cooling, and other environmental controls such as a chemical fire-suppression system, because you do not want to spray water on electronics. These rooms are also equipped with high physical security centralized environmental monitoring and management. All of this, like any bit of real estate, is obtained on a long-term lease, and the terms of the lease are divided by the number of square feet in the facility.
Power

In the past, hosting facilities issued a monthly bill based on the kilowatt hours (KWH) consumed by the game’s equipment. Today, in some locations, hosters cannot charge their customers for electrical power by the KWH (as if they were an electrical utility). Hosters have to find another way to recoup electrical costs. Like many creative ventures, hosters often turn this potential operational liability into a revenue-generating mechanism. In other words, many colocation providers see power as an opportunity to add to their margins by charging for power at a profit. For these, and many other reasons, the monthly power fee will often vary by the type of power connection rather than by the variable KWH. On the surface this might look like a good deal to the customer— a consistent, predictable cost. Unfortunately, this model eliminates the ability to reap any real benefit from having a modern, power-efficient system design. Even if your system is super-efficient, you still pay the flat rate.
UPS

Short for uninterruptible power source, UPS systems serve as critical redundant systems in case of power outages. Most of the larger units use batteries, flywheels, diesel generators or a combination of the three, acting as a backup power source during power quality or utility disturbances. Some of the more recent models dramatically reduce space requirements, temperature restrictions and maintenance costs, while others remain huge hunks of equipment that have not changed much over the past 25 years. At any rate, UPS

The Interactive Entertainment Industry Page 21

is absolutely essential in datacenter environments to ensuring system reliability and operational integrity.
Cabling PDUs

A power distribution unit or PDU is nothing more than a glorified power strip. Designed to compliment UPS systems, advanced cabling PDUs can distribute power in high-density rack environments or anywhere conditioned power must be distributed to multiple pieces of equipment. From a single power input, power can be managed and distributed normally up to 12 receptacles. For example, in the U.S., most states require licensed bonded electrical installation professionals to install anything more capable than the lowly power strip, and while installation fees can be expensive, this task is better left to an expert.
Networking

Network bandwidth is likely the most costly component of hosting a multiplayer online game. The bandwidth required for a typical game far exceeds that of even your largest Web sites, and as everyone in the game industry is well aware, it is not necessarily the size of the pipe but the average ping time (a measure of latency) that makes or breaks a multiplayer online game experience. While rent is merely a fact of life, it also holds a bizarre link to ping time. That is, facilities within close proximity to an Internet backbone center tend to have higher rent rates than those farther away and without good ping times.
Cooling

The efficacy and efficiency of cooling a room full of modern, high-density systems is an important concern. Most datacenters today were built at the height of the dot-com gold rush when a CPU called Pentium® was the new kid on the block. At that time, HVAC or “roofing” was relatively inexpensive. Likewise, many analysts thought 1 GHz CPUs were not physically possible, let alone practical for all but the most esoteric of situations. Fast-forward 10 or 15 years and we commonly have 42 dual-processor systems running at 4 GHz in a single rack, pumping out as many BTUs as a blow dryer. By the same token, it takes literally tons of cooling equipment to cool even the modest machine room. The more heat there is to move, the more cooling tonnage is needed to do the job. Back in 1990, most datacenter roofs were designed with the notion that singleprocessor 2u computers would come down the road eventually. Now three racks

The Interactive Entertainment Industry Page 22

Highlights Technology that minimizes floor space, cooling, and power needs will pay off in the long term, and will likely reduce the initial hosting bill. Selecting a hosting provider experienced in multiplayer online game operation seems like a no-brainer, but nothing is as easy as it seems at first.

full of 42 (or 84+) servers each are installed. The lack of foresight has caught up with customers when they are told by co-location facilities to leave an empty rack space between each rack, and pay for special connector equipment and increased installation fees to compensate for rising demands on cooling. The moral of this story is to understand that the cost of hosting a multiplayer online game will likely outweigh the initial capital cost of the equipment, but there are technical decisions that you can make to keep these hosting costs down. Selecting technology that minimizes floor space, cooling, and power requirements will pay off in the long term, and will likely reduce the initial hosting bill. Successful online operations get deployed in multiple hosting facilities in many different regions. Equipment vendors that have direct operations or at least good value-added reseller operations in all specified target regions should be selected, making sure to get the names of 7x24x365 service providers in every region before buying anything. Selecting a hosting provider experienced in multiplayer online game operation seems like a no-brainer, but nothing is as easy as it seems at first. Dealing with a dud game hosting company All too often, online game operations managers have the frequent task of managing their hosting provider company, when it is realized that they cannot live up to service agreements. Unfortunately, these facts are usually discovered during the first few days of a game’s launch, when activity is at its peak. It is usually not a lack of doing up-front research, negotiating a good deal or forging a good interpersonal relationship. These contracts are extensive and drafted to cover all possible contingencies, but anyone who has ever signed an operations contract knows that when networks fail, players are very vocal to the media about games being down, and contracts are not of much help. If the hosting company’s Internet service provider is offline, so is the game. For a game operations manager, the first thought is not to call attorneys for contract enforcement. The first order of business is a call placed to the provider’s technical team in trying to get network back up. Problem resolution is the first order of business, because in reality contracts are nearly useless when it comes to day-to-day operation of a multiplayer online game. Once the issue has been resolved, and a way has been figured out to appease the player community, then attention can turn to negotiated remediation for the outage.

The Interactive Entertainment Industry Page 23

Highlights Problem resolution is the first order of business, because in reality contracts are nearly useless when it comes to day-to-day operation of a multiplayer online game.

Outage aftermath—the resolution

In a difficult situation in which a contract is involved, here are some tips to help get out of immediate trouble more quickly—and help determine whether a long-term relationship should be rebuilt or terminated: 1. Clearly communicate issues Issues should be stated firmly and clearly with quick resolution of the problem demanded, but without going overboard. While it can be difficult to keep cool under certain circumstances, it must be remembered that the hoster is still a partner and as a partner it’s important to keep a good working relationship—at least until things can be moved to another facility. Even if change is imminent, work will still remain during any transition to another hosting situation. 2. Remain calm and professional Staying calm and professional—yet firm—is the best policy. Throwing a temper tantrum might work, it might feel good, and it will get the company’s attention, but screaming will not get immediate issues resolved any faster. It will likely have the opposite effect and is not a helpful tactic in the continuing management of an existing issue. 3. Postmortem Once the issue is resolved, it’s helpful to conduct an active postmortem. This is very important and will lead to improved relations and reduced repeat events. At this point it’s helpful to walk through the sales process with the client representative and ask all of the questions that are initially asked during the sales process, making sure that the same value proposition offered by the hosting company still exists. An onsite visit helps determine whether the same network, power and cooling arrangements are in place and model changes and upgrades have recently taken place. It’s also important to review the company’s security procedures again. Having talked to some of the operations staff to gauge their talent, confidence and morale, changes should be noted to identify any unforeseen impact on operations. Even subtle changes in staff,

The Interactive Entertainment Industry Page 24

Highlights If a hosting company is not living up to expectations, it’s crucial to find the causes and work with the hosting company before another crushing incident occurs. It’s important that companies increase the number of players they can support in MMOGs, but doing so requires ever more complex and expensive infrastructure.

morale or equipment can have negative consequences. If a hosting company is not living up to expectations, it’s crucial to find the causes and work with the hosting company before another crushing incident occurs. 4. Renegotiate It’s a good idea to use this opportunity as an advantage and negotiate a better deal. Issues in the existing contract that need to be addressed, such as a lapse of service, should be discussed at this time. Even if the contract specifies some type of remediation for service failure, leverage can be applied. Reasonable demands can be made, allowing the hosting company to show the depth of their customer commitment; however, it’s important to back them up with functional and price comparisons to similar companies.

Industry direction In order to grow in the Interactive Entertainment industry, companies need to increase the number of players they can support in MMOGs. This in turn requires ever more complex and expensive infrastructure. If this statement is true, you may wonder whether the business model of offering free online play, supported solely by box revenue, is viable. This is an excellent question that deserves a concrete and clear answer, but also begs elaboration. The simple answer is no. A business model that relies solely on revenue generated from the one-time sale of the title is not a viable business model for next-generation multiplayer online games. MMOGs represent ongoing costs after title release. Infrastructure costs will be ongoing and continue often even after the title is no longer on store shelves. But this does not necessarily mean that all online games will have to charge monthly subscription fees. For many types of games and many types of game play, monthly reoccurring subscription charges are not viable options for the target market. The industry is experimenting on many creative alternative revenue sources and garnering some measure of success. In-game advertising is an excellent example of this.

The Interactive Entertainment Industry Page 25

Highlights Many creative revenue sources are being experimented on within the industry, as alternative to the monthly reoccurring subscription charges.

There are also creative ways of building a multiplayer online game such that it does not need a centralized system. Another way to offset the ongoing infrastructure costs is to begin partnering with broadband providers, who provide the server-side code that broadband providers can place on their networks. Even more ways to generate and add revenue include micro payments for item trades, mission/content updates, product licenses, professional competitions, in-game events, as well as game-related magazines and Web sites. All of these constitute creative ways in which to add additional and recurring revenue channels that can add substantial value to the traditional retail revenue channel. Conclusion This paper has taken the reader from an introduction to an industry, through description of its core elements, to speculation on where the industry is going. The intent of this paper is to give readers an introduction to an industry that is very open from a deep technical perspective, but not so much in other areas. For game development, a quick Google search will find dozens of Web sites that discuss the technical aspects of game programming, and many universities offer degree programs in art creation for interactive entertainment. The industry as a whole, including its architectures, best practices, methodologies, and operation, could benefit from more sharing of information. It is a sincere wish that this paper has been found to be informative, enlightening, and even the slightest bit entertaining. To learn how IBM® Solutions for Games can help get your MMOG infrastructure sized and optimally equipped, contact your IBM sales representative or visit:
ibm.com/solutions/games ibm.com/solutions/digitalmedia

© Copyright IBM Corporation 2006 IBM Corporation 1133 Westchester Avenue White Plains, NY 10604 U.S.A.

Produced in the United States of America 05-06 All Rights Reserved.

IBM and the IBM logo are registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Pentium is a registered trademark of Intel Corporation or its subsidiaries in the United States, other countries, or both. Xbox is a registered trademark of Microsoft Corporation in the United States, other countries, or both. Other company, product, and service names may be trademarks or service marks of others. References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates. This paper discusses strategy and plans that are subject to change because of IBM business and techical judgments. All statements regarding IBM’s future direction and intent are subject to change or withdrawal without notice, and represents goals and objectives only. All information in this publication is subject to change without notice. All information is provided on an “AS IS” basis, without warranty or any kind. The IBM home page on the Internet can be found at ibm.com

G565-1461-00


								
To top