PKTown A Peer-to-Peer Middleware

Document Sample
PKTown A Peer-to-Peer Middleware Powered By Docstoc
					             PKTown: A Peer-to-Peer Middleware to Support
                 IPTV and Multiplayer Online Games
                   Hai Jin1, Hong Yao1, 2, Xiaofei Liao1, Sirui Yang1, Wei Liu1, Yong Jia1
         Cluster and Grid Computing Lab.1                                           School of Computer Science2
   Huazhong University of Science and Technology                                   China University of Geosciences
              Wuhan, 430074, China                                                     Wuhan, 430074, China

ABSTRACT                                                              which suits for the efforts to lower down the service cost for Mul-
Peer-to-Peer (P2P) can support service for lots of end users with     tiplayer Online Games (MOG) and Massively Multiplayer Online
little hardware investment, which suits for the efforts to lower      Games (MMOG). Some interesting works are reported recently,
down the service cost for Multiplayer Online Games (MOG). In          which focus on the problems when replacing the C/S architecture
this paper, we introduce PKTown, a P2P middleware inserted into       with P2P in MMOG design, including game consistency [14],
Star Craft and the network layer. PKTown captures all packets         cheat prevention [12], etc. These issues are new and further stud-
generated by Star Craft. Application Layer Multicast (ALM) is a       ies are still needed. However, to advocate these issues, new MOG
popular technique adopted by IPTV. We apply ALM to transmit           or MMOG prototypes design are introduced correspondingly.
the broadcast packets through overlay network. PKTown extends         Age of Empires [1], Counter-Strike [2], and Star Craft [7], are
the LAN game experiences to players belong to different LAN.          typical MOGs. To play these MOGs, there are three ways. First,
We design a k-regular random overlay network based on game            players who have the game copy can launch a game and enjoy
specific requirements analysis. No change is made on binary code      together if they are in the same Local Area Network (LAN). How-
of Star Craft. Preliminary test results show that PKTown works        ever, if the assumption is not satisfied, players can not enter the
well and supports other MOG in the same way. Our contribution         same game session even if they are near, or have small latency in
is to provide a new game experience method with little hardware       network communication. Alternatives require hardware invest-
investment.                                                           ment. Second, players can play the game over Internet using the
                                                                      game service provided by the game provider [8], which means
                                                                      game servers deployed by the game provider transmit the game
Categories and Subject Descriptors                                    packets. Third, they can use game services provided by third party,
C.2.4 [Distributed Systems] Distributed applications; H.4.m           commercial game service providers [3] who deploy the transmis-
[INFORMATION SYSTEMS APPLICATIONS]: Miscellane-                       sion servers. The difference between the second and the third way
ous                                                                   is not only who will deploy the hardware, but also some skills in
                                                                      detail. Third party game service providers can be supported by the
General Terms                                                         game provider which means they can act as the second way, or
Management, Design                                                    they have to capture the game packets in network layer and trans-
                                                                      mit the packets in a method they design, out of the control by the
                                                                      original game software. Generally speaking, the third way pro-
Keywords                                                              vides a virtual LAN over Internet.
Multiplayer Online Games, Peer-to-Peer, Overlay, Middleware,
Application Layer Multicast                                           This paper introduces PKTown, a Peer-to-Peer middleware work-
                                                                      ing under the MOG software. PKTown captures all packets gen-
                                                                      erated by games, and transmits them through a P2P overlay net-
1. INTRODUCTION                                                       work, instead of through game servers deployed as transmitters.
Based on Peer-to-Peer (P2P) overlay networks, there are many          Application Layer Multicast (ALM) [26][18] is a popular tech-
researches to provide interesting products, including file sharing    nique adopted by streaming live media. Media data is delivered
[4] [5], streaming media [26], etc. Base on streaming media,          by ALM commonly. We show that ALM can also be applied to
IPTV [9][10] has gained much attention recently. P2P can sup-         achieve the broadcast function in LAN game application. Star
port service for lots of end users with little hardware investment,   Craft is chosen as a representative real MOG to validate the mid-
                                                                      dleware. No change is made on the binary code of the game. The
                                                                      purpose of the middleware is to offer the ability to interconnect
                                                                      players who have small latency with each other, enabling them to
                                                                      play together with little additional hardware investment.
                                                                      The rest of paper is organized as follows. Section 2 introduces
                                                                      related works. Section 3 summarizes missions for PKTown to
                                                                      achieve with analysis of Star Craft LAN game option and services
 IPTV Workshop, International World Wide Web Conference,              provided by third party. Section 4 describes the design of
 May 23, 2006, Edinburgh, Scotland, United Kingdom
PKTown. Section 5 shows the preliminary test results, and section       3.1 Analysis of Game Packets in LAN Game
6 concludes this paper.                                                 To make the description simple, two PCs (named A and B) run-
                                                                        ning Windows XP act as end players, each with a Star Craft copy.
2. RELATED WORKS                                                        Sniffer is chosen as the analysis tool to capture packets generated
                                                                        by Star Craft in the network layer. We start Sniffer first, and
2.1 Replace C/S Architecture with P2P in
                                                                        launch a LAN game in UDP manner when selecting the game
MOG or MMOG Design                                                      option on both PCs.
Ghost [14] studies how to manage game consistency across a set
of players with heterogeneous network resources and a protocol          There are two kinds of packets from our observation.
provides cheat-free for P2P games. DHT and a zoned federation           Broadcast Packets: the IP packets that contain native address (A
model [19] are used to adapt MOG to P2P networks. SimMud [16]           or B) as source address and treat as destination
uses DHT and ALM for message exchange. A low-latency event              address.
ordering protocol, called NEO [12], is designed for MMOG to
prevent protocol level cheats. Time-Prisoners [21], a fully distrib-    Unicast Packets: the IP packets that contain native address A as
uted MMOG, is designed in P2P architecture by JXTA.                     the source address, and B as the destination address, and vice
Similarity in all researches mentioned above is to replace the C/S
architecture with P2P in MOG or MMOG design. When we ana-               The usage of broadcast packets is to enquire who is here, and
lyze game software, game servers always take care of data consis-       announce itself at the same time. The usage of unicast packets is
tency, state updating, etc. The challenge is to shift to a fully dis-   to exchange the detail information needed by the game session
tributed architecture.                                                  next. Based on our analysis, both functions and the usage are very
                                                                        common compared with the method used by ARP (Address Reso-
                                                                        lution Protocol) [23] in network layer of TCP/IP.
2.2 P2P Middleware Support Game Reuse
PeerBooster [15] is the most similar way to PKTown to the best          The observation explains why players can not enjoy together in
of our knowledge. The problem of reusing MOG is that services           LAN game manner if they are not in a same LAN even if they
cannot continue to be provided by game venders as time goes by,         have small latency. The transmission range of broadcast packets is
and there is still need for consumers to play the game whether the      limited to LAN, and no router will transmit this kind of packets
service exist or not. A middleware called PeerBooster works be-         over Internet.
tween network layer and games. One module of PeerBooster
called Message Handling captures packets and transmits them             It is obvious that broadcasting from A and B will stop once the
through P2P overlay, which approximates PKTown.                         unicast packets are received. In other words, only a newcomer
                                                                        will generate broadcast packets. This broadcasting will stop as
However, there are some obvious differences between Peer-               soon as other peers reply unicast packets.
Booster and PKTown. First, the preliminary assumption of Peer-
Booster is the server binaries are released by its game provider
because one of the modules needs to act as server application.
                                                                        3.2 Analysis of Methods Used in Game Ser-
This assumption can not be satisfied by every MOG vender. Sec-          vice Provided by Third Party
ond, a module called Synchronization plays important role. This         We choose Holdfast [3], the largest online game platform, to con-
module is embedded in each peer to keep a global clock using            tinue our analysis. Holdfast offers various hottest games service,
NTP or GPS algorithm. Third, another Zone Management module             including Star Craft, Counter-Strike, War Craft, Age of Empires,
filters the packets. However, when we refer the problem of reus-        etc. The similarity of these games is the ability to allow players to
ing MOG, there is only one zone actually. The zone management           play in same LAN, based on UDP communication.
issue always comes with MMOG. At last, Topology Construction            When a player login the nearest game server, enter a game room
module is reported using algorithms for tree-like ALM. However,         related to one game, a unique temporary reserved address, as
tree-like ALM contains only one data source, topologies opti-           192.168.X.X, is assigned. All players in the same room are allo-
mized like tree may not be appropriate in designing MOG mid-            cated in a virtual LAN. This indicates why we find at most 250
dleware.                                                                players in one room.
For PKTown, there is no need for server source code to be re-           The packets generated by games are captured and reconstructed.
leased as an assumption. The design of PKTown keeps simple,             The real IP addresses are used to communicate between the player
dispenses with considering synchronization, zone management,            and the room server.
etc. Such issues are left to the original games, all of which have
been solved by each MOG in LAN game manner. This is the main            A basic assumption is that any player can find a near room server
difference between the works mentioned above and ours. We start         with small latency with him. So we can understand the deploy-
our design based on the observations introduced in next section.        ment of many game servers working as transmitters in different
We analyze the first way and the third way mentioned in section 1,      3.3 Missions of PKTown
then, we bring forward the missions of PKTown.                          PKTown needs to provide the ability to capture packets generated
                                                                        by Star Craft. Considering the method used in [15] caused delay
                                                                        on every packets for about 60-80 milliseconds without network
transmission, the Hook programming skill can be used. We re-           ity of 1−1/n using only 9 neighbors, and, k-regular graphs exhibit
write Socket APIs to send and receive data. When Star Craft is         the highest resilience. The conditions mentioned above satisfy
launched, process injection can be used to replace the socket func-    design of PKTown quite well. Based on this study, we adopt a k-
tions used by Star Craft. Other software using network in the          regular random overlay construction, with each node is given a
background is not affected.                                            fixed degree 9, the number of neighbors.
We need to build an overlay with latency constraints, which
means players are grouped if they are near. This requirement will                                   Star Craft
cause partition of the overlay network. Nevertheless, we prefer to
divide the whole overlay. If two players have large latency, it is
meaningless to bring them together. There will be several over-         PKTown
lays to accommodate players to different groups. However, when
                                                                                   Game Packets Capturer
we refer single overlay, it must be well connected in a global view,
accompanied with random departure of peers.
The broadcast packets should be transmitted to peers as many as                           Packets Type
possible, and as soon as possible. Players would not stay too long                          Checker
to examine all the available servers.                                        Member
                                                                              Table               Broadcast Packets
The unicast packets should be transmitted between game players                                        Rebuilder
in a same session as quickly as possible.                                                                                   Incoming
                                                                            Overlay                                       Type Checker
4. SYSTEM DESIGN OF PKTOWN                                                 Constructor
4.1 Assumptions of PKTown
The first assumption is that MOG is designed to support LAN
game based on UDP communication. We design PKTown to ex-                                        Packets                Packets
                                                                                                Sender                 Receiver
tend the game experience in the first way mentioned in section 1.
Some old games using IPX are not considered. Almost all MOGs
satisfy this assumption.
Second, reserved network addresses are not assigned to peers.                          Broadcast Packets
There are many peers behind NAT [24]. However, we leave this
issue in the near future.                                                              Unicast Packets
                                                                                       Member Table Maintenance
Third, game ports and other unoccupied ports are available. Two
                                                                                       Broadcast Packets With Tag
more ports are used to reconstruct UDP packets from broadcast
packets to unicast packets. We assume that there are no firewalls                      Overlay Packets
or the communications through the firewalls are allowed.
                                                                             Figure 1. Architecture and Data Flow of PKTown

4.2 Overlay Design                                                     4.2.1 Peers Join
The architecture and data flow of PKTown is shown in Figure 1.         It is still an open question to predict latency among peers without
When broadcast packets are captured, how to deliver them to            direct measure. Unlike GNP [22], Vivaldi [11] or Meridian [25],
other players over the same overlay is equivalent to ALM adopted       we use GID based service scheme. A table containing full correla-
by IPTV. In media data delivery, there is always a single source       tion between IP address and real address all over China is col-
to distribute data, so, spanning tree optimized overlay is ideal.      lected. The correlation table is given to each peer and a unique
However, any peer in our overlay will send out broadcast packets       GID in the global network can be constructed. The GID value of
when they enter the game selection interface, in other words,          an end host is a 128-bits integer encoded by geometrical informa-
looking for other players in same overlay. Additionally, without       tion corresponding to ISPs, cities, campuses, and buildings, re-
resources to share, players have no reason to stay in overlay if       spectively. This method is lack of theoretical proofs, but to our
they leave the game. This causes trouble in overlay connectivity       experience in ALM work [18], this skill is sufficient for initializa-
maintenance.                                                           tion.

These requirements exclude tree like topologies applied in ALM.        A Bootstrapping server keeps information about each overlay
The appropriate choice is mesh. However, mesh like Gnutella 0.4        partition. All peers alive in one overlay are sorted by ascending
[6] follows the power-law [20]. Gnutella 0.6 [5] and KaZaA [4]         order. When a player launches PKTown first, the GID is sent to
even strengthen the power-law distribution. Super peers are sup-       Bootstrap server. One peer is selected as a representative of the
posed to be stable. However, it is hard to find one in our overlay     overlay, which is alive and adjacent to the new comer. One ping-
with extremely high churn rate.                                        pong delay is measured. If the latency is small, the new comer is
                                                                       considered as near to the rest of others. We set the latency restric-
It is reported [17] recently that each user in a system with n=100     tion to 30ms in prototype.
billion peers, 30-minute average lifetime, and 1-minute node-
replacement delay can stay connected to the graph with probabil-
After the latency is affirmed, the new peer sends Join requirement      range in overlay. If the destination address is not broadcast, the
packet to bootstrapping server for a candidate list. A candidate list   packets are sent out without any change.
is constructed with 5 to 8 peers selected randomly within the
                                                                        To extend the transmission range of broadcast packets from LAN
overlay. Join insistently packets are sent to each candidate.
                                                                        to overlay network, an understandable solution is flooding like
                                                                        Gnutella. However, flooding generates too many redundant pack-
4.2.2 Overlay Maintenance                                               ets and wastes bandwidth greatly. We take K-random walk [13]
Each peer holds a 9 by 8 two-dimensional array. The content of          [20] as our choice.
each line is the neighbor at neighbor list. Heartbeat packets are
sent to each neighbor 6 times per minute. If degree is changed, the     When packets are received at the destination peer, packets tagged
change detail will be carried in heartbeat packets.                     with CHANGED will be reconstructed to broadcast packets and
                                                                        sent to game. At the same time, these packets are also recon-
When Join insistently packets are received, any peer gets a full        structed with a random neighbor’s address as the destination ad-
degree will send Cut packet to the neighbor with the highest de-        dress. The original source address, of which a peer sends the
gree, if all degrees are full, select one randomly.                     broadcast packets out, is carried within the new packets. Once a
When broadcast packets are sent through overlay, if any peer            former receives the transmitted broadcast packet and sends back
receives the packets, they will check both of the original sender       unicast packets, original source address will be used as destination
and itself whether degree is full or not. If so, they will send An-     address.
nounce packet to declare their existence. This mechanism pro-
vides an opportunity for a new peer and former make connection.         5. EXPERIMENT AND DISCUSSION
The broadcast packets are generated by game, and we reuse them          5.1 Experiment Environment
for overlay maintenance at the same time.
                                                                        We evaluate the prototype of PKTown in environment illustrated
When Cut packets are received, no operation happens.                    in Figure 2.
Heartbeat packets are sent to Bootstrapping server also, with a
lower frequency.

4.2.3 Peers Leave
If a peer leaves normally, Bye packets are sent to each of the peers
in its member table. Abnormal leave is detected by heartbeat
When one neighbor leaves, if the degree decreases from full to 8,
no operation happens; otherwise, a peer will be randomly selected
from the corresponding neighbor list, and a Join insistently pack-
ets will be sent. The primary packets used in PKTown are summa-
rized in Table 1.
          Table 1. Primary Packets Used in PKTown
  Packets Type        Function                                             Figure 2. Separated LANs as Experiment Environment

  Join insistently    Used for Initialization, or Peer Leaves           There are three LANs separated in Huazhong University of Sci-
                                                                        ence and Technology. All LANs are 100Mbps Ethernet with la-
  Announce            Hope To Add New Connection                        tency less than 5 ms.
  Join                Randomly Select From Peers Announced              Computers are heterogeneous from AMD 2500+ CPU with 1GB
                                                                        RAM to Intel Celeron3 1.1GHz with 256MB RAM. We gather
  Heartbeat           Overlay Maintenance                               together 26 peers in three LANs assigned with IP address in
  Cut                 Join insistently Packets Received                 202.114.14.X, and 202.114.5.X, and 202.114.8.X.

  Bye                 Peers Leave Normally
                                                                        5.2 Experiment Design
                                                                        All other 24 peers have PKTown and Star Craft: Brood War in-
4.3 Management of Packets                                               stalled. Two PCs illustrated in Figure 2, namely A and B, have
All packets generated by Star Craft are captured by Hook. If the        special edition of PKTown to collect trace. Both PCs and boot-
destination address is broadcast, the packets are reconstructed to      strapping server are Intel P4 2.66GHz with 512MB RAM.
overlay multicast by replacing the broadcast address with some of
neighbor addresses. Ports used by game are also replaced by ports       The two special editions of PKTown add native time stamp on
used by PKTown. The original information about source address           every packet when they are captured. A little dissimilarity with
and ports are carried on. Another CHANGED tag is attached               other copies in source code is that A and B are always settled as
which means this packet need to be rebuilt at the other end. A          neighbors. All the packets transmitted between A and B is sent
Time to Live (TTL) value is also added to control the transmission      back, additionally with the original capture time stamp carried on.
                                                                        Other PKTown edition used by peers can read these special pack-
ets, but treat them as normal packets. When these packets are sent                      mitted through overlay. We apply ALM in mesh simulating the
back, a new native time stamp is added on. Without bothering                            broadcast process in LAN. Preliminary test results show that our
clock synchronization, half of the values are estimated as transac-                     design is correct and game experience in LAN is extended suc-
tion latency and communication latency. When we measure the                             cessfully.
bandwidth consumption, this option is turned off.
                                                                                        We leave issues such as packets synchronization to game software,
                                                                                        based on the assumption that we group peers with small latency.
5.3 Preliminary Test Results and Analysis                                               We focus on how to make the use of P2P networks to provide
The latency trace result is shown in Figure 3. When players are                         game service without change the MOG binary code. This differs
well connected, PKTown provides the game experience just like                           with many other works.
in the same LAN. From Figure 3, we can draw the conclusion that
latency consists of transmission delay and thread schedule by                           Future research includes several issues. First, NAT problem will
operating system. The time cost by hook and packets rebuilding                          be solved in the near future; otherwise PKTown can not be attrac-
are ignorable.                                                                          tive in reality. Second, the scalability of PKTown will be studied
                                                                                        more carefully. Third, the Bootstrapping server may cause single
           10                                                                           point of failure, and fully distributed solution like Meridian will
                                                                                        be studied in the near future. We will also study how to support
                   8                                                                    Counter-Strike in the same way.
Delay (ms)

                   6                                                                    7. ACKNOWLEDGEMENTS
                                                                                        This work is supported by National Natural Science Foundation
                   4                                                                    of China under grant No.60433040, and China CNGI projects
                                                                                        under grants No.CNGI-04-12-2A and CNGI-04-12-1D.
                                                                                        8. REFERENCES
                                                                                        [1] Age of Empires.
                       0          100     200      300        400       500       600   [2] Counter-Strike. http://
                                                 Time (s)
                                                                                        [3] Holdfast.
                              Figure 3. Latency Measured in 600 Seconds
                                                                                        [4] KaZaA.
The bandwidth consumption is shown in Figure 4. From Figure 4,
we find the bandwidth is very low, and most bandwidth is con-                           [5] Gnutella protocol specification 0.6. http://rfc-
sumed by Star Craft. The low bandwidth consumption of Star                    
Craft gives us great confidence in the scalability of PKTown.                           [6] Gnutella protocol specification 0.4. http://rfc-
                                                          • Bandwidth Overall 
                                                          ♦ Bandwidth of Star Craft
                                                                                        [7] Star Craft.
                   2.6                                                                  [8] Star Craft service.
                                                                                        [9] B. Alfonsi, "I Want My IPTV: Internet Protocol Television-
Bandwidth (KB/s)

                   2.2                                                                  Predicted a Winner", IEEE Distributed Systems Online, Vol. 6,
                   2.0                                                                  No. 2, 2005.
                   1.8                                                                  [10] S. Cherry, “The Battle for Broadband [Internet protocol tele-
                   1.6                                                                  vision]”, IEEE Spectrum, vol. 42, issue 1, Jan. 2005.
                                                                                        [11] F. Dabek, R. Cox, F. Kaashoek, and R. Morris, “Vivaldi: A
                   1.2                                                                  Decentralized Network Coordinate System”, Proceedings of SIG-
                   1.0                                                                  COMM, Portland, OR, August 2004.
                                                                                        [12] C. G. Dickey, D. Zappala, V. Lo, and J. Marr, “Low Latency
                         0         100     200      300        400      500       600   and Cheat-proof Event Ordering for Peer-to-Peer Games”, Pro-
                                                 Time (s)                               ceedings of NOSSDAV, Cork, Ireland, June 2004
                             Figure 4. Bandwidth Measured in 600 Seconds                [13] C. Gkantsidis, M. Mihail, and A. Saberi, “Random Walks in
                                                                                        Peer-to-Peer Networks”, Proceedings of INFOCOM, Hong Kong,
6. CONCLUSIONS AND FUTURE WORK                                                          China, March 2004.
In this paper, we introduce a novel method to combine peer-to-                          [14] A. S. John and B. N. Levine, “Supporting P2P Gaming When
peer overlay network with a representative MOG, Star Craft. We                          Players Have Heterogeneous Resources”, Proceedings of NOSS-
build a k-regular overlay to maintain the connectivity with high                        DAV, Stevenson, Washington, USA, June 2005.
churn rate. Broadcast packets act key function to find other play-
ers in different LANs. When captured, they are rebuilt and trans-
[15] Y. Kaneda, H. Takahashi, M. Saito, H. Aida, and H. Tokuda,      [21] M. Merabti and A. El Rhalibi, “Peer-to-Peer Architecture
“A Challenge for Reusing Multiplayer Online Games without            and Protocol for a Massively Multiplayer Online Game”, Pro-
Modifying Binaries”, Proceedings of NetGames, Hawthorne, New         ceedings of GLOBECOM Workshops, Dallas, TX, November
York, USA, October 2005.                                             2004.
[16] B. Knutsson, H. Lu, W. Xu, and B. Hopkins, “Peer-to-peer        [22] T. Ng and H. Zhang, “Predicting Internet Network Distance
support for massively multiplayer games”, Proceedings of INFO-       with Coordinates-Based Approaches”, Proceedings of INFOCOM,
COM, Hong Kong, China, March 2004.                                   New York, NY, June 2002.
[17] D. Leonard, V. Rai, and D. Loguinov, “On Lifetime-Based         [23] D. C. Plummer, “An Ethernet Address Resolution Protocol --
Node Failure and Stochastic Resilience of Decentralized Peer-to-     or -- Converting Network Protocol Addresses to 48.bit Ethernet
Peer Networks”, Proceedings of SIGMETRICS, Banff, Alberta,           Address for Transmission on Ethernet Hardware”, STD 37, RFC
Canada, June 2005.                                                   826, MIT, November 1982.
[18] X. Liao, H. Jin, Y. Liu, L. M. Ni, and D. Deng, “AnySee:        [24] P. Srisuresh and M. Holdrege, “IP Network Address Transla-
Inter-Overlay Optimization based P2P Live Streaming System”,         tor (NAT) Terminology and Considerations”, RFC 2663, 1999.
Proceedings of INFOCOM, Barcelona, Spain. April 2006.
                                                                     [25] B. Wong, A. Slivkins, and E. G. Sirer, “Meridian: A Light-
[19] T. Limura, H. Hazeyama, and Y. Kadobayashi, “Zoned Fed-         weight Network Location Service without Virtual Coordinates”,
eration of Game Servers: a Peer-to-peer Approach to Scalable         Proceedings of SIGCOMM, Philadelphia, Pennsylvania, USA
Multi-player Online Games”, Proceedings of SIGCOMM Work-             August 2005.
shops, Portland, Oregon, USA, August 2004.
                                                                     [26] X. Zhang, J. Liu, B. Li, and T.-S. P. Yum, “CoolStream-
[20] Q. Lv, P. Cao, E. Cohen, K. Li, and S. Shenker, “Search and     ing/DONet: a data-driven overlay network for live media stream-
replication in unstructured peer-to-peer networks”, Proceedings of   ing”, Proceedings of INFOCOM, Miami, FL, USA, March 2005.
International Conference on Supercomputing (ICS), New York
City, NY, USA, June 2002.

Shared By: