Counter - Strike Traffic Analysis with Network Emulation by wuzhenguang


									                 Counter-Strike Traffic Analysis with Network Emulation
                                         ARTHUR DE CASTRO CALLADO1
                                         SUZANA DE FRANÇA DANTAS1
                                RODRIGO DOS SANTOS BACELAR GOUVEIA BARBOSA1
                                        PAULO GONÇALVES DE BARROS1
                                             VERONICA TEICHRIEB1
                                               JUDITH KELNER1
                                         DJAMEL FAWZI HADJ SADOK 1
                        Centro de Informática – Universidade Federal de Pernambuco (UFPE)
                    Caixa Postal 7851, CEP 50732-970, Cidade Universitária, Recife – PE – Brazil
                                  {acc2, sfd, rsbgb, pgb, vt, jk, jamel}

         Network action games face a number of problems which limit their efficiency. It is
         important to ensure that all players are at equal levels during a session, which is
         highly dependent on the information integrity available to a user. This work
         analyses problems such as the impact of delay and packet loss on the playability of
         the Counter-Strike game. An effort is made to relate such parameters to other
         qualitative parameters obtained from the analysis of user game perception.
         Keywords: Models and Infrastructure for network games, Playability, Usability.

     1 Introduction                                      games are based on real-time actions, where
Guaranteeing the well functioning of advanced            players interact with each other and with the
applications, like action games and virtual              environment simultaneously. In this case, there
reality environments over infrastructures as             are many obstacles in the development of a
unreliable as the Internet is still an open problem      game, for one must guarantee that players
and a challenge. Many research groups are                perceive events at the right time and that such
involved on the arduous job of identifying the           guarantee is restricted by the capacity of the
variables and resolve the problems that affect           network connecting the participants’ machines.
these applications.                                      So far, Quality of Service (QoS) metrics are not
                                                         completely defined to comply with players’
     Networked games are the latest sensation in         needs, i.e. which network parameters should be
the world of electronic games. They allow                managed and how they could be controlled to
matches with dozens or sometimes hundreds of             make the game fair to all players. These are are
people interacting with each other on the same
                                                         still open problems.
virtual environment. Nowadays, big companies
will not put money on the development of a                     Many papers that present typical problems
game if it doesn’t have on-line multiplayer              on networked games performance have been
characteristics; because these features can both         published and among them are [12], [1].
increase the number of consumers the game is             However, studies presenting quantitative and
targeted to as well as enlarge its life period.          qualitative analysis are still to be seen. We made
                                                         many tests using the action game Counter-Strike
     The access to the game environment in
multiplayer games can be based on turns, where           (CS) [5], one of the preferred games among
                                                         those who like to improve their terrorist or
each player can take actions to the game at a
time, like in chess. However, most interesting           counter-terrorists tactics. CS is one of the most
                                                         played First Person Shooter (FPS) games
worldwide. The virtual struggle between              same time. This is generally solved with a
terrorists and counter-terrorists has gathered       signaling packet sent to each player at a
thousands of players for the last 4 years.           different time, with a delay inversely
     In this project, we tried to make both a        proportional to the players’ connection delay
quantitative and a qualitative analysis of the       [14].
behavior of the game. The qualitative analysis            The end of the match synchronism makes
was based on a questionnaire answered by             the final state of the match consistent to all
players at the end of each match and on statistics   players. So, each client machine can determine,
of performance for each player, according to the     for example, who won the match [2], [14].
score and the number of deaths. The quantitative          The actions of a player must be reflected in
analysis was based on packet traces gathered         the world1, so they can be seen by all players (all
during the matches, with all the generated           players must perceive the actions in a consistent
traffic.                                             way). To make it possible, games use the lag
     We noticed that problems like packet loss       compensation       technique        in   centralized
and delay do influence the players’ performance      architectures, which consists of updating the
throughout the match. Network bandwidth is           state of the world in the server according to the
another parameter that can also be considered in     states of the clients. Still, this technique creates
the analysis, but it is not being treated here.      some problems like, for example, when there is
     In [7] the Counter-Strike traffic was           inconsistency of the state of a client, it can be
modeled for local networks. This paper intends       seen and shot through a wall [2], [14], [8].
to further characterize Counter-Strike behavior           Collision detection is also another problem
in WAN networks using network emulation. It          and should be consistent to all players. The
will also evaluate user’s satisfaction among each    collision of a car with another car should be
other on different network conditions during a       noticed by the car that hit the other as well as by
game match.                                          the car that was hit and should also be seen by
Section 2 of this paper presents the major           the cars that were passing by, all at the same
problems observed on action games in order to        time. It is fundamental that this information has
work satisfactorily on such an heterogeneous         a high propagation priority [14].
network like the Internet. Section 3 describes the        Another relevant aspect of applications like
methodology used to accomplish the tests and         action games is the discarding of packet. Some
details the network scenarios created. The           of the problems caused by packet loss are:
qualitative and quantitative results are             warping, lead targeting and dead reckoning.
commented in Section 4. Section 5 draws some              Warping is characterized by the loss of
conclusions and shows some suggestions for           packets referring to the position of a player
future works.                                        during the match. Its movement will not be
                                                     continuos, making other players see the player
     2 Networked games vs. Internet: typical         as if “teleporting” himself from one position to
problems                                             another in the scenario. This occurs due to the
Networked games require important network            unpredictability of the movement for long
resources to work efficiently. Synchronism is a      periods of time by the interpolation algorithms
critical aspect and it consists of making all        used [11], [2], [10].
players notice an event at the same time.                 Lead targeting is the “technique” used by
Synchronism is important in many moments of a        some players to aim and hit their opponents in
game and can be categorized according to the         First Person Shooter games when they are not
problem it solves: start of the match, end of the    accessible. This happens because the state of the
match, user actions, collisions, sound of events
and random elements.
     The start of the match synchronism is to                1
                                                               “World” refers to the virtual environment where
make every player see the match starting at the      the match takes place.

    Anais do SBgames, outubro de 2004
world presented to the player is delayed                  Once it was needed to increase the number
according to its network connection. This way,        of clients and to have an isolated network2, a
the player does not see what’s happening to the       Lan House was contacted and their
world at that moment, but what happened a little      infrastructure was used. This choice solved two
time before. The enemy is not where the player        problems: traffic isolation and the participation
sees it, but a little dislocated from that position   of specialized game players, with excellent
[11].                                                 knowledge of its rules and tactics.
    Dead reckoning comes from the necessity of
establishing a common perception of major                 Scenarios description
information in the game, such as start time, end          The test environment was constituted of 25
time, winner(s), who died, etc. [12].                 personal computers (PCs) acting as client
    When we consider games with a very large          machines, 1 PC acting as game server, 2
number of players, network bandwidth becomes          Ethernet hubs operating at 100Mbps, 1 PC
an important issue. The greater the number of         running NIST.Net [13] and another PC runs the
players, more information should be processed         TCPDump network sniffer [16], as shown in
and sent [9], [6], [8], increasing network load.      Figures 1 and 2.
However, depending on the architecture adopted
for the game, this load can be greatly reduced.
Distributed games require more bandwidth.                                                     Scenario 1: LAN
Centralized (client/server) architectures, like the
one used in Counter-Strike, concentrate their                     Client 1

load on a single machine [11], [8]. On that case,
the server machine will need a network link with
greater bandwidth to work properly. If it is not

possible, that link will act as a bottleneck,
decreasing game performance to players [11],                      Client 2

Even though Counter-Strike uses all these
techniques, they merely minimize network                                                          Monitoring
problems, not solving them. Therefore, it is
extremely necessary to evaluate the diversity of
network conditions in order to improve user’s
playability with Counter-Strike. This can be                      Client 25

done by identifying thresholds for acceptable                                Figure 1: LAN scenario
values of network parameters.
                                                          NIST.Net is a network emulation software
    3 Testing environment and methodology             that allowed us to configure network links with
                                                      bandwidth, delay and loss characteristics. We
In this section we describe the environment           could emulate different kinds of connection for
where tests have taken place and the                  each client by configuring the NIST.Net
methodology used to accomplish them. We also          machine as a router and putting all clients in one
describe the criteria adopted to do the objective     network (connected by 1 hub) and the server in
and subjective analysis.                              another network (using another hub). The
    Before the experiments presented on this          Monitoring machine, connected to both
paper take place, other minor scale tests (4          networks, allowed us to capture all traffic from
clients, for example) have been made on the
laboratories of CIn/UFPE. Those tests helped to
devise what types of scenarios were interesting
to be evaluated, their parameters and also the                2
                                                                All CIn/UFPE laboratories are integrated, so it
metrics that should be used to analyze them.          was not possible to isolate a special traffic.
both sides and comparatively analyze it to check                            packet loss in local networks were considered
packet delay and loss.                                                      insignificant.
                                                                                SCENARIO 2: Analyzing packet delay
                                  Scenarios 2 to 4: WAN
                                                                                On this scenario, all links were configured
     Client 1
                                                                            not to have packet loss during transmissions.
                                                                            Two groups (Group 1 and Group 2) received
                                                                            player#1 profile (see Table 1) with delay
                                      NIST.Net                              variation of 5 ms (maximum variation from the

                                                                            average). Group 3 and Group 4 received the

     Client 2
                                                                            player#2 profile (see Table 1) with 2ms of delay
                                                                            variation. The network topology is presented in
                                                                            Figure 2.
                                                                                SCENARIO 3: Analyzing packet loss
    Client 25
                                                                                At this moment, all links were configured
                                                                            not to have packet delay during transmission.
                   Figure 2: WAN Scenarios                                  Groups 1 and 3 received player#1 profile (5%
                                                                            packet loss) and 0.5 of loss correlation (how
    In order to emulate users with different                                bursty is the loss). Groups 2 and 4 were
network     characteristics,  NIST.Net     was                              configured with player#2 profile (0.5% packet
configured to have 4 different kinds of players                             loss) and 0.5 of loss correlation. The network
according to Table 1:                                                       topology is presented in Figure 2.
                Table 1. Players’ network profile                               SCENARIO 4: Combining delay and
                                                                            packet loss
                                     Delay           Packet loss
                                (each direction)   (each direction)             Table 2 presents the 4 distinct
                                                                            configurations done for this scenario. Each
  Player #1                          60ms                       5%          group was configured according to Table 1. All
  Player #2                          20ms                       0.5%        groups had 0.5 loss correlation. The network
  Player #3                          60ms                       0.5%        topology is presented in Figure 2.
  Player #4                          20ms                       5%             Table 2. Groups’ network profile for Scenario 4.
                                                                                      Packet delay   Delay variation    Packet loss
    We built 4 scenarios to evaluate the tests                                           (ms)             (ms)             (%)
and all the players were divided in 4 groups. The                           Group 1        20              ±2              0.5
network topologies (as physically implemented)
are shown in Figure 1 (scenario 1) and Figure 2                             Group 2        60              ±5              0.5
(scenarios 2, 3 and 4). Delay measurement was                               Group 3        20              ±2               5
possible because the same machine (monitoring                               Group 4        60              ±5               5
machine) captured traffic on the two networks
(no need of clock synchronization).
                                                                                Experiments duration
    SCENARIO 1: Local network
                                                                            Each match was considered an experiment and
    The game was played in a local network.                                 was configured to last 30 minutes. On Counter-
NIST.Net was not used on this scenario and all                              Strike, each match is divided in 5-minute
players had the same network conditions.                                    rounds, so that when a player dies it returns on
Therefore, the players’ performance depended                                the next round. We collect the results for each
solely on their skills in the game. Delay and                               round and for the whole game.

    Anais do SBgames, outubro de 2004
    Analysis methodology
     The game allows the capture of traces of the                            Subjective analysis of game quality
entire match on a file for later viewing. These
files have been used for a statistical analysis of                                Objective analysis of game quality
players’ performance during the match.
     Basically, these files were used to count                                                Network quality
how many “people” the player killed and how
many times he was “killed” on the game. In
                                                                                     Game                          Game
Counter-Strike, opponent killing gives the                                  I/O      client       Transport        server
player points and money to buy better weapons.
Friendly fire is also possible, but it takes out                      Figure 3. Different analysis on a game [15].
points and money. With the information about a
player’s kills and deaths, we applied the                          The subjective analysis of our experiments
following performance index (I) equation:                      was made using a questionnaire where, for each
                                                               match, the players should inform how was its
     Number of times            Minimum                        perception of the events in the game and how
     player A killed            number of deaths               was the system response to its commands,
 I = during the match x 0,9 +   from all players x 0,1   (1)   assigning one of the following options: “very
     Maximum                    Number of times                bad”, “bad”, “reasonable”, “good” and “very
     number of kills            player A died                  good”. To calculate the average, each option
     from all players           during the match               was translated into a number ranging from 1 to
     This index will allow us to illustrate how                5.
network conditions (on this case, generated by                     The objective analysis was made collecting
NISt.Net) influence player’s performance during                network packets with the network analysis
the match.                                                     software TCPDump and using the performance
    There is a description of techniques used to               indexes obtained from the traces captured by the
analyze the quality of a game in [15]. These                   the game itself. The packet we collected
techniques are basically split in two categories:              confirmed the NIST.Net behavior as we
subjective analysis and objective analysis. The                configured it (about delay and discard packets).
subjective analysis consists of the players’                   Based on the philosophy of TCPStat [17], the
general perceptions of the game and can be                     IPStat3 tool was developed to improve the
strongly influenced by the input and output                    viewing of packet delay and loss statistics
devices (e.g.: monitor, keyboard and joystick),                through an analysis of packet traces generated
by the capacity of the machines (clients and                   by TCPDump. After being gathered by IPstat,
server), by the capacity of the software itself and            these data could be plotted in graphics.
by the support offered by the network
infrastructure. The objective analysis can be
done by the metrics captured by a piece of                          4 Tests Results
software in the behavior of the network, game                       All data were originated from the TCPDump
processing speed or I/O devices processing                     files and the statistical analyzer of Counter-
speed. Figure 3 illustrates the different types of             Strike. To be able to analyze subjectively and
game analysis.                                                 objectively the collected data, we used the
                                                               performance index (I) and the IPStat,

                                                                          The source code and more information about
                                                               the functionality of the IPStat tool can be obtained from
     Scenario 1 (see item 3.1) offers the best                                                 Groups 3 and 4. For the first groups, the average
conditions to play, i.e., users are playing on a                                               was 3.5, while for the latter groups the average
local network and packet delay and loss are                                                    was 3.91. The average for all groups was 3.7,
irrelevant. This scenario was important to know                                                which means a worse match in general, when
about the players’ abilities and to present them                                               comparing to the first scenario. Many players,
the rules of the experiments. Among the rules                                                  especially from groups 1 and 2 added comments
we have: the need to keep the same name and                                                    to the questionnaire saying that the game was
be on the same team – terrorist or counter-                                                    not responding very well to their commands.
terrorist – throughout the entire play (in                                                          It is important to notice that even though all
order not to confuse nor reset the metrics).                                                   players received some delay, those who received
     On the subjective analysis, the average                                                   a smaller delay didn’t notice much difference
value for Scenario 1 was 4.0 (range: 1.0 – 5.0),                                               because they had some “technical advantage”
which means that the perceptions of the players                                                over the ones with a much higher delay. This
to the events in the game were considered good.                                                helped them “kill” much more adversaries and
     On Scenario 2, the main goal was to analyze                                               “die” fewer times, so they considered the game
the effects of network delay on the game.                                                      “good”.
Therefore, 50% of the players had a 20ms delay                                                      Scenario 3 didn’t show the expected results.
while the others had a 60ms delay. The results                                                 The goal of Scenario 3 was to show how
generated for each group and the aggregated                                                    performance would degrade with packet losses,
result for same-condition groups are shown in                                                  but the difference from 0.5% to 5% in loss rates
Figure 4. We observed that the extra delay really                                              showed no influence. As for the analysis of the
affected the performance for players in groups 1                                               questionnaires for Scenario 3, players
and 2. The average performance index (I) for                                                   considered the match between “reasonable” and
players in groups 1 and 2 was 0.43, whereas for                                                “good”, with an average of 3.7 – same value
players in groups 3 and 4 it was 0.63.                                                         obtained in Scenario 2 – but here we noticed no
                                                                                               difference in the average obtained from each
                                                                                                    Scenario 4 was made from a mixture of
     packet delay (ms)

                                                                                               parameters used in scenarios 2 and 3, making 4

                                                                                               distinct groups of players. Once again, the losses
                                                                                               didn’t have any impact on performance and the
                                                                                               results were very similar to those from scenario
                         20                                                                    2. The 60ms delays had a very strong impact on
                                                                                               the performance, again. Figure 5 shows the
                              5   205   405   605    805   10 0 5   12 0 5   14 0 5   16 0 5
                                                                                               client to server delay for each group. The
                                                    time (s)                                   questionnaires showed an average grade of 3.9
                                              Groups 1 and 2                                   for this scenario, but there was a reasonable
                                              Groups 3 and 4                                   difference between players with and without
                                                                                               delay. Also, there were many comments about
     Figure 4. Client to server delay in Scenario 2.
                                                                                               the game not answering to player’s commands.

    On the questionnaires, Groups 1 and 2                                                                 Figure 5. Client to server delay in Scenario 4

considered the second scenario worse than
                                                                                                   packet delay (ms)


































                                                                                                                                                                                             time (s)
                                                                                                                                                                                           Groups 1 and 3
                                                                                                                                                                                           Groups 2 and 4
    Anais do SBgames, outubro de 2004
It is important to emphasize that in both
Scenarios 2 and 4 the graphics with delay data
confirmed that NIST.Net was working properly
to generate such delays.                                                                                   Delay from Client to Server

                         Comparison between average performances
    Performance index

                                                                                  delay (ms)

                        0,80                                                                    40

                                                               Scenario 1                       30

                                                               Scenario 2                       20

                                                               Scenario 3
                        0,00                                                                    10

                                                               Scenario 4
                                1      2       3      4                                          0

                                                                                                     Group 1
                                                                                                               Group 2
                                                                                                                         Group 3
                                                                                                                                    Group 4
                                                                                                                                               Group 1
                                                                                                                                                          Group 2
                                                                                                                                                                     Group 3
                                                                                                                                                                                Group 4
                                                                                                                                                                                           Group 1
                                                                                                                                                                                                      Group 2
                                                                                                                                                                                                                 Group 3
                                                                                                                                                                                                                            Group 4
                                                                                                                                                                                                                                       Group 1
                                                                                                                                                                                                                                                  Group 2
                                                                                                                                                                                                                                                             Group 3
                                                                                                                                                                                                                                                                        Group 4
                                     Player groups

                        Figure 5. Performance indexes comparison                                        Scenario 1                                Scenario 2                                  Scenario 3                                  Scenario 4

    Figure 6 presents a comparison of
                                                                                                Figure 6. Comparison of group delays
performance between the average performances
of each group in each scenario. Since players on
the first group did well in the LAN match (equal
conditions to all). We can suppose that the “best                                                                Jitter from Client to Server
players” of this experiment are on this group.                                                   8

    An average delay comparison for each
group in each scenario is presented in Figure 7.

On Scenario 2, players from the first and second
                                                                                  Jitter (ms)

group had a higher delay and their index was

much lower than the other groups, even though

they were better players (conclusions from                                                       3

Scenario 1). On Scenario 3 we could not                                                          2

conclude anything, since players from Groups 1                                                   1

and 2 (different packet loss rates) had very                                                     0
                                                                                                     Group 1
                                                                                                               Group 2
                                                                                                                          Group 3
                                                                                                                                     Group 4
                                                                                                                                                Group 1
                                                                                                                                                           Group 2
                                                                                                                                                                      Group 3
                                                                                                                                                                                 Group 4
                                                                                                                                                                                            Group 1
                                                                                                                                                                                                       Group 2
                                                                                                                                                                                                                  Group 3
                                                                                                                                                                                                                             Group 4
                                                                                                                                                                                                                                        Group 1
                                                                                                                                                                                                                                                   Group 2
                                                                                                                                                                                                                                                              Group 3
                                                                                                                                                                                                                                                                          Group 4
similar results, as well as players from groups 3
and 4. The fourth scenario, with each group
receiving a different network condition, allowed                                                         Scenario 1                                Scenario 2                                   Scenario 3                                  Scenario 4

us to verify that packet delay is the big problem
in action games.                                                                                 Figure 7. Comparison of group jitter
Figure 6 and Figure 7 show that, even though                                    Clearly, small variations in packet delays
the delay configured in NIST.Net was very well                              affect reasonably the behavior of Conter-Strike
implemented on the network (in terms of                                     from the user perspective. The same could not
average delay), the jitter was not very well                                be concluded for packet drops, which we justify
controlled. The difference did not seem to affect                           with the techniques presented in Section 2.
considerably the results, but it probably it would
have affect if the delays configured were not
very different.                                                                  5 Conclusions and future work
                                                                                 Real-time applications like action games
                                                                            must have rigid QoS requirements to work
                                                                            efficiently in a network like the Internet. On this
                                                                            paper we studied traffic generated by the action
                                                                            game Counter-Strike and did some qualitative
and quantitative analysis on the performance of            of Empires and Beyond”, Gamasutra: The Art &
players over different network conditions.                 Science of Making Games.
Therefore, we could analyze how network                [3] Cheshire, S. (1996) “It’s the Latency Stupid!”,
conditions contribute or hinders gameplay        
during a match.                                            ncy.html (2/8/2004).
     We observed that small packet drops do not        [4] Cheshire, S. (1996) “Latency and The Quest for
affect gameplay noticeably. The results of a tests         Interactivity”,
which analyzed the effects of discarding up to   
5% of the packets did not show any difference in           ml.
player’s performance,. It has shown that a 5%
packet loss is irrelevant for this game. Other         [5] Counter-Strike (2002) “The official Counter-
tests should be run to analyze greater loss rates,         Strike web site”,
where the idea is to find a threshold below                (2/8/2004).
which losses are acceptable and above which            [6] Diot, C. and Gautier, L. (1999) “A Distributed
losses are unacceptable for game play.                     Architecture    for     Multiplayer Interactive
     As for the delay, it became clear that it             Applications on the Internet”, IEEE Network,
impacts a lot on the playability of Counter-               vol. 13, number 4, p. 6-15.
Strike, confirming the diagnostics of [3], [4].        [7] Färber, J. (2002) “Network Game Traffic
Both subjective and objective analyses indicate            Modelling”, ACM NetGames, Braunschweig,
that the delay strongly affected even the best             Germany.
                                                       [8] Gautier, L. and Diot, C. (1998) “Design and
     It was concluded that different network               Evaluation of MiMaze, a Multi-Player Game on
conditions could affect game perceptions                   the Internet”, IEEE Multimedia Systems
amongst players. For instance, suppose two                 Conference, Austin.
players that use equivalent access networks, but
one of them has a longer delay to reach the            [9] InternetGames (2002) “Planetside: Massive
server than the other. The latter player will              Multiplayer                             Action”,
probably have a better game perception despite   
the fact that both players do not have a suitable          020301a.htm (9/2/2002).
network environment.                                   [10] Lincroft, P. (1999) “The Internet Sucks: Or,
     For future work, we found that a deeper                What I Learned Coding X-Wing vs. TIE
analysis of acceptable thresholds for delay and             Fighter”, Game Developer’s Conference.
loss would be very interesting. It is believed that    [11] Lowe, N. and Strauss, J. (2000) “Realtime 3D
evaluating the performance of players with                  Multiplayer Internet Gaming”, University of
different machines (slower and faster machines              Western Australia, Nedlands.
and with different types of devices) on the same
                                                       [12] Ng, Yu-Shen (1997) “Designing Fast-Action
match further contributes to complete the
                                                            Games”, Gamasutra: Features.
analysis of the Counter-Strike game network
requirements.                                          [13] NIST Net (2002) “NIST Net home page”,
    6 References                                       [14] Pantel, L. and Wolf, C. L. (2002) “On the Impact
                                                            of Delay on Real-Time Multiplayer Games”,
   [1] Bernier, W. Y. (2001) “Latency Compensating          International Workshop on Network and
       Methods in Client/Server In-Game Protocol            Operating System Support for Digital Audio and
       Design”, Game Developer’s Conference, San            Video: Network Issues for Video and Games,
       Jose, California, USA.                               New York, NY, USA, ACM Press, p. 23-29.
   [2] Bettner, P. and Terrano, M. (2001) “1500        [15] Schaefer, C.; Enderes, T.; Ritter, H. and
       Archers on a 28.8: Network Programming in Age        Zitterbart, M. (2002) “Subjective Quality
                                                            Assessment for Multiplayer Real-Time Games”,

    Anais do SBgames, outubro de 2004
    ACM NetGames, Braunschweig, Germany, p.
[16] TCPDump, “TCPDUMP public repository”, (2/8/2004).
[17] TCPStat,      “tcpstat        home         page”, (2/8/2004).

To top