Get documents about "Counter - Strike Traffic Analysis with Network Emulation
Shared by: wuzhenguang
-
Stats
- views:
- 0
- posted:
- 10/2/2012
- language:
- English
- pages:
- 9
Document Sample
Counter-Strike Traffic Analysis with Network Emulation
ARTHUR DE CASTRO CALLADO1
SUZANA DE FRANÇA DANTAS1
RODRIGO DOS SANTOS BACELAR GOUVEIA BARBOSA1
PAULO GONÇALVES DE BARROS1
VERONICA TEICHRIEB1
JUDITH KELNER1
DJAMEL FAWZI HADJ SADOK 1
1
Centro de Informática – Universidade Federal de Pernambuco (UFPE)
Caixa Postal 7851, CEP 50732-970, Cidade Universitária, Recife – PE – Brazil
{acc2, sfd, rsbgb, pgb, vt, jk, jamel}@cin.ufpe.br
__________________________________________________________________________________________
Abstract
Network action games face a number of problems which limit their efficiency. It is
important to ensure that all players are at equal levels during a session, which is
highly dependent on the information integrity available to a user. This work
analyses problems such as the impact of delay and packet loss on the playability of
the Counter-Strike game. An effort is made to relate such parameters to other
qualitative parameters obtained from the analysis of user game perception.
Keywords: Models and Infrastructure for network games, Playability, Usability.
__________________________________________________________________________________________
1 Introduction games are based on real-time actions, where
Guaranteeing the well functioning of advanced players interact with each other and with the
applications, like action games and virtual environment simultaneously. In this case, there
reality environments over infrastructures as are many obstacles in the development of a
unreliable as the Internet is still an open problem game, for one must guarantee that players
and a challenge. Many research groups are perceive events at the right time and that such
involved on the arduous job of identifying the guarantee is restricted by the capacity of the
variables and resolve the problems that affect network connecting the participants’ machines.
these applications. So far, Quality of Service (QoS) metrics are not
completely defined to comply with players’
Networked games are the latest sensation in needs, i.e. which network parameters should be
the world of electronic games. They allow managed and how they could be controlled to
matches with dozens or sometimes hundreds of make the game fair to all players. These are are
people interacting with each other on the same
still open problems.
virtual environment. Nowadays, big companies
will not put money on the development of a Many papers that present typical problems
game if it doesn’t have on-line multiplayer on networked games performance have been
characteristics; because these features can both published and among them are [12], [1].
increase the number of consumers the game is However, studies presenting quantitative and
targeted to as well as enlarge its life period. qualitative analysis are still to be seen. We made
many tests using the action game Counter-Strike
The access to the game environment in
multiplayer games can be based on turns, where (CS) [5], one of the preferred games among
those who like to improve their terrorist or
each player can take actions to the game at a
time, like in chess. However, most interesting counter-terrorists tactics. CS is one of the most
played First Person Shooter (FPS) games
worldwide. The virtual struggle between same time. This is generally solved with a
terrorists and counter-terrorists has gathered signaling packet sent to each player at a
thousands of players for the last 4 years. different time, with a delay inversely
In this project, we tried to make both a proportional to the players’ connection delay
quantitative and a qualitative analysis of the [14].
behavior of the game. The qualitative analysis The end of the match synchronism makes
was based on a questionnaire answered by the final state of the match consistent to all
players at the end of each match and on statistics players. So, each client machine can determine,
of performance for each player, according to the for example, who won the match [2], [14].
score and the number of deaths. The quantitative The actions of a player must be reflected in
analysis was based on packet traces gathered the world1, so they can be seen by all players (all
during the matches, with all the generated players must perceive the actions in a consistent
traffic. way). To make it possible, games use the lag
We noticed that problems like packet loss compensation technique in centralized
and delay do influence the players’ performance architectures, which consists of updating the
throughout the match. Network bandwidth is state of the world in the server according to the
another parameter that can also be considered in states of the clients. Still, this technique creates
the analysis, but it is not being treated here. some problems like, for example, when there is
In [7] the Counter-Strike traffic was inconsistency of the state of a client, it can be
modeled for local networks. This paper intends seen and shot through a wall [2], [14], [8].
to further characterize Counter-Strike behavior Collision detection is also another problem
in WAN networks using network emulation. It and should be consistent to all players. The
will also evaluate user’s satisfaction among each collision of a car with another car should be
other on different network conditions during a noticed by the car that hit the other as well as by
game match. the car that was hit and should also be seen by
Section 2 of this paper presents the major the cars that were passing by, all at the same
problems observed on action games in order to time. It is fundamental that this information has
work satisfactorily on such an heterogeneous a high propagation priority [14].
network like the Internet. Section 3 describes the Another relevant aspect of applications like
methodology used to accomplish the tests and action games is the discarding of packet. Some
details the network scenarios created. The of the problems caused by packet loss are:
qualitative and quantitative results are warping, lead targeting and dead reckoning.
commented in Section 4. Section 5 draws some Warping is characterized by the loss of
conclusions and shows some suggestions for packets referring to the position of a player
future works. during the match. Its movement will not be
continuos, making other players see the player
2 Networked games vs. Internet: typical as if “teleporting” himself from one position to
problems another in the scenario. This occurs due to the
Networked games require important network unpredictability of the movement for long
resources to work efficiently. Synchronism is a periods of time by the interpolation algorithms
critical aspect and it consists of making all used [11], [2], [10].
players notice an event at the same time. Lead targeting is the “technique” used by
Synchronism is important in many moments of a some players to aim and hit their opponents in
game and can be categorized according to the First Person Shooter games when they are not
problem it solves: start of the match, end of the accessible. This happens because the state of the
match, user actions, collisions, sound of events
and random elements.
The start of the match synchronism is to 1
“World” refers to the virtual environment where
make every player see the match starting at the the match takes place.
Anais do SBgames, outubro de 2004
world presented to the player is delayed Once it was needed to increase the number
according to its network connection. This way, of clients and to have an isolated network2, a
the player does not see what’s happening to the Lan House was contacted and their
world at that moment, but what happened a little infrastructure was used. This choice solved two
time before. The enemy is not where the player problems: traffic isolation and the participation
sees it, but a little dislocated from that position of specialized game players, with excellent
[11]. knowledge of its rules and tactics.
Dead reckoning comes from the necessity of
establishing a common perception of major Scenarios description
information in the game, such as start time, end The test environment was constituted of 25
time, winner(s), who died, etc. [12]. personal computers (PCs) acting as client
When we consider games with a very large machines, 1 PC acting as game server, 2
number of players, network bandwidth becomes Ethernet hubs operating at 100Mbps, 1 PC
an important issue. The greater the number of running NIST.Net [13] and another PC runs the
players, more information should be processed TCPDump network sniffer [16], as shown in
and sent [9], [6], [8], increasing network load. Figures 1 and 2.
However, depending on the architecture adopted
for the game, this load can be greatly reduced.
Distributed games require more bandwidth. Scenario 1: LAN
Centralized (client/server) architectures, like the
one used in Counter-Strike, concentrate their Client 1
load on a single machine [11], [8]. On that case,
the server machine will need a network link with
greater bandwidth to work properly. If it is not
Ethernet
Server
possible, that link will act as a bottleneck,
decreasing game performance to players [11], Client 2
[10].
Even though Counter-Strike uses all these
techniques, they merely minimize network Monitoring
problems, not solving them. Therefore, it is
extremely necessary to evaluate the diversity of
network conditions in order to improve user’s
playability with Counter-Strike. This can be Client 25
done by identifying thresholds for acceptable Figure 1: LAN scenario
values of network parameters.
NIST.Net is a network emulation software
3 Testing environment and methodology that allowed us to configure network links with
bandwidth, delay and loss characteristics. We
In this section we describe the environment could emulate different kinds of connection for
where tests have taken place and the each client by configuring the NIST.Net
methodology used to accomplish them. We also machine as a router and putting all clients in one
describe the criteria adopted to do the objective network (connected by 1 hub) and the server in
and subjective analysis. another network (using another hub). The
Before the experiments presented on this Monitoring machine, connected to both
paper take place, other minor scale tests (4 networks, allowed us to capture all traffic from
clients, for example) have been made on the
laboratories of CIn/UFPE. Those tests helped to
devise what types of scenarios were interesting
to be evaluated, their parameters and also the 2
All CIn/UFPE laboratories are integrated, so it
metrics that should be used to analyze them. was not possible to isolate a special traffic.
both sides and comparatively analyze it to check packet loss in local networks were considered
packet delay and loss. insignificant.
SCENARIO 2: Analyzing packet delay
impact
Scenarios 2 to 4: WAN
On this scenario, all links were configured
Client 1
not to have packet loss during transmissions.
Two groups (Group 1 and Group 2) received
player#1 profile (see Table 1) with delay
NIST.Net variation of 5 ms (maximum variation from the
Ethernet
average). Group 3 and Group 4 received the
Ethernet
Client 2
Server
player#2 profile (see Table 1) with 2ms of delay
variation. The network topology is presented in
Figure 2.
SCENARIO 3: Analyzing packet loss
Monitoring
impact
Client 25
At this moment, all links were configured
not to have packet delay during transmission.
Figure 2: WAN Scenarios Groups 1 and 3 received player#1 profile (5%
packet loss) and 0.5 of loss correlation (how
In order to emulate users with different bursty is the loss). Groups 2 and 4 were
network characteristics, NIST.Net was configured with player#2 profile (0.5% packet
configured to have 4 different kinds of players loss) and 0.5 of loss correlation. The network
according to Table 1: topology is presented in Figure 2.
Table 1. Players’ network profile SCENARIO 4: Combining delay and
packet loss
Delay Packet loss
(each direction) (each direction) Table 2 presents the 4 distinct
configurations done for this scenario. Each
Player #1 60ms 5% group was configured according to Table 1. All
Player #2 20ms 0.5% groups had 0.5 loss correlation. The network
Player #3 60ms 0.5% topology is presented in Figure 2.
Player #4 20ms 5% Table 2. Groups’ network profile for Scenario 4.
Packet delay Delay variation Packet loss
We built 4 scenarios to evaluate the tests (ms) (ms) (%)
and all the players were divided in 4 groups. The Group 1 20 ±2 0.5
network topologies (as physically implemented)
are shown in Figure 1 (scenario 1) and Figure 2 Group 2 60 ±5 0.5
(scenarios 2, 3 and 4). Delay measurement was Group 3 20 ±2 5
possible because the same machine (monitoring Group 4 60 ±5 5
machine) captured traffic on the two networks
(no need of clock synchronization).
Experiments duration
SCENARIO 1: Local network
Each match was considered an experiment and
The game was played in a local network. was configured to last 30 minutes. On Counter-
NIST.Net was not used on this scenario and all Strike, each match is divided in 5-minute
players had the same network conditions. rounds, so that when a player dies it returns on
Therefore, the players’ performance depended the next round. We collect the results for each
solely on their skills in the game. Delay and round and for the whole game.
Anais do SBgames, outubro de 2004
Analysis methodology
The game allows the capture of traces of the Subjective analysis of game quality
entire match on a file for later viewing. These
files have been used for a statistical analysis of Objective analysis of game quality
players’ performance during the match.
Basically, these files were used to count Network quality
how many “people” the player killed and how
many times he was “killed” on the game. In
Game Game
Counter-Strike, opponent killing gives the I/O client Transport server
player points and money to buy better weapons.
Friendly fire is also possible, but it takes out Figure 3. Different analysis on a game [15].
points and money. With the information about a
player’s kills and deaths, we applied the The subjective analysis of our experiments
following performance index (I) equation: was made using a questionnaire where, for each
match, the players should inform how was its
Number of times Minimum perception of the events in the game and how
player A killed number of deaths was the system response to its commands,
I = during the match x 0,9 + from all players x 0,1 (1) assigning one of the following options: “very
Maximum Number of times bad”, “bad”, “reasonable”, “good” and “very
number of kills player A died good”. To calculate the average, each option
from all players during the match was translated into a number ranging from 1 to
This index will allow us to illustrate how 5.
network conditions (on this case, generated by The objective analysis was made collecting
NISt.Net) influence player’s performance during network packets with the network analysis
the match. software TCPDump and using the performance
There is a description of techniques used to indexes obtained from the traces captured by the
analyze the quality of a game in [15]. These the game itself. The packet we collected
techniques are basically split in two categories: confirmed the NIST.Net behavior as we
subjective analysis and objective analysis. The configured it (about delay and discard packets).
subjective analysis consists of the players’ Based on the philosophy of TCPStat [17], the
general perceptions of the game and can be IPStat3 tool was developed to improve the
strongly influenced by the input and output viewing of packet delay and loss statistics
devices (e.g.: monitor, keyboard and joystick), through an analysis of packet traces generated
by the capacity of the machines (clients and by TCPDump. After being gathered by IPstat,
server), by the capacity of the software itself and these data could be plotted in graphics.
by the support offered by the network
infrastructure. The objective analysis can be
done by the metrics captured by a piece of 4 Tests Results
software in the behavior of the network, game All data were originated from the TCPDump
processing speed or I/O devices processing files and the statistical analyzer of Counter-
speed. Figure 3 illustrates the different types of Strike. To be able to analyze subjectively and
game analysis. objectively the collected data, we used the
performance index (I) and the IPStat,
respectively.
3
The source code and more information about
the functionality of the IPStat tool can be obtained from
www.cin.ufpe.br/~rsbgb/ipstat.
Scenario 1 (see item 3.1) offers the best Groups 3 and 4. For the first groups, the average
conditions to play, i.e., users are playing on a was 3.5, while for the latter groups the average
local network and packet delay and loss are was 3.91. The average for all groups was 3.7,
irrelevant. This scenario was important to know which means a worse match in general, when
about the players’ abilities and to present them comparing to the first scenario. Many players,
the rules of the experiments. Among the rules especially from groups 1 and 2 added comments
we have: the need to keep the same name and to the questionnaire saying that the game was
be on the same team – terrorist or counter- not responding very well to their commands.
terrorist – throughout the entire play (in It is important to notice that even though all
order not to confuse nor reset the metrics). players received some delay, those who received
On the subjective analysis, the average a smaller delay didn’t notice much difference
value for Scenario 1 was 4.0 (range: 1.0 – 5.0), because they had some “technical advantage”
which means that the perceptions of the players over the ones with a much higher delay. This
to the events in the game were considered good. helped them “kill” much more adversaries and
On Scenario 2, the main goal was to analyze “die” fewer times, so they considered the game
the effects of network delay on the game. “good”.
Therefore, 50% of the players had a 20ms delay Scenario 3 didn’t show the expected results.
while the others had a 60ms delay. The results The goal of Scenario 3 was to show how
generated for each group and the aggregated performance would degrade with packet losses,
result for same-condition groups are shown in but the difference from 0.5% to 5% in loss rates
Figure 4. We observed that the extra delay really showed no influence. As for the analysis of the
affected the performance for players in groups 1 questionnaires for Scenario 3, players
and 2. The average performance index (I) for considered the match between “reasonable” and
players in groups 1 and 2 was 0.43, whereas for “good”, with an average of 3.7 – same value
players in groups 3 and 4 it was 0.63. obtained in Scenario 2 – but here we noticed no
difference in the average obtained from each
70
group.
Scenario 4 was made from a mixture of
packet delay (ms)
parameters used in scenarios 2 and 3, making 4
60
50
distinct groups of players. Once again, the losses
40
didn’t have any impact on performance and the
30
results were very similar to those from scenario
20 2. The 60ms delays had a very strong impact on
10
the performance, again. Figure 5 shows the
5 205 405 605 805 10 0 5 12 0 5 14 0 5 16 0 5
client to server delay for each group. The
time (s) questionnaires showed an average grade of 3.9
Groups 1 and 2 for this scenario, but there was a reasonable
Groups 3 and 4 difference between players with and without
delay. Also, there were many comments about
Figure 4. Client to server delay in Scenario 2.
the game not answering to player’s commands.
On the questionnaires, Groups 1 and 2 Figure 5. Client to server delay in Scenario 4
70
considered the second scenario worse than
packet delay (ms)
60
50
40
30
20
10
5
65
125
185
245
305
365
425
485
545
605
665
725
785
845
905
965
1025
1085
1145
1205
1265
1325
1385
1445
1505
1565
1625
time (s)
Groups 1 and 3
Groups 2 and 4
Anais do SBgames, outubro de 2004
It is important to emphasize that in both
Scenarios 2 and 4 the graphics with delay data
confirmed that NIST.Net was working properly
to generate such delays. Delay from Client to Server
70
60
Comparison between average performances
1,00
Performance index
delay (ms)
50
0,80 40
0,60
Scenario 1 30
0,40
Scenario 2 20
0,20
Scenario 3
0,00 10
Scenario 4
1 2 3 4 0
Group 1
Group 2
Group 3
Group 4
Group 1
Group 2
Group 3
Group 4
Group 1
Group 2
Group 3
Group 4
Group 1
Group 2
Group 3
Group 4
Player groups
Figure 5. Performance indexes comparison Scenario 1 Scenario 2 Scenario 3 Scenario 4
Figure 6 presents a comparison of
Figure 6. Comparison of group delays
performance between the average performances
of each group in each scenario. Since players on
the first group did well in the LAN match (equal
conditions to all). We can suppose that the “best Jitter from Client to Server
players” of this experiment are on this group. 8
An average delay comparison for each
group in each scenario is presented in Figure 7.
7
On Scenario 2, players from the first and second
6
Jitter (ms)
group had a higher delay and their index was
5
much lower than the other groups, even though
4
they were better players (conclusions from 3
Scenario 1). On Scenario 3 we could not 2
conclude anything, since players from Groups 1 1
and 2 (different packet loss rates) had very 0
Group 1
Group 2
Group 3
Group 4
Group 1
Group 2
Group 3
Group 4
Group 1
Group 2
Group 3
Group 4
Group 1
Group 2
Group 3
Group 4
similar results, as well as players from groups 3
and 4. The fourth scenario, with each group
receiving a different network condition, allowed Scenario 1 Scenario 2 Scenario 3 Scenario 4
us to verify that packet delay is the big problem
in action games. Figure 7. Comparison of group jitter
Figure 6 and Figure 7 show that, even though Clearly, small variations in packet delays
the delay configured in NIST.Net was very well affect reasonably the behavior of Conter-Strike
implemented on the network (in terms of from the user perspective. The same could not
average delay), the jitter was not very well be concluded for packet drops, which we justify
controlled. The difference did not seem to affect with the techniques presented in Section 2.
considerably the results, but it probably it would
have affect if the delays configured were not
very different. 5 Conclusions and future work
Real-time applications like action games
must have rigid QoS requirements to work
efficiently in a network like the Internet. On this
paper we studied traffic generated by the action
game Counter-Strike and did some qualitative
and quantitative analysis on the performance of of Empires and Beyond”, Gamasutra: The Art &
players over different network conditions. Science of Making Games.
Therefore, we could analyze how network [3] Cheshire, S. (1996) “It’s the Latency Stupid!”,
conditions contribute or hinders gameplay http://rescomp.stanford.edu/~cheshire/rants/Late
during a match. ncy.html (2/8/2004).
We observed that small packet drops do not [4] Cheshire, S. (1996) “Latency and The Quest for
affect gameplay noticeably. The results of a tests Interactivity”,
which analyzed the effects of discarding up to www.stuartcheshire.org/papers/LatencyQuest.ht
5% of the packets did not show any difference in ml.
player’s performance,. It has shown that a 5%
packet loss is irrelevant for this game. Other [5] Counter-Strike (2002) “The official Counter-
tests should be run to analyze greater loss rates, Strike web site”, http://www.counter-strike.net
where the idea is to find a threshold below (2/8/2004).
which losses are acceptable and above which [6] Diot, C. and Gautier, L. (1999) “A Distributed
losses are unacceptable for game play. Architecture for Multiplayer Interactive
As for the delay, it became clear that it Applications on the Internet”, IEEE Network,
impacts a lot on the playability of Counter- vol. 13, number 4, p. 6-15.
Strike, confirming the diagnostics of [3], [4]. [7] Färber, J. (2002) “Network Game Traffic
Both subjective and objective analyses indicate Modelling”, ACM NetGames, Braunschweig,
that the delay strongly affected even the best Germany.
players.
[8] Gautier, L. and Diot, C. (1998) “Design and
It was concluded that different network Evaluation of MiMaze, a Multi-Player Game on
conditions could affect game perceptions the Internet”, IEEE Multimedia Systems
amongst players. For instance, suppose two Conference, Austin.
players that use equivalent access networks, but
one of them has a longer delay to reach the [9] InternetGames (2002) “Planetside: Massive
server than the other. The latter player will Multiplayer Action”,
probably have a better game perception despite http://internetgames.about.com/library/weekly/aa
the fact that both players do not have a suitable 020301a.htm (9/2/2002).
network environment. [10] Lincroft, P. (1999) “The Internet Sucks: Or,
For future work, we found that a deeper What I Learned Coding X-Wing vs. TIE
analysis of acceptable thresholds for delay and Fighter”, Game Developer’s Conference.
loss would be very interesting. It is believed that [11] Lowe, N. and Strauss, J. (2000) “Realtime 3D
evaluating the performance of players with Multiplayer Internet Gaming”, University of
different machines (slower and faster machines Western Australia, Nedlands.
and with different types of devices) on the same
[12] Ng, Yu-Shen (1997) “Designing Fast-Action
match further contributes to complete the
Games”, Gamasutra: Features.
analysis of the Counter-Strike game network
requirements. [13] NIST Net (2002) “NIST Net home page”,
www.antd.nist.gov/nistnet/ (2/8/2004).
6 References [14] Pantel, L. and Wolf, C. L. (2002) “On the Impact
of Delay on Real-Time Multiplayer Games”,
[1] Bernier, W. Y. (2001) “Latency Compensating International Workshop on Network and
Methods in Client/Server In-Game Protocol Operating System Support for Digital Audio and
Design”, Game Developer’s Conference, San Video: Network Issues for Video and Games,
Jose, California, USA. New York, NY, USA, ACM Press, p. 23-29.
[2] Bettner, P. and Terrano, M. (2001) “1500 [15] Schaefer, C.; Enderes, T.; Ritter, H. and
Archers on a 28.8: Network Programming in Age Zitterbart, M. (2002) “Subjective Quality
Assessment for Multiplayer Real-Time Games”,
Anais do SBgames, outubro de 2004
ACM NetGames, Braunschweig, Germany, p.
74-78.
[16] TCPDump, “TCPDUMP public repository”,
http://www.tcpdump.org (2/8/2004).
[17] TCPStat, “tcpstat home page”,
www.frenchfries.net/paul/tcpstat/ (2/8/2004).
