Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

Scalable Peer-to-Peer Networked Virtual Environment

VIEWS: 17 PAGES: 56

									Scalable Peer-to-Peer Networked Virtual Environment
Master Thesis Oral Examination
Dept. of CSIE, Tamkang Univ. Advisor: Dr. Chen Jui-Fa
Shun-Yun Hu
2005/01/07

1

Outline
    

Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

2

What is Networked Virtual Environment (NVE)?


Virtual Reality + Internet 3D environment with people (avatar), objects, terrain, agents Military simulations (’80)  Massively Multiplayer Online Games (mid-‘90) Trends: larger scale, more realistic simulation
3







4

NVE: A Shared Space

5

Issues for Creating NVE
  

Consistency (events/states) Responsiveness Security Scalability Persistency Reliability (Fault-tolerance)

multiplayer

  

massively multiplayer

6

The Scalability Problem
  

Many nodes on a 2D plane ( > 1,000) Message exchange with those within Area of Interest (AOI) How does each node receive the relevant messages?

Area of Interest

7

A simple solution (point-to-point)

Source: [Funkhouser95]

N * (N-1) connections ≈ O(N2)  Not scalable!
8

A better solution (client-server)

Source: [Funkhouser95]

Message filtering at server to reduce traffic N connections = O(N)  server is bottleneck
9

Current solution (server-cluster)

Source: [Funkhouser95]

Still limited by servers. Expansive to deploy & maintain.
10

Scalability Analysis


Scalability constrains
 

Computing resource Network resource

(CPU) (Bandwidth)

Non-scalable system

vs.

Scalable system

Resource limit

x: number of entities y: resource consumption at the limiting system component
11

What Next?


Strategies
 

Increase resource Decrease consumption

 

More servers Message filtering



Architectures
   

Scale
tens hundreds thousands millions 10^1 10^2 10^3 10^6 …

Point-to-point (LAN) Client-server Server-cluster ? Peer-to-Peer

12

What is Peer-to-Peer (P2P)?
[Stoica et al. 2003]  Distributed systems without any centralized control or hierarchical organization  Runs software with equivalent functionality



Examples
  

File-sharing: Napster, Gnutella, eDonkey Distributed computing: SETI@Home (UC Berkeley) VoIP: Skype
13

Peer-to-Peer Overlay

A P2P overlay network

source: [Keller & Simon 2003]
14

Promise & Challenge of P2P


Promises




Growing resource, decentralized  Scalable Commodity hardware  Affordable



Challenges



Topology maintenance  dynamic join/leave Efficient content retrieval  no global knowledge

15

Issues for Creating P2P NVE
  

Consistency (events/states) Responsiveness Security Scalability Persistency Reliability (Fault-tolerance) Consistency (topology)

multiplayer

  

massively multiplayer



 P2P NVE
16

Related Works (1): SimMUD
[Knutsson et al. 2004] (Univ. of Pennsylvania)
  

Pastry + Scribe Regions Coordinators (super-nodes)




Fixed-size region Relay overhead
17

Related Works (2)
[Kawahara et al. 2004] (Univ. of Tokyo)
  

Fully-distributed Nearest-neighbors List exchange

 

High transmission Overlay partition
18

Related Works (3): Solipsis
[Keller & Simon 2003] (France Telecomm R&D)
  

Links with AOI neighbor Mutual cooperation Inside convex hull

 

Potentially slow discovery Inconsistent topology
19

Outline
    

Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

20

Design Goals


Observation:




for virtual environment applications, the contents we want are messages from AOI neighbors Content discovery is a neighbor discovery problem



Solve the Neighbor Discovery Problem in a fullydistributed, message-efficient manner. Specific goals:
 



Scalable Responsive

 Limit & minimize message traffics  Direct connection with AOI neighbors
21

Voronoi Diagram




2D Plane partitioned into regions by sites, each region contains all the points closest to its site Can be used to find k-nearest neighbor easily
Neighbors

Region Site
22

Design Concepts
Use Voronoi to solve the neighbor discovery problem
  



Identify enclosing and boundary neighbors Each node constructs a Voronoi of its neighbors Enclosing neighbors are minimally maintained Mutual collaboration in neighbor discovery
Circle White Yellow L. Blue Pink Green Area of Interest (AOI) self enclosing neighbor (E.N.) boundary neighbor (B.N.) E.N. & B.N. AOI neighbor
23

D. Blue unknown neighbor

Procedure (JOIN)
1) Joining node sends coordinates to any existing node Join request is forwarded to acceptor 2) Acceptor sends back its own neighbor list joining node connects with other nodes on the list

Joining node Acceptor’s region
24

Procedure (MOVE)
1) Positions sent to all neighbors, mark messages to B.N. B.N. checks for overlaps between mover’s AOI and its E.N. 2) Connect to new nodes upon notification by B.N. Disconnect any non-overlapped neighbor
Boundary neighbors Non-overlapped neighbors

New neighbors
25

Procedure (LEAVE)
1) Simply disconnect 2) Others then update their Voronoi new B.N. is discovered via existing B.N.

Leaving node (also a B.N.)

New boundary neighbor
26

Dynamic AOI
Crowding within AOI can overload a particular node

It’s better if AOI-radius can be adjusted in real time
27

Adjustment Conditions


AOI-radius decrease


Number of connections > maximum allowable connections



AOI-radius increase
 

Maximum connections not exceeded Current AOI-radius < preferred AOI-radius



Delay counter


To avoid fluctuations
28

Demonstration
Simulation video
  

General movements (20 nodes, 800x600 world) Local vs. global view Dynamic AOI adjustment

29

Outline
    

Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

31

Simulation Method


C++ implementation of Voronoi-based algorithm World size: 1000 x 1000, AOI: 150 Trials from 10 – 250 nodes Connection limit per node: 10 1000 time-steps
(~ 100 simulated seconds, assuming 10 updates/seconds)




 



Behavior model
  

Random movement: Constant velocity: Movement duration:

random direction 5 units/step random (1 – 25 steps)
32

Consistency Metrics


Topology Consistency [Kawahara, 2004]
Number of observed AOI neighbors Number of actual AOI neighbors



Drift Distance [Diot, 1999]
Distance between observed position and actual position (average over all nodes)

33

Basic Model

Topology Consistency
Topology Consistency Measurements 100 99 98 97 96 95 94 93 92 91 90 0 50 100 150 200 250 Number of Nodes
34

Topology Consistency (%)

Basic Model

Scalability (1)
Transmission Size Per Node Per Second 6.0 5.0
send (max)

Size (kb)

4.0 3.0 2.0 1.0 0.0 0 25 50 75 100 125 150 175 200 225 250 Number of Nodes

send (avg) recv (max) recv (avg)

35

Basic Model

Scalability (2)
Average Neighbor Size Measurements 18 16 14

Neighbor Size

12 10 8 6 4 2 0 0 50 100 150 200 250

connected AOI

Number of Nodes
36

Dynamic AOI Model
Effect of Node Increase on Average AOI-radius 160 140

Avg. AOI Radius

120 100 80 60 40 20 0 0 50 100 150 200 250
37

Number of Nodes

Dynamic AOI

Scalability (1)
Average Neighbor Size Measurements 10 9 8
Neighbor Size

7 6 5 4 3 2 1 0 0 50 100 150 200 250
38

connected AOI

Number of Nodes

Dynamic AOI

Scalability (2)
Transmission Size Per Node Per Second 4.0 3.5 3.0
send (max) send (avg) recv (max) recv (avg)

Size (kb)

2.5 2.0 1.5 1.0 0.5 0.0 0 25 50 75 100 125 150 175 200 225 250 Number of Nodes

39

Dynamic AOI

Scalability (3)
Comparison of Voronoi-based P2P and Client-Server 180 160 140
send (avg) recv (avg) CS-send (avg) CS-recv (avg)

Size (kb)

120 100 80 60 40 20 0 0 25 50 75 100 125 150 175 200 225 250 Number of Nodes

40

Dynamic AOI

Topology Consistency (1)
Topology Consistency Measurements
100

Topology Consistency (%)

99 98 97 96 95 94 93 92 91 90 10 30 50 70 90 110 130 150 170 190 210 230 250 Number of Nodes
41

Dynamic AOI

Topology Consistency (2)
Drift Distance Measurements 100 80

Drift Distance

60 40 20 0 0 50 100 150 200 250 Number of Nodes

max avg

42

Dynamic AOI

Reliability (1)
Change in Consistency of over 1000 time-steps (step 150 - 250) 100

Topology Consistency (%)

90

per step node 1 node 91

80

70 150 160 170 180 190 200 time-step
43

210

220

230

240

250

Dynamic AOI

Reliability (2)
Average Number of Steps to Recover from Inconsistency 4

3

Steps

2

1

0 0 50 100 150 200 250
44

Number of Nodes

Outline
 


 

Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

45

Analysis of Design
Consistency (Topology)


Topology is fully connected & consistent  enclosing neighbors

Responsiveness


Lowest latency  direct connection, no relay

Scalability


Resource-growing & decentralized resource consumption

Reliability


Self-organizing for small number of node failures
46

P2P NVE Comparisons
SimMUD Consistency (topology) Neighbor-list Solipsis exchange VON

Supernode Neighbor listexchange (partitioning)
High overhead

Neighbor Neighbor notify&query notify (undiscovery) (consistent)
Medium overhead Fullydistributed N/A Low overhead Fullydistributed Selforganizing
47

Responsive- High ness overhead Scalability Reliability

Relied on Fullysupernode distributed Long uptime N/A

Problems of Voronoi Approach


Message traffic
 

Circular round-up of nodes Redundant message sending
(inherent to fully-distributed design)



Incomplete neighbor discovery
 

Can happen with inconsistent / incorrect neighbor list Fast moving node

48

Outline
    

Introduction Voronoi-based Overlay Network (VON) Simulation Results Analysis Conclusion

49

Conclusion


NVE scalability is achievable with P2P architecture and is a neighbor discovery problem A promising solution: Voronoi-based P2P Overlay




Leverage knowledge of each peer to maintain topology



Properties
  

Scalable: fully-distributed, dynamic AOI Efficient: low irrelevant messages, zero relay Robust: consistent and self-organizing topology
50

Potential Applications


Online games
Relieve server from position updates in current MMOGs



Military
Enable large-scale, affordable military training simulation



3D Web
Provide multi-user interactivity to static 3D world



Scientific simulations
Distribute spatial simulation requiring frequent synchronization
51

Future Works


Short-term
 



Reliability measurements  latency, packet loss, node fail Distributed event/state consistency Recovery from overlay partition



Long-term





Persistency issue (P2P-based database) Security issue (protection from malicious nodes) 3D content distribution (3D streaming on P2P)

Massive, persistent 3D environment sharable by all!
52

Acknowledgements
          

Dr. Jui-Fa Chen (陳瑞發老師) Dr. Wei-Chuan Lin (林偉川老師) Members of the Alpha Lab, TKU CS Guan-Ming Liao (廖冠名) Dr. Chin-Kun Hu (胡進錕老師) LSCP, Institute of Physics, Academia Sinica

Joaquin Keller Bart Whitebook Jon Watte

(France Telecomm R&D, Solipsis) (butterfly.net) (there.com)
(洪文斌老師) (黃俊堯老師)
53

Dr. Wen-Bing Horng Dr. Jiung-yao Huang

Inconsistency caused by dAOI

54

Reliability (0-500 steps)
Change in Consistency of over 1000 time-steps (step 1 - 500) 100

Topology Consistency (%)

90

per step node 1 node 91

80

70 0 100 200 time-step
55

300

400

500

Reliability (501-1000 steps)
Change in Consistency of over 1000 time-steps (step 501 - 1000) 100

Topology Consistency (%)

90

per step node 1 node 91

80

70 500 600 700 time-step 800 900 1000
56


								
To top